Camera monitoring system for emergency response to abnormal condition of smart home

Abstract

The invention relates to the technical field of image processing, in particular to a camera monitoring system for emergency response to an abnormal condition of a smart home, and the system comprises a video collection module which obtains video frame data; the recognition and segmentation module is used for recognizing and segmenting a smoke target in the video frame data by using a semantic segmentation network; the analysis and calculation module is used for analyzing the particle texture irregularity of the smoke target to determine a structural quantity, and analyzing the dynamic characteristics of the smoke target based on the structural quantity to determine a flow diffusion coefficient; and the classification judgment module is used for determining classification parameters of the smoke target according to the structural quantity and the flow diffusion coefficient and judging the smoke type.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a video surveillance system for emergency response to abnormal situations in smart homes. Background Art

[0002] The home abnormal situation monitoring system can use video image processing technology to monitor and identify various abnormal situations in the user's home in real time. For example, fire or smoke detection, intrusion detection, abnormal behavior analysis, and sound abnormality. Fire or smoke detection has a higher priority in the early warning processing of home abnormal situations. Usually, in order to play the role of early warning detection of fire or smoke, image processing is often used to analyze the video monitoring results, and evaluation and judgment are made based on the monitoring results, which plays a high-efficiency and high-accuracy early warning effect.

[0003] In general household scenarios, the generation of smoke or flames is mainly related to factors such as improper use of electrical appliances. When distinguishing such harmful smoke from harmless smoke (such as smoke produced by humidifiers), image algorithm processing of video frames often fails to effectively distinguish them. This is because harmful smoke and harmless smoke (such as steam or steam during cooking) are visually very similar, both appearing as grayish-white diffuse shapes, making it very difficult to distinguish them based on color and shape alone. Steam from a humidifier or smoke during cooking may be misjudged as dangerous smoke, causing the system to issue unnecessary alarms; in some cases, real fire smoke may not be correctly identified by the system due to insufficient density or fusion with the background, posing a safety hazard or even causing serious consequences. Summary of the invention

[0004] In order to solve the technical problem of effectively identifying the type of smoke, the purpose of the present invention is to provide a video surveillance system for emergency response to abnormal situations in smart homes. The technical solution adopted is as follows:

[0005] The present invention provides a video surveillance system for emergency response to abnormal situations in smart homes, comprising: a video acquisition module for acquiring video frame data; an identification and segmentation module for identifying and segmenting smoke targets in the video frame data using a semantic segmentation network; an analysis and calculation module for analyzing the irregularity of particle texture of the smoke target to determine a structural quantity, and analyzing the dynamic characteristics of the smoke target based on the structural quantity to determine a flow diffusion coefficient; and a classification and determination module for determining classification parameters of the smoke target according to the structural quantity and the flow diffusion coefficient, and determining the type of smoke.

[0006] Furthermore, the system also includes: a preprocessing module, used to preprocess each frame image in the video frame data to form an enhanced image of each frame image, wherein the recognition and segmentation of the smoke target are performed based on the enhanced image, and the recognition and segmentation module is also used to mark independent areas in the enhanced image where the smoke target exists.

[0007] Furthermore, the determination of the structural quantity satisfies: Wherein, Q represents the structural quantity corresponding to the independent region, n represents the total number of sub-regions in the independent region, Con represents the overall gray level co-occurrence matrix contrast result value of the independent region, Con(k) represents the gray level co-occurrence matrix contrast value in the kth sub-region of the independent region, represents the overall gray level co-occurrence matrix contrast mean of the independent region, and Ent(k) represents the pixel value entropy value in the sub-region.

[0008] Furthermore, the analysis and calculation module is further used to analyze the moving direction and moving speed of the independent area in different frames to obtain a flow coefficient, wherein the flow coefficient satisfies: Where Li represents the flow coefficient of the independent region in different frames, Q(x,y) t represents the geometric center coordinates of the independent region in frame t; Q(x,y) t―1 Represents the coordinates of the geometric center point of the independent area in the t-1 frame.

[0009] Furthermore, the flow diffusion coefficient satisfies: Wherein, Ex represents the diffusion coefficient of the independent region, T represents the total number of different frames involved in the calculation, Q(t) represents the structural quantity corresponding to the independent region in the t frame, Q(t-1) represents the structural quantity corresponding to the independent region in the t-1 frame; and the classification parameters satisfy: Wherein, Cla(T) represents the flow diffusion coefficient of the independent region calculated under the total amount of T frames, and α represents the weight ratio of fluidity and diffusivity.

[0010] Furthermore, the determination result of the classification determination module includes harmful smoke and harmless smoke, wherein when Cla(T) is in [0,0.5], the determination result is harmful smoke, and when Cla(T) is in (0.5,1], the determination result is harmless smoke.

[0011] Furthermore, the semantic segmentation network uses a cross entropy function to predict the difference between the probability distribution and the true label.

[0012] Furthermore, the analysis and calculation module calculates the structural quantity change of the independent area between adjacent frames.

[0013] Furthermore, the system also includes: an alarm module, which is in communication with the classification determination module and triggers a corresponding alarm mechanism according to the smoke type.

[0014] Furthermore, the alarm module includes: an alarm triggering submodule, which is used to trigger an alarm when the determination result of the harmful smoke is obtained; a notification submodule, which is used to send and notify alarm information; and an emergency response procedure starting submodule, which is used to guide security personnel to implement on-site assessments and safety measures.

[0015] The present invention has the following beneficial effects:

[0016] Through machine vision image processing, abnormal situations in the home, such as fire and smoke, are identified, and smoke classification and analysis of whether to issue an alarm are performed based on the identification results.

[0017] Different types of smoke are identified through image processing, the particle texture of the target is determined, and the corresponding structural quantity is obtained. The acquisition of the structural quantity helps to preliminarily distinguish them. On this basis, further error elimination is carried out in combination with the structural quantity flow diffusion coefficient under different frames to obtain accurate smoke classification and identification results, which plays an effective early warning role, thereby avoiding misjudgment or missed judgment due to the high similarity between harmful smoke and harmless smoke. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A block diagram of a video surveillance system for emergency response to abnormal situations in a smart home provided by an embodiment of the present invention;

[0020] Figure 2 A block diagram of an alarm module of a video surveillance system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a video surveillance system for emergency response to abnormal situations in a smart home proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0022] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0023] A specific scheme of a video surveillance system for emergency response to abnormal situations in a smart home provided by the present invention is described in detail below in conjunction with the accompanying drawings.

[0024] See also Figure 1 , which shows a block diagram of a video surveillance system 100 provided by an embodiment of the present invention. The video surveillance system 100 includes a video acquisition module 110, a recognition and segmentation module 120, an analysis and calculation module 130, and a classification and determination module 140. The video acquisition module 140 is used to obtain video frame data, which may be composed of multiple frames of images in a time sequence. The recognition and segmentation module 120 uses a semantic segmentation network to identify and segment smoke targets in the video frame data. The analysis and calculation module 130 is used to analyze the irregularity of the particle texture of the smoke target to determine the structural quantity, and analyze the dynamic characteristics of the smoke target based on the structural quantity to determine the flow diffusion coefficient. The classification and determination module 140 determines the classification parameters of the smoke target according to the structural quantity and the flow diffusion coefficient, and determines the type of smoke.

[0025] In a possible implementation, the video acquisition module 110 can acquire video frame data in real time to achieve real-time monitoring of the environment. In order to meet the real-time requirements, the video acquisition module 110 can have a sufficiently high frame rate for continuous video capture, for example, its video frame rate is 30fps or 60fps. The video acquisition module 110 may also have network connection capabilities, support wired or wireless connections, and be able to transmit captured real-time video. As an example, the video acquisition module 110 includes one or a combination of a high-definition camera, an infrared camera, a panoramic camera, a motion detection camera, and a surveillance camera.

[0026] In a possible implementation, the video surveillance system 100 includes a preprocessing module 150. The preprocessing module 150 preprocesses each frame of the video frame data to form an enhanced image of each frame of the image, and the recognition and segmentation module 120 recognizes and segments the smoke target based on the enhanced image. In other words, after the preprocessing module 150 performs image enhancement on each frame of the video frame data, the recognition and segmentation module 120 recognizes and segments the target smoke on the enhanced image, and marks the independent area where the smoke target exists in the enhanced image.

[0027] Due to changes in ambient lighting conditions or differences in the imaging quality of the video surveillance system 100, some video frames may have low contrast, making it difficult to distinguish smoke targets from the background, or easily causing confusion with other types of smoke (such as water vapor). In order to improve the effective recognition efficiency of smoke targets, a histogram equalization preprocessing method is used to enhance different frame images to increase the visibility of the target. The histogram can improve the global contrast of the image, redistribute the grayscale of the image, make the histogram of the output image more uniform, and thus enhance the contrast of the image.

[0028] As an example, by performing histogram analysis on the video frame data, the frequency of each gray level in the image is obtained, and the gray value of each pixel is adjusted by calculating the cumulative distribution function of the original image, and the original gray value is mapped to the new gray level, so that the brightest and darkest pixels are distributed more evenly. The equalized image has a higher contrast, making the features of the smoke target more obvious and easier to distinguish. At the same time, the enhanced image can improve the recognition of smoke edges and improve the segmentation accuracy of smoke areas. Histogram equalization can adapt to different lighting conditions and scenes, and improve the robustness of the system in various environments.

[0029] The enhanced image can be used as an input for subsequent image processing, such as smoke target recognition and segmentation, texture analysis, and dynamic feature analysis by the recognition and segmentation module 120. The enhanced image has better contrast and clearer smoke features, and the recognition and segmentation module 120 can extract features and perform semantic segmentation on it. If there is a smoke target, the corresponding segmentation mark is performed for subsequent recognition and classification processing.

[0030] In a possible implementation, the video surveillance system 100 also includes using a machine learning algorithm, such as a deep neural network, to further optimize the classification of smoke types. The above algorithm can learn the characteristics of smoke targets based on labeled data, and automatically adjust the parameters of the classifier to achieve higher recognition accuracy. Use a deep learning model to extract features from the enhanced image, and these features include information such as the shape, texture, and color of the smoke target. Use a semantic segmentation network with recognition and segmentation functions to identify and segment the smoke targets in each frame. The semantic segmentation network can distinguish different objects in the image, including smoke and other background or foreground objects. Its goal is to assign a category label to each pixel in the image. The deep learning model identifies the area in the image that belongs to smoke through the smoke features learned from the training data, and marks it on the image to form an independent area. It should be noted that the independent area represents a set of all pixels in the image that are identified as smoke.

[0031] Furthermore, the boundary can be refined by at least one of morphological operations or conditional random fields (CRF) to ensure accurate segmentation of the smoke area. The accuracy and reliability of the segmentation can be improved by removing small noise areas, filling gaps or smoothing boundaries. In the video sequence, since the smoke is constantly changing and spreading, the movement and change trend of the smoke target can be analyzed by tracking the independent areas in the continuous frames to identify the diffusion pattern of the smoke.

[0032] The semantic segmentation network can use a loss function to segment and identify smoke targets. In semantic segmentation, each pixel of the deep learning model will output a corresponding probability distribution, indicating the possibility that the pixel belongs to each category (such as smoke and non-smoke). For each pixel, a loss function can be used to calculate the difference between the probability distribution predicted by the network and the true label (such as the category of the pixel). The value of each pixel in the true label represents the category of the pixel. In the case of smoke detection, the smoke area can be marked as 1 and the non-smoke area can be marked as 0.

[0033] The loss function provides a quantitative measure for the detection of smoke targets to represent the gap between the current network prediction and the actual label. The smaller the value of the loss function, the closer the network's prediction is to the true label and the better the segmentation effect. By minimizing the loss, the model can learn how to identify and segment smoke targets more accurately.

[0034] As an example, the loss function includes a cross entropy function. The trained model will use the minimized cross entropy loss to identify and segment smoke targets in real-time video frames. The semantic segmentation network can use the cross entropy function to calculate the difference between the predicted probability distribution and the actual label.

[0035] As an example, deep learning models include convolutional neural networks (CNNs), which use convolutional layers for feature extraction, effectively reducing the total number of model parameters and the complexity of the model. Its parameter sharing mechanism not only reduces the computational burden, but also improves the model's ability to generalize to new data. In each frame of a video image, if a smoke target is detected, CNN can automatically learn and extract the features of the smoke through its deep structure, and distinguish between harmful and harmless smoke.

[0036] In a certain frame of video image, if there is a smoke target, it needs to be detected and identified, and the corresponding smoke target needs to be analyzed to determine whether it is harmful smoke or harmless smoke, so as to make a subsequent emergency response judgment. For harmful smoke and harmless smoke, their morphological characteristics are mainly reflected in the texture structure. Harmful smoke contains larger particles and has a messy texture. These characteristics are manifested in the image as high contrast and irregular texture patterns. In contrast, harmless smoke, such as water vapor produced by a humidifier or cooking, has a more delicate and uniform texture and smooth fluidity, which is reflected in the image as a more consistent grayscale distribution and slow morphological changes. Through multi-level convolution and pooling operations, CNN can capture these subtle texture and structural features, thereby accurately classifying the type of smoke.

[0037] In the dimension of morphological features, different types of smoke correspond to different structural quantities, but structural quantities are only applicable to the analysis of smoke morphology in a single frame image, and are limited by recognition accuracy, image quality, lighting conditions, background complexity, etc., resulting in large recognition errors. In order to improve the accuracy and robustness of recognition, a multi-frame image sequence analysis method can be used. By analyzing the dynamic changes of smoke in consecutive frames, such as diffusion speed, directionality, morphological changes, etc., the behavioral characteristics of smoke can be more comprehensively understood. Combined with time series information, CNN can further optimize its prediction and reduce the uncertainty caused by single-frame analysis.

[0038] In addition, in order to prevent the above errors from affecting the discrimination results, the dynamic characteristics of different smoke types can be analyzed. Specifically, harmful smoke diffuses quickly, usually diffuses rapidly upward or around, and its shape changes quickly; harmless smoke diffuses slowly, rises steadily, and changes shape slowly. Harmful smoke and harmless smoke have different corresponding diffusion coefficients, and thus error-free classification judgment results can be finally obtained based on the flow diffusion coefficient.

[0039] In practical applications, deep learning models can also enhance the processing capabilities of video frame data by introducing recurrent neural networks (RNN) or long short-term memory networks (LSTM). The fusion model can simultaneously consider spatial and temporal features, more accurately capture the dynamic characteristics of smoke, and thus improve the accuracy of smoke type recognition. The deep learning model further improves its recognition capabilities through ensemble learning, model fusion and other technologies. By combining the prediction results of multiple models, the possible deviations and overfitting problems of a single model can be effectively reduced, and the stability and reliability of the model in complex environments can be improved.

[0040] In a possible implementation, the analysis and calculation module 130 is used to analyze the irregularity of the particle texture of the smoke target to determine the structural quantity. The preprocessed video frame enhanced image can be obtained, and the enhanced image is processed by the recognition and segmentation module 120, and the independent area corresponding to the smoke target is marked. The independent area is grayed, and the gray-level co-occurrence matrix (Gray-Level Co-occurrence Matrix, GLCM) algorithm is used to analyze the contrast result Con corresponding to the pixel grayscale values ​​in several sub-areas of the independent area. It should be noted that the contrast result value of the co-occurrence matrix is ​​a unique feature of the matrix, which is not the same concept as the contrast in the existing image processing. The contrast result combines specific texture information. Furthermore, a sub-area of ​​the independent area is, for example, a 3*3 area; in different applications, the sub-areas of the independent area can use other sizes, such as 2*2, 4*4, etc., and there is no specific limitation on this.

[0041] The determination of the structural quantity satisfies: Among them, Q represents the structural quantity corresponding to the independent region, Con represents the overall gray level co-occurrence matrix contrast result value of the independent region, n represents the total number of sub-regions in the independent region, Con(k) represents the gray level co-occurrence matrix contrast value in the kth sub-region in the independent region, It represents the contrast mean of the gray-level co-occurrence matrix in the kth sub-region of the independent region, and Ent(k) represents the entropy value of the pixel value in the sub-region.

[0042] It can be understood that the summation of the whole is intended to quantify the complexity or irregularity of the smoke texture type in the independent region. When the co-occurrence matrix contrast results corresponding to different sub-regions in the independent region are inconsistent, that is, the larger the variance result value of the distribution of the overall value, the higher the corresponding irregularity, then the closer the characteristics corresponding to the structural quantity are at this time, the harmful type, that is, the smoke caused by the fire, contains larger particles and the texture is more messy. The purpose of dividing the sub-regions here is: when the grayscale co-occurrence matrix calculates the texture features, the object needs to have a more regular structure. Here, it will be divided into several regular squares, which can effectively play the role of the algorithm in calculating the texture features. As for the entropy value, this part is intended to be used as the weight value when calculating the variance. The larger the entropy value of the pixel grayscale value in the sub-region, that is, the more chaotic the distribution of the pixel grayscale value, that is, close to the overall characteristics of the harmful smoke, the higher the weight value of the corresponding sub-region when calculating the structural quantity, and the larger the corresponding structural quantity.

[0043] For each independent pixel set in an independent region, the overall contrast result is calculated through the gray level co-occurrence matrix (GLCM), which reflects the difference in the grayscale values ​​of pixels in the region. For the sum coefficient, the larger the contrast result, the rougher the texture of the independent region. This rough texture feature is usually consistent with the harmful smoke produced in emergency situations such as fires. Harmful smoke is often composed of a large number of irregular particles, resulting in obvious grayscale changes in the image.

[0044] Therefore, when the amount of structure in an independent area is large, the smoke in the area has higher irregularity and roughness, increasing the possibility that the area is harmful smoke. On the contrary, if the amount of structure is small, it indicates that the texture of the smoke is relatively smooth and uniform, which is usually consistent with harmless smoke, such as the water mist produced by a humidifier. The particles of this type of smoke are small and evenly distributed, resulting in little grayscale change in the image. Through this operation, the video surveillance system 100 is able to evaluate the potential danger of smoke, thereby providing accurate information for emergency response. The recognition method based on texture analysis not only improves the accuracy of smoke detection, ensuring that timely action can be taken in emergency situations such as real fires, but also reduces false alarms caused by harmless smoke.

[0045] Due to the accuracy and efficiency of the video acquisition module 110, a single frame image may not provide enough information to accurately distinguish harmful smoke from harmless smoke. Therefore, when only a single frame image is used for identification, large errors may be introduced due to factors such as changes in ambient lighting, image noise, or insufficient smoke concentration, which directly affects the accuracy and reliability of emergency response. In order to overcome this limitation and achieve the purpose of accurate identification, the results of multiple frames of images can be jointly analyzed by the analysis and calculation module 130. The analysis and calculation module 130 can determine another more important feature, that is, the dynamic features of different smoke targets, and combine the structural quantity and the dynamic features as the basis for subsequent smoke type determination.

[0046] When processing the video frame data, the analysis and calculation module 130 not only analyzes the morphological characteristics of the smoke target at a single moment, but also analyzes the evolution of the smoke target over time. Such dynamic characteristics include the diffusion speed, direction change, and morphological change rate of the smoke target. Furthermore, the flow diffusion coefficient of the smoke target can be obtained. Harmful smoke usually diffuses rapidly in the air and changes its shape significantly due to its large and irregular particles; while harmless smoke, such as steam or water mist, diffuses relatively slowly and changes its shape more steadily and orderly.

[0047] The video surveillance system 100 evaluates the dynamic characteristics of smoke by calculating the changes in structural quantities between consecutive frames. If the structural quantity remains stable or changes little in consecutive frames, it indicates that the smoke diffuses slowly and is more likely to be harmless smoke; on the contrary, if the structural quantity changes significantly, it indicates that the smoke diffuses quickly and the shape changes dramatically, which is more likely to be a characteristic of harmful smoke. In addition, the system also analyzes the movement trajectory and direction of the smoke area in the video frame to further confirm the diffusion pattern of the smoke. Combining structural quantities and dynamic features, the system constructs a more comprehensive smoke feature model. The model not only takes into account the static texture characteristics of the smoke, but also its dynamic behavior over time. Through this multi-dimensional analysis, the system's recognition algorithm can more accurately distinguish smoke types, reduce false alarms and missed alarms, and improve the efficiency and effectiveness of emergency response.

[0048] In a possible implementation, the analysis and calculation module 130 analyzes the dynamic characteristics of the smoke target based on the structural quantity to determine the flow diffusion coefficient. The flow diffusion coefficient can be calculated by obtaining the structural quantity value corresponding to the independent area and obtaining several neighboring frames. The flow diffusion coefficient of the independent area satisfies: Where Ex represents the diffusion coefficient of the independent region; T represents the total number of different frames involved in the calculation; Q(t) represents the structural quantity corresponding to the independent region in frame t, and Q(t-1) represents the structural quantity corresponding to the independent region in frame t-1.

[0049] By analyzing the changes in the structural quantities of the independent regions of the smoke target in the video frame sequence, the video surveillance system 100 can accurately calculate the diffusion coefficient. Specifically, the video surveillance system 100 will compare the structural quantities of the independent regions in the consecutive frames. If the morphological features of the smoke target maintain a high degree of consistency between adjacent frames, showing a slow and regular diffusion pattern, such features indicate harmless smoke. In this case, the calculated diffusion coefficient will be close to 1, because the diffusion effect of harmless smoke is small, and the change in the structural quantity is not large, reflecting the stability of the smoke. On the contrary, if the morphological features of the smoke target change significantly in the consecutive frames, the diffusion speed is fast, and the shape changes dramatically, which is usually consistent with harmful smoke, the calculated diffusion coefficient will be close to 0 at this time, because the diffusion effect of harmful smoke is obvious, and the structural quantity changes greatly in the consecutive frames.

[0050] In the process of environmental monitoring and anomaly identification, accurate identification of harmful smoke and harmless smoke needs to be achieved by analyzing the diffusion characteristics of smoke targets in video frame data. The system further analyzes the flow and diffusion effects of smoke by capturing the structural quantity changes of independent areas of smoke targets in consecutive video frames. If the structural quantity changes of independent areas in adjacent frames are small, it indicates that the morphological characteristics of the smoke target fog remain consistent in time, showing a stable and regular slow diffusion pattern, which is usually consistent with harmless smoke. If the structural quantity changes significantly, it indicates that the texture and morphology of the smoke have changed significantly in a short period of time, which may be the rapid diffusion of harmful smoke.

[0051] The video surveillance system 100 quantifies the diffusion behavior of smoke through the above operations, and uses the magnitude of the flow diffusion coefficient to characterize the potential danger of smoke. This not only improves the accuracy of smoke identification, but also enables the video surveillance system 100 to respond to possible security threats more quickly. At the same time, through image analysis technology, the video surveillance system 100 can achieve real-time monitoring of the home environment and promptly detect and distinguish between harmful and harmless smoke.

[0052] Furthermore, the analysis and calculation module 130 is further used to analyze the moving direction and moving speed of the independent area in different frames to obtain a flow coefficient, and the flow coefficient satisfies: Where Li represents the flow coefficient of the independent region in different frames; Q(x,y) t Indicates the coordinates of the geometric center point of the independent area in frame t, x is the horizontal coordinate of the geometric center point, and y is the vertical coordinate of the geometric center point; Q(x,y) t―1 Indicates the coordinates of the geometric center point of the independent area in the t-1 frame.

[0053] By comparing the displacement changes of the geometric center of the independent area in different frames, its specific flow coefficient is obtained. The larger the sum value, the higher the fluidity of the smoke in these frames, and the closer the overall distribution characteristics are to harmful smoke, because the specific characteristics of harmful smoke are rapid upward diffusion; conversely, the smaller the value, the slower the corresponding displacement change, and the more consistent with the flow characteristics of harmless smoke. The image position changes of the smoke area between different frames can be monitored. If the movement of the independent area is obvious, that is, the position changes greatly in consecutive frames, this indicates that the smoke has the characteristics of rapid diffusion, which is more consistent with the diffusion pattern of harmful smoke. By comprehensively considering these dynamic characteristics, the system can quantify the diffusion and flow characteristics of smoke and provide a basis for the classification of smoke types.

[0054] After obtaining the corresponding flow diffusion coefficient, the flow diffusion coefficient is integrated according to its specific ratio result to obtain the classification parameter: In the formula, Cla(T) represents the flow diffusion coefficient of the independent smoke area calculated under the total amount of T frames; α represents the weight ratio of fluidity and diffusivity. Since the diffusion type is the main means of distinguishing harmful smoke from harmless smoke, the weight of the diffusion feature can be increased, for example, its value is 0.6. After normalization, the closer the value is to 1, the closer the classification result represented by the flow diffusion coefficient is to the harmless smoke type; conversely, the closer the value is to 0, the closer the characterization result is to the harmful smoke type.

[0055] The analysis and calculation module 130 can calculate the structural quantity changes of independent areas between adjacent frames to distinguish harmful smoke from harmless smoke. For each independent smoke area, the structural quantity is an indicator to measure the irregularity of the smoke texture, which is obtained by analyzing the distribution of pixel grayscale values ​​and texture characteristics. These independent areas need to be tracked in continuous video frames to monitor their changes over time, which can be achieved through image processing technology or optical flow algorithm to determine the movement of the smoke area in the video sequence. By comparing the structural quantities of the same smoke area in adjacent frames, change analysis is performed, and the change can indicate the diffusion speed and direction of the smoke. By calculating the structural quantity changes of independent areas between adjacent frames, the system can more accurately identify and classify smoke, thereby improving the reliability and effectiveness of the intelligent monitoring system.

[0056] Harmful smoke (such as smoke from a fire) is usually characterized by rapid diffusion, messy texture and irregularity, while harmless smoke (such as steam or cooking smoke) is usually slow to diffuse and has a more uniform texture. The dynamic behavior of the smoke is analyzed by combining the structural quantity change and the motion characteristics of the smoke area. For example, smoke that spreads rapidly upward or around may be harmful. Based on the structural quantity change and the motion of the smoke area, the flow diffusion coefficient is calculated. This coefficient quantifies the diffusion and motion characteristics of the smoke in consecutive frames. The structural quantity and the flow diffusion coefficient are integrated into classification parameters for the final smoke type classification of the classification determination module 140.

[0057] In a possible implementation, the classification determination module 140 can be used to determine the classification parameters of the smoke target according to the acquired structural quantity and flow diffusion coefficient, and the classification of different types of smoke can be determined based on the classification parameters. The determination results of the classification determination module 140 include harmful smoke and harmless smoke. If Cla(T) is in [0,0.5], the determination result is harmful smoke, and if Cla(T) is in (0.5,1], the determination result is harmless smoke. Whether the smoke is harmful or harmless is determined based on the value of the classification parameter. If the classification parameter is close to 1, it may indicate harmless smoke; if it is close to 0, it may indicate harmful smoke.

[0058] As an example, the total number of frames involved in the calculation is the total number of frames that recognize smoke within a certain period of time, which is, for example, 10 seconds. Taking the video frame rate of the video acquisition module 110 as 30fps, the total number of frames involved in the calculation here is 30*10=300 frames.

[0059] Figure 2 This is a block diagram of an alarm module of a video surveillance system provided by an embodiment of the present invention. Figure 2 The video surveillance system 100 further includes an alarm module 160. The alarm module 160 is in communication with the classification determination module 140 and triggers a corresponding alarm mechanism according to the type of smoke determined by the classification determination module 140. The alarm module 160 can promptly notify relevant personnel and initiate corresponding emergency measures when an abnormal situation is detected. Once the system identifies harmful smoke, it will automatically trigger the alarm system.

[0060] In a possible implementation, the alarm module 160 includes an alarm triggering submodule 161, a notification submodule 162 and an emergency response program starting submodule 163. The alarm triggering submodule 161 is used to trigger an alarm when a determination result of harmful smoke is obtained; the notification submodule 162 is used to send and notify alarm information; and the emergency response program starting submodule 163 is used to guide security personnel to implement it.

[0061] The alarm trigger submodule 161 can provide visual and audio warnings, such as warning lights and buzzer alarms, to remind people in the environment of danger and the need for immediate action through sound, light or other sensory stimulation; the notification submodule 162 can notify firefighters, family members or facility managers through electronic communication methods (such as text messages, emails or automatic telephone systems, etc.). It can also set up an automated system, or notify through a smart home or building automation control system. The emergency response program initiation submodule 163 can guide and assist security personnel in following the predetermined emergency response procedures. For example, clear instructions and steps are provided to help security personnel quickly assess the situation and take appropriate measures, such as evacuating personnel, using fire extinguishers or other treatment plans.

[0062] The coordinated work of the trigger submodule 161, the notification submodule 162 and the emergency response program initiation submodule 163 can ensure that when a fire or other emergency occurs in a home environment, an office environment or a factory environment, timely warning notifications of danger and the adoption of safety measures are carried out, which helps to ensure the safety of personnel and reduce property losses.

[0063] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0064] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A video surveillance system for emergency response to abnormal situations in smart homes, characterized in that: The system comprises: Video acquisition module, obtaining video frame data; A recognition and segmentation module, which uses a semantic segmentation network to recognize and segment the smoke target in the video frame data; An analysis and calculation module, used for analyzing the irregularity of the particle texture of the smoke target to determine the structural quantity, and analyzing the dynamic characteristics of the smoke target based on the structural quantity to determine the flow diffusion coefficient; The classification determination module determines the classification parameters of the smoke target according to the structural quantity and the flow diffusion coefficient, and determines the type of smoke.

2. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 1, characterized in that: Also includes: A preprocessing module is used to preprocess each frame of the video frame data to form an enhanced image of each frame, wherein the recognition and segmentation of the smoke target are performed based on the enhanced image. The recognition and segmentation module is also used to mark the independent area where the smoke target exists in the enhanced image.

3. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 2, characterized in that: The determination of the structural quantity satisfies: Wherein, Q represents the structural quantity corresponding to the independent region, n represents the total number of sub-regions in the independent region, Con represents the overall gray level co-occurrence matrix contrast result value of the independent region, Con(k) represents the gray level co-occurrence matrix contrast value in the kth sub-region of the independent region, represents the gray level co-occurrence matrix contrast mean value in the kth sub-region of the independent region, and Ent(k) represents the pixel value entropy value in the sub-region.

4. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 2, characterized in that: The analysis and calculation module is also used to analyze the moving direction and moving speed of the independent area in different frames to obtain the flow coefficient. Wherein, the flow coefficient satisfies: Where Li represents the flow coefficient of the independent region in different frames, Q(x,y) t represents the geometric center coordinates of the independent region in frame t; Q(x,y) t―1 Represents the coordinates of the geometric center point of the independent area in the t-1 frame.

5. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 4, characterized in that: The flow diffusion coefficient satisfies: Wherein, Ex represents the diffusion coefficient of the independent region, T represents the total amount of different frames involved in the calculation, Q(t) represents the structural quantity corresponding to the independent region in the t frame, and Q(t-1) represents the structural quantity corresponding to the independent region in the t-1 frame; The classification parameters satisfy: Wherein, Cla(T) represents the flow diffusion coefficient of the independent region calculated under the total amount of T frames, and α represents the weight ratio of fluidity and diffusivity.

6. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 5, characterized in that: The determination result of the classification determination module includes harmful smoke and harmless smoke, wherein when Cla(T) is in [0,0.5], the determination result is harmful smoke, and when Cla(T) is in (0.5,1], the determination result is harmless smoke.

7. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 1, characterized in that: The semantic segmentation network uses the cross entropy function to calculate the difference between the predicted probability distribution and the true label.

8. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 1, characterized in that: The analysis and calculation module calculates the structural quantity change of the independent area between adjacent frames.

9. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 6, characterized in that: Also includes: The alarm module is connected to the classification and determination module for triggering a corresponding alarm mechanism according to the smoke type.

10. A video surveillance system for emergency response to abnormal situations in smart homes according to claim 9, characterized in that: The alarm module comprises: An alarm triggering submodule, used for triggering an alarm when the determination result of the harmful smoke is obtained; Notification submodule, used for sending and notifying alarm information; The emergency response procedure initiates the submodule, which is used to guide security personnel in implementing on-site assessments and safety measures.

Images