Electrical equipment operation monitoring system and method based on flame smoke video analysis

Abstract

The application discloses an electrical equipment operation monitoring system and method based on flame smoke video analysis, and relates to the field of electrical equipment operation monitoring.The method comprises the following steps: preliminarily evaluating an electrical equipment area, and constructing a three-dimensional model of the electrical equipment area; analyzing and identifying a key monitoring position, installing a high-definition camera, and performing a view angle test; collecting electrical equipment area video in real time, and marking out a suspected flame or smoke area; extracting a flame and smoke area according to the suspected flame or smoke area; calculating the position of a fire source in a three-dimensional space according to the flame area; when the smoke area is extracted but the flame area is not extracted, analyzing the position of the fire source in the three-dimensional space in combination with the smoke shape, flow direction and convergence point detected by multiple high-definition cameras; after the fire source is marked out, triggering an alarm device, sending a three-dimensional model of the electrical equipment area with a fire source position mark to a terminal of a user, and performing emergency power-off.

Description

Technical Field

[0001] This invention relates to the field of electrical equipment operation monitoring, specifically to an electrical equipment operation monitoring system and method based on flame and smoke video analysis. Background Technology

[0002] With the widespread application of electrical equipment, traditional electrical equipment monitoring systems generally rely on manual inspections or basic temperature sensors and smoke detectors for monitoring. However, these methods suffer from drawbacks such as slow response and insufficient accuracy. Especially with the complex layout of electrical equipment, traditional monitoring methods struggle to comprehensively, in real-time, and accurately monitor the specific location of a fire. In recent years, fire monitoring technology based on video image analysis has gradually been applied. However, existing flame and smoke recognition methods still suffer from insufficient recognition accuracy and long processing delays, failing to accurately locate the fire source in the first instance and affecting the effectiveness of fire early warning.

[0003] Existing technologies largely rely on single monitoring devices, such as smoke sensors or infrared temperature sensors, which have limitations in terms of accuracy and response speed. Especially when electrical equipment areas are complex and spatially irregular, traditional monitoring methods have many blind spots and often cannot accurately locate the fire source. Summary of the Invention

[0004] The purpose of this invention is to provide an electrical equipment operation monitoring system and method based on flame and smoke video analysis to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for monitoring the operation of electrical equipment based on flame and smoke video analysis, the method comprising:

[0006] A preliminary assessment of the layout and spatial location of the electrical equipment area is conducted, and a three-dimensional model of the electrical equipment area is constructed.

[0007] Based on the 3D model of the electrical equipment area, key monitoring locations are analyzed and identified, high-definition cameras are installed, and viewing angle tests are conducted to ensure that there are no blind spots in the key areas.

[0008] Based on the installed high-definition cameras, real-time video of the electrical equipment area is collected; and the presence of flames or smoke in the video is identified, marking areas suspected of having flames or smoke.

[0009] When flames or smoke are detected in the video, the flame or smoke area is extracted based on the suspected flame or smoke area.

[0010] Based on the flame area, calculate the position of the fire source in three-dimensional space, and mark the position of the fire source in three-dimensional space onto the corresponding position of the three-dimensional model of the electrical equipment area;

[0011] When the smoke area is extracted but the flame area is not extracted, the position of the fire source in three-dimensional space is analyzed by combining the smoke shape, flow direction and convergence point detected by multiple high-definition cameras, and the position of the fire source in three-dimensional space is marked on the corresponding position of the three-dimensional model of the electrical equipment area.

[0012] Once the fire source is marked in the 3D model of the electrical equipment area, the alarm device is immediately triggered, sending the 3D model of the electrical equipment area with the fire source location marked to the user's terminal, and performing an emergency power cut-off.

[0013] According to the above scheme, high-definition cameras are installed at each of the key monitoring locations, and viewing angle tests are conducted. The viewing angle test includes analyzing the viewing angle range covered by each high-definition camera, the size of the monitoring area, and the space and layout around the electrical equipment to ensure that the high-definition cameras cover all key areas and that the images of the key areas are clear and without blind spots.

[0014] If the perspective test reveals blind spots or unclear images in the key area, adjust the installation angle or position of the high-definition camera or increase the number of high-definition cameras.

[0015] According to the above scheme, the video of the electrical equipment area is processed into frames of images to form video frames;

[0016] Gaussian filtering is applied to the video frames to remove high-frequency noise and smooth the image;

[0017] Using a Gaussian mixture model, multiple Gaussian distributions are established for each pixel's color or brightness value. Based on these multiple Gaussian distributions, background modeling is performed on each pixel, with different Gaussian distributions representing different states in the background.

[0018] When the difference between the pixel value in the background and the Gaussian distribution in the model exceeds a set threshold, the pixel value will be marked as a suspected flame or smoke area.

[0019] According to the above scheme, when identifying the presence of flames in a video, the flame region is extracted based on the suspected flame or smoke area, including:

[0020] Based on the suspected flame or smoke area, color feature analysis is performed to obtain the color information of the flame, and it is marked as the first verified flame area; based on the suspected flame or smoke area, edge detection is performed to obtain the outline information of the flame, and it is marked as the second verified flame area; combining the first verified flame area and the second verified flame area, the flame area is analyzed and extracted.

[0021] According to the above scheme, the color feature analysis includes:

[0022] The suspected flame or smoke area is converted from the RGB color space to the HSV color space. The HSV color space is based on hue, saturation, and lightness. By setting a range of hue, saturation, and lightness, the HSC value of each pixel in the suspected flame or smoke area is compared with the range of hue, saturation, and lightness. If the HSV value of a pixel is within the range of hue, saturation, and lightness, then the pixel is marked as the first verified flame area.

[0023] The edge detection includes:

[0024] The suspected flame or smoke area is converted from an RGB color image to a grayscale image;

[0025] The grayscale image is Gaussian smoothed. The Gaussian smoothing process involves constructing a Gaussian kernel centered on each pixel in the grayscale image and performing a convolution operation between the Gaussian kernel and the pixels of the grayscale image to achieve Gaussian smoothing of the grayscale image and reduce the impact of noise.

[0026] Calculate the gradient magnitude and gradient direction of each pixel in the Gaussian smoothed grayscale image. The gradient magnitude represents the degree of drastic change in pixel value in the grayscale image, and the gradient direction represents the direction of the fastest change in pixel value.

[0027] Based on the gradient direction θ1 of pixel s1 in the grayscale image, determine the two adjacent pixels s0 and s2 of the pixel in the grayscale image along the gradient direction, and compare the gradient magnitude G1 of pixel s1 with the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2; when the gradient magnitude G1 of pixel s1 in the grayscale image is greater than the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2, then retain the gradient magnitude G1 of pixel s1 in the grayscale image; when the gradient magnitude G1 of pixel s1 in the grayscale image is less than either the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2, then set the gradient magnitude G1 of pixel s1 in the grayscale image to 0.

[0028] Set a high threshold T h and low threshold T l Used to detect edge pixels; when the gradient magnitude of a pixel in the grayscale image is greater than a high threshold T h When the gradient magnitude of a pixel in the grayscale image is less than a low threshold T, it is determined to be a pixel with a strong edge. l If the gradient magnitude of a pixel in the grayscale image is between a high threshold T and an edge pixel, then the pixel in the grayscale image is determined to be a non-edge pixel. h and low threshold T lWhen the strong edge pixel is connected to the strong edge pixel, the weak edge pixel adjacent to the strong edge pixel is checked. When the adjacent weak edge pixel is connected to the strong edge pixel, the adjacent weak edge pixel is marked as an edge pixel. The edge connection is completed to obtain clear flame outline information, and the pixel area in the flame outline information is marked as the second verification flame area.

[0029] Perform an intersection operation on the pixels of the first verification flame region and the second verification flame region; when a pixel is simultaneously a pixel in both the first verification flame region and the second verification flame region, it is determined to be a pixel in the flame region; extract all pixels that are simultaneously in both the first verification flame region and the second verification flame region to form the flame region.

[0030] According to the above scheme, when smoke is detected in a video, the smoke area is extracted based on the suspected flame or smoke area, including:

[0031] Based on the suspected flame or smoke area, grayscale threshold segmentation is performed to generate a binarized region image, which is marked as the first verification smoke region. Based on the first verification smoke region, edge detection and contour detection are used to obtain the smoke shape within the smoke template range, which is marked as the second verification smoke region. Based on the suspected flame or smoke region, optical flow calculation is performed to obtain the smoke motion features, which are marked as the third verification smoke region. Combining the second and third verification smoke regions, the smoke region is analyzed and extracted.

[0032] According to the above scheme, the grayscale threshold segmentation includes:

[0033] The suspected flame or smoke area is converted from an RGB color image to a grayscale image;

[0034] Based on the grayscale characteristics of smoke, a grayscale threshold H is set through sample analysis; the grayscale value of each pixel in the grayscale image is compared with the grayscale threshold H; when the grayscale value of a pixel in the grayscale image is less than or equal to the grayscale threshold H, the pixel is marked as a pixel in the first verification smoke region; when the grayscale value of a pixel in the grayscale image is greater than the grayscale threshold H, it is marked as a non-smoke region; all pixels marked as the first verification smoke region are combined to form the first verification smoke region.

[0035] The edge detection and contour detection include:

[0036] Edge detection is performed on the first verification smoke area to obtain smoke edge information;

[0037] Based on the smoke edge information, a contour detection algorithm is used to detect the contour of the smoke region, calculate the perimeter of the smoke region contour, determine the area enclosed by the smoke region contour, and calculate the aspect ratio using the length and width of the bounding rectangle of the smoke region contour.

[0038] The perimeter, area, and aspect ratio of the smoke region outline are compared with the smoke model; when the perimeter, area, and aspect ratio of the smoke region outline are within the range of the smoke template, the area within the smoke region outline is marked as the second verification smoke region.

[0039] The optical flow method includes:

[0040] Based on the video of the electrical equipment area, extract two consecutive frames of images containing the suspected flame or smoke area, and perform grayscale and Gaussian filtering on the suspected flame or smoke area in the images to obtain a denoised grayscale image.

[0041] Based on the denoised grayscale image, feature points are selected using the Shi-Tomasi corner detection algorithm. The Shi-Tomasi corner detection algorithm calculates the grayscale changes of each pixel in the denoised grayscale image in different directions, obtaining points with significant grayscale changes in multiple directions as corner points. These corner points are then filtered, removing points with overly smooth edges or insignificant grayscale changes, retaining a representative set of feature points P = {p1, p2, ..., p...}. n}, where p n Let P be a feature point, and n be the total number of feature points; i Adjacent pixels are set as feature points P. i For each of the feature points P, the points within the neighborhood are... i Optical flow constraint equations are established for points in the neighborhood to obtain feature point P. i The optical flow vector is (u i ,v i ), where (u,v) represents the displacement of the feature point between two consecutive frames containing the suspected flame or smoke area; calculate the optical flow vector of each feature point P to obtain the optical flow vector set V={(u1,v1),(u2,v2),…,(u n ,v n )}; Calculate the direction of each optical flow vector and perform statistical analysis on the optical flow directions of all feature points; when the optical flow direction shows a diffusion trend, it is judged as a smoke region and marked as the third verification smoke region;

[0042] Combining the second verification smoke region and the third verification smoke region, the overlapping pixel region of the second verification smoke region and the third verification smoke region is analyzed, and the overlapping pixel region is the smoke region.

[0043] According to the above scheme, the step of calculating the position of the fire source in three-dimensional space based on the flame area, and marking the position of the fire source in three-dimensional space onto the corresponding position of the three-dimensional model of the electrical equipment area, includes:

[0044] When only one high-definition camera detects the flame area, the location of the fire source in three-dimensional space is calculated based on the installation position and viewing angle of the high-definition camera and the three-dimensional model of the electrical equipment area.

[0045] When multiple high-definition cameras detect the flame area, the location of the fire source is jointly estimated by utilizing the differences in the viewing angle and position of each camera and through spatial geometric relationships.

[0046] The location of the fire source in three-dimensional space is marked on the corresponding position of the three-dimensional model of the electrical equipment area.

[0047] According to the above scheme, based on the installation position, viewing angle and imaging principle of the high-definition camera, a projection relationship from the two-dimensional image coordinates captured by the high-definition camera to the three-dimensional spatial coordinates is established.

[0048] Based on the smoke area detected by the high-definition camera, and according to the projection relationship, the smoke shape, flow direction and convergence point of the two-dimensional image are converted into coordinate information in three-dimensional space;

[0049] Based on the smoke flow direction and convergence point detected by the multiple high-definition cameras, the motion vector of the smoke and the range of the convergence point are drawn in the three-dimensional space;

[0050] Based on the fire source direction obtained by reverse calculation of the smoke flow and the location information of the convergence point, a comprehensive analysis is performed to estimate the location of the fire source in three-dimensional space, and the location of the fire source in three-dimensional space is marked on the corresponding position of the three-dimensional model of the electrical equipment area.

[0051] An electrical equipment operation monitoring system based on flame and smoke video analysis, the system includes: a 3D model module, a flame or smoke detection module, a fire source location module, and a fire source alarm module;

[0052] The 3D model module includes a perspective testing module, a video acquisition module, and a model construction module. The model construction module performs a preliminary assessment of the electrical equipment area and constructs a 3D model of the electrical equipment area. The perspective testing module identifies key monitoring locations based on the model construction module and performs perspective testing by installing high-definition cameras to ensure that each key monitoring area is clearly covered by the high-definition cameras. The video acquisition module acquires video data of the electrical equipment area in real time through the installed high-definition cameras.

[0053] The flame or smoke detection module includes a flame or smoke extraction module, a flame verification module, and a smoke verification module. The flame or smoke extraction module uses a Gaussian mixture model to analyze the color or brightness changes of each pixel and detect differences from the background to determine if a suspected flame or smoke area exists. The flame verification module extracts the suspected flame or smoke area through color feature analysis and edge detection, and analyzes it in conjunction with a first verified flame area and a second verified flame area to ultimately confirm the accurate location of the flame area. The smoke verification module generates a first smoke verification area through grayscale threshold segmentation, obtains a second smoke verification area through edge detection and contour detection, and further analyzes the motion characteristics of the smoke using optical flow to obtain a third smoke verification area. Combining the second and third smoke verification areas confirms the accurate location of the smoke area.

[0054] The fire source location module includes a flame fire source location module and a smoke fire source location module; the flame fire source location module calculates the fire source location based on one or more cameras; the smoke fire source location module calculates the fire source location based on the shape, flow direction and convergence point of the smoke.

[0055] The fire source alarm module displays a 3D model with the fire source location marked to the user through the terminal interface, and provides real-time monitoring, alarm and operation feedback functions.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] 1. This invention reduces misjudgments of flames and smoke through a multi-step verification process, thereby improving the accuracy and reliability of monitoring the operation of electrical equipment;

[0058] 2. This invention locates fire sources based on various scenarios, providing accurate three-dimensional model locations of the fire sources, thus providing accurate location information for timely action.

[0059] 3. This invention uses an alarm device to send a three-dimensional model with the location of the fire source to the user terminal and performs an emergency power cut-off to ensure the safety of personnel and equipment. Attached Figure Description

[0060] Figure 1The following is a flowchart of the electrical equipment operation monitoring method based on flame and smoke video analysis of the present invention;

[0061] Figure 2 This is a step-by-step verification process for the flame and smoke multi-step method for monitoring the operation of electrical equipment based on flame and smoke video analysis, as described in this invention.

[0062] Figure 3 This is a schematic diagram of the electrical equipment operation monitoring system based on flame and smoke video analysis according to the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example: Figures 1-3 As shown, this invention provides a technical solution: a method for monitoring the operation of electrical equipment based on flame and smoke video analysis; the method includes the following steps:

[0065] S1. Conduct a preliminary assessment of the layout and spatial location of the electrical equipment area, and construct a three-dimensional model of the electrical equipment area;

[0066] Specifically, for example: a large electrical equipment room, with an area of ​​50m × 30m, including distribution cabinets, transformers, cable channels and equipment control panels; using laser scanning to conduct a preliminary assessment of the layout and spatial location of the electrical equipment area, and using 3D modeling software to create a 3D model of the electrical equipment area;

[0067] S2. Based on the three-dimensional model of the electrical equipment area, analyze and identify key monitoring locations, install high-definition cameras, and conduct viewing angle tests to ensure that there are no blind spots in the key areas.

[0068] Specifically, high-definition cameras are installed at each key monitoring location. For example, four high-definition cameras are installed near the distribution cabinet, covering the front and sides of the cabinet; six high-definition cameras are installed on the transformer, covering the four corners and the middle area; two high-definition cameras are installed in the cable channel, located at both ends of the channel; and two high-definition cameras are installed on the equipment control panel, covering the top and sides of the control panel. Next, viewing angle testing is conducted. Viewing angle testing includes analyzing the viewing angle range covered by each high-definition camera, the size of the monitored area, and the space and layout around the electrical equipment to ensure that the high-definition cameras cover key areas and that the images in key areas are clear and without blind spots. If the viewing angle test finds blind spots or unclear images in key areas, the installation angle and position of the high-definition cameras are adjusted, or the number of high-definition cameras is increased.

[0069] S3. Based on the installed high-definition camera, real-time video of the electrical equipment area is collected; and the presence of flames or smoke in the video is identified, and areas suspected of flames or smoke are marked.

[0070] Specifically, the video of the electrical equipment area is processed into individual frames to form video frames. Gaussian filtering is applied to the video frames to remove high-frequency noise and smooth the image. Using a Gaussian mixture model, multiple Gaussian distributions are established for each pixel's color or brightness value. Based on these distributions, background modeling is performed for each pixel, with different distributions representing different states in the background. When the difference between a pixel value in the background and the modeled Gaussian distribution exceeds a set threshold, the pixel value is marked as a suspected flame or smoke area. For example, in a certain frame, the pixel located at (500, 600) has a color value that differs significantly from the background model, exceeding a set threshold of 3 times the standard deviation. In this case, the pixel is marked as a suspected flame or smoke area.

[0071] S4. When flames are detected in the video, the flame area is extracted based on the suspected flame or smoke area.

[0072] Specifically, based on the suspected flame or smoke area, color feature analysis is performed to obtain the color information of the flame, and it is marked as the first verified flame area; based on the suspected flame or smoke area, edge detection is performed to obtain the outline information of the flame, and it is marked as the second verified flame area; combining the first verified flame area and the second verified flame area, the flame area is analyzed and extracted.

[0073] Further, color feature analysis includes: converting suspected flame or smoke areas from the RGB color space to the HSV color space; the HSV color space, based on hue, saturation, and lightness, sets hue, saturation, and lightness ranges, for example: setting the hue range of flames to [0,30] (0-360 degrees), the saturation range to [120,255], and the lightness range to [120,255], and comparing the HSC value of each pixel in the suspected flame or smoke area with the hue, saturation, and lightness ranges; if the HSV value of a pixel is within the hue, saturation, and lightness ranges, then the pixel is marked as the first verified flame area; for example, a pixel with coordinates (550,650) in the suspected flame or smoke area has an HSV value of (20,150,180), which meets the set range, and is marked as a pixel in the first verified flame area.

[0074] Further edge detection includes: converting suspected flame or smoke areas from RGB color images to grayscale images; performing Gaussian smoothing on the grayscale images. Gaussian smoothing involves constructing a Gaussian kernel centered on each pixel in the grayscale image, and convolving the Gaussian kernel with the pixels of the grayscale image to achieve Gaussian smoothing and reduce the impact of noise; calculating the gradient magnitude and gradient direction of each pixel in the Gaussian-smoothed grayscale image, where the gradient magnitude represents the drastic change in pixel value and the gradient direction represents the direction of the fastest change in pixel value; and determining the two adjacent pixels s0 and s2 in the gradient direction of pixel s1 in the grayscale image based on the gradient direction θ1, and comparing the gradient magnitude G1 of pixel s1 in the grayscale image. The gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2 are compared. If the gradient magnitude G1 of pixel s1 in the grayscale image is greater than the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2, then the gradient magnitude G1 of pixel s1 in the grayscale image is retained. If the gradient magnitude G1 of pixel s1 in the grayscale image is less than either of the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2, then the gradient magnitude G1 of pixel s1 in the grayscale image is set to 0. For example, for pixel s1, its gradient magnitude G1 = 50 and gradient direction θ1 = 45°. The two adjacent pixels s0 and s2 in this direction are determined according to the gradient direction. Assuming that the gradient magnitude G0 of s0 is 30 and the gradient magnitude G2 of s2 is 40, since G1 > G0 and G1 > G2, G1 is retained.

[0075] Set a high threshold T h and low threshold T l It is used to detect edge pixels; when the gradient magnitude of a pixel in a grayscale image is greater than a high threshold T. h When the gradient magnitude of a pixel in the grayscale image is less than a low threshold T, it is determined to be a pixel with a strong edge. lIf the gradient magnitude of a pixel in the grayscale image is between a certain threshold T, then the pixel is determined to be a non-edge pixel; if the gradient magnitude of a pixel in the grayscale image is between a certain threshold T, then the pixel is determined to be a non-edge pixel. h and low threshold T l When the pixel in the grayscale image is between strong and weak edge pixels, it is determined to be a weak edge pixel. Starting from the strong edge pixel, the weak edge pixels adjacent to the strong edge pixel are checked. When the adjacent weak edge pixel is connected to the strong edge pixel, the adjacent weak edge pixel is marked as an edge pixel. The edge connection is completed to obtain clear flame outline information, and the pixel area in the flame outline information is marked as the second verification flame area.

[0076] Perform an intersection operation on the pixels of the first verification flame region and the second verification flame region; when a pixel is simultaneously a pixel in both the first and second verification flame regions, it is determined to be a pixel in the flame region; extract all pixels that are simultaneously in both the first and second verification flame regions to form the flame region.

[0077] S5. When smoke is detected in the video, the smoke area is extracted based on the suspected flame or smoke area.

[0078] Specifically, based on the suspected flame or smoke area, grayscale threshold segmentation is performed to generate a binarized region image, which is then marked as the first verification smoke region. Based on the first verification smoke region, edge detection and contour detection are used to obtain the smoke shape within the smoke template range, which is then marked as the second verification smoke region. Based on the suspected flame or smoke area, optical flow calculation is performed to obtain the smoke motion features, which are then marked as the third verification smoke region. Combining the second and third verification smoke regions, the smoke region is analyzed and extracted.

[0079] Furthermore, the grayscale threshold segmentation includes: converting the suspected flame or smoke area from an RGB color image to a grayscale image; setting a grayscale threshold H based on the grayscale characteristics of smoke through sample analysis; comparing the grayscale value of each pixel in the grayscale image with the grayscale threshold H; when the grayscale value of a pixel in the grayscale image is less than or equal to the grayscale threshold H, the pixel is marked as a pixel in the first verified smoke area; when the grayscale value of a pixel in the grayscale image is greater than the grayscale threshold H, it is marked as a non-smoke area; all pixels marked as the first verified smoke area are combined to form the first verified smoke area.

[0080] Further, edge detection and contour detection include: performing edge detection on the first verification smoke region to obtain smoke edge information; based on the smoke edge information, using a contour detection algorithm to detect the contour of the smoke region, calculating the perimeter of the smoke region contour, determining the area enclosed by the smoke region contour, and calculating the aspect ratio using the length and width of the bounding rectangle of the smoke region contour; comparing the perimeter, area, and aspect ratio of the smoke region contour with the smoke model; when the perimeter, area, and aspect ratio of the smoke region contour are within the range of the smoke template, the region within the smoke region contour is marked as the second verification smoke region. For example, when detecting the outline of a smoke region, the perimeter of the outline is calculated to be 200 pixels, and the area enclosed by the outline is 1500 pixels². The length and width of the bounding rectangle of the outline are calculated to be 50 pixels and 30 pixels, respectively, and the aspect ratio is calculated to be 5 / 3. The perimeter range of the smoke model is set to [150, 250] pixels, the area range is [1000, 2000] pixels², and the aspect ratio range is [1, 2]. If the parameters of the detected smoke region outline are within the range of the smoke template, then the area within the detected smoke region outline is marked as the second verification smoke region. This is just an example and is not a limitation.

[0081] Furthermore, the optical flow method includes: extracting two consecutive frames of images containing suspected flame or smoke areas from video of the electrical equipment area; performing grayscale conversion and Gaussian filtering on the suspected flame or smoke areas in the images to obtain a denoised grayscale image; selecting feature points based on the denoised grayscale image using the Shi-Tomasi corner detection algorithm; the Shi-Tomasi corner detection algorithm calculates the grayscale changes of each pixel in the denoised grayscale image in different directions to obtain points with significant grayscale changes in multiple directions as corner points; filtering the corner points to remove points with overly smooth edges or insignificant grayscale changes, retaining a representative set of feature points P = {p1, p2, ..., p...} n}, where p n Let P be a feature point, and n be the total number of feature points; i Adjacent pixels are set as feature points P. i For each feature point P within the neighborhood... i Optical flow constraint equations are established for points in the neighborhood to obtain feature point P. i The optical flow vector is (u i ,v i ), where (u,v) represents the displacement of a feature point between two consecutive frames containing a suspected flame or smoke area; calculate the optical flow vector of each feature point P to obtain the optical flow vector set V={(u1,v1),(u2,v2),…,(u n ,v n)}; Calculate the direction of each optical flow vector and perform statistical analysis on the optical flow directions of all feature points; when the optical flow direction shows a diffusion trend, it is judged as a smoke region and marked as the third verification smoke region;

[0082] By combining the second and third verification smoke regions, the overlapping pixel regions of the second and third verification smoke regions are analyzed, and the overlapping pixel regions are the smoke regions.

[0083] S6. Based on the flame area, calculate the position of the fire source in three-dimensional space, and mark the position of the fire source in three-dimensional space onto the corresponding position of the three-dimensional model of the electrical equipment area;

[0084] Specifically, when only one high-definition camera detects a flame area, the location of the fire source in three-dimensional space is calculated based on the installation position and viewing angle of the high-definition camera, as well as the three-dimensional model of the electrical equipment area. For example, if the high-definition camera is installed at position (10, 5, 2) (unit: meters, with a coordinate system established with a corner of the room as the origin), with a horizontal viewing angle of 120° and a vertical viewing angle of 90°, the location of the fire source in three-dimensional space is calculated as (15, 8, 3) (unit: meters) based on the installation position and viewing angle of the high-definition camera, as well as the three-dimensional model of the electrical equipment area, using triangulation, the location of the fire source in three-dimensional space is calculated as (15, 8, 3) (unit: meters).

[0085] When multiple high-definition cameras detect a flame area, the location of the fire source is jointly estimated by utilizing the differences in the viewing angle and position of each camera and through spatial geometric relationships. For example, if three cameras detect a flame area located at (10,5,2), (20,10,2.5), and (30,15,3) (unit: meters), the location of the fire source in three-dimensional space is calculated as (18,12,3.2) (unit: meters) by utilizing the differences in the viewing angle and position of each camera, and through the principle of triangulation, combined with the position information of the flame area captured by the three cameras in the image, and then marked on the corresponding position in the three-dimensional model.

[0086] S7. When the smoke area is extracted but the flame area is not extracted, the position of the fire source in three-dimensional space is analyzed by combining the smoke shape, flow direction and convergence point detected by multiple high-definition cameras, and the position of the fire source in three-dimensional space is marked to the corresponding position of the three-dimensional model of the electrical equipment area.

[0087] Specifically, based on the installation location, viewing angle, and imaging principle of the high-definition camera, a projection relationship is established from the coordinates of the two-dimensional image captured by the high-definition camera to the coordinates in three-dimensional space. Based on the smoke area detected by the high-definition camera, the shape, flow direction, and convergence point of the smoke in the two-dimensional image are converted into coordinate information in three-dimensional space according to the projection relationship. Based on the smoke flow direction and convergence point detected by multiple high-definition cameras, the motion vector of the smoke and the range of the convergence point are drawn in three-dimensional space. Based on the fire source direction and convergence point location information obtained by reverse calculation from the smoke flow direction, a comprehensive analysis is performed to estimate the position of the fire source in three-dimensional space, and the position of the fire source in three-dimensional space is marked on the corresponding position of the three-dimensional model of the electrical equipment area.

[0088] S8. When the fire source is marked in the three-dimensional model of the electrical equipment area, the alarm device is immediately triggered to send the three-dimensional model of the electrical equipment area with the fire source location marked to the user's terminal and to cut off the power in an emergency.

[0089] Specifically, for example, once the location of the fire source is determined, the system sends a 3D model image containing the location of the fire source (19,14,3.5) to the user through the terminal interface, and automatically cuts off the power supply to the relevant electrical equipment to prevent the fire from spreading further.

[0090] This invention provides another technical solution: an electrical equipment operation monitoring system based on flame and smoke video analysis; the system includes: a three-dimensional model module, a flame or smoke detection module, a fire source location module, and a fire source alarm module;

[0091] The 3D model module includes a perspective testing module, a video acquisition module, and a model construction module. The model construction module performs a preliminary assessment of the electrical equipment area and constructs a 3D model of the electrical equipment area. The perspective testing module identifies key monitoring locations based on the model construction module and performs perspective testing by installing high-definition cameras to ensure that each key monitoring area is clearly covered by the high-definition cameras. The video acquisition module acquires video data of the electrical equipment area in real time through the installed high-definition cameras.

[0092] The flame or smoke detection module includes a flame or smoke extraction module, a flame verification module, and a smoke verification module. The flame or smoke extraction module uses a Gaussian mixture model to analyze the color or brightness changes of each pixel and detect differences from the background to determine if a suspected flame or smoke area exists. The flame verification module extracts the suspected flame or smoke area through color feature analysis and edge detection, and analyzes it in conjunction with a first verified flame area and a second verified flame area to ultimately confirm the accurate location of the flame area. The smoke verification module generates a first smoke verification area through grayscale threshold segmentation, obtains a second smoke verification area through edge detection and contour detection, and further analyzes the motion characteristics of the smoke using optical flow to obtain a third smoke verification area. Combining the second and third smoke verification areas confirms the accurate location of the smoke area.

[0093] The fire source location module includes a flame fire source location module and a smoke fire source location module; the flame fire source location module calculates the fire source location based on one or more cameras; the smoke fire source location module calculates the fire source location based on the shape, flow direction and convergence point of the smoke.

[0094] The fire source alarm module displays a 3D model with the fire source location marked to the user through the terminal interface, and provides real-time monitoring, alarm and operation feedback functions.

[0095] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for monitoring the operation of electrical equipment based on flame and smoke video analysis, characterized in that: The method includes: A preliminary assessment of the layout and spatial location of the electrical equipment area is conducted, and a three-dimensional model of the electrical equipment area is constructed. Based on the 3D model of the electrical equipment area, key monitoring locations are analyzed and identified, high-definition cameras are installed, and viewing angle tests are conducted to ensure that there are no blind spots in the key areas. Based on the installed high-definition cameras, real-time video of the electrical equipment area is collected; and the presence of flames or smoke in the video is identified, marking areas suspected of having flames or smoke. When flames or smoke are detected in the video, the flame or smoke area is extracted based on the suspected flame or smoke area. When smoke is detected in a video, the smoke region is extracted based on the suspected flame or smoke area, including: performing grayscale threshold segmentation on the suspected flame or smoke area to generate a binarized region image, which is then marked as a first verified smoke region; based on the first verified smoke region, using edge detection and contour detection, obtaining the smoke shape within the smoke template range, which is then marked as a second verified smoke region; based on the suspected flame or smoke region, performing optical flow calculation to obtain smoke motion features, which are then marked as a third verified smoke region; and combining the second and third verified smoke regions to analyze and extract the smoke region. The grayscale threshold segmentation includes: The suspected flame or smoke area is converted from an RGB color image to a grayscale image; Based on the grayscale characteristics of smoke, a grayscale threshold H is set through sample analysis; the grayscale value of each pixel in the grayscale image is compared with the grayscale threshold H; when the grayscale value of a pixel in the grayscale image is less than or equal to the grayscale threshold H, the pixel is marked as a pixel in the first verification smoke region; when the grayscale value of a pixel in the grayscale image is greater than the grayscale threshold H, it is marked as a non-smoke region; all pixels marked as the first verification smoke region are combined to form the first verification smoke region. The edge detection and contour detection include: Edge detection is performed on the first verification smoke area to obtain smoke edge information; Based on the smoke edge information, a contour detection algorithm is used to detect the contour of the smoke region, calculate the perimeter of the smoke region contour, determine the area enclosed by the smoke region contour, and calculate the aspect ratio using the length and width of the bounding rectangle of the smoke region contour. The perimeter, area, and aspect ratio of the smoke region outline are compared with the smoke model; when the perimeter, area, and aspect ratio of the smoke region outline are within the range of the smoke template, the area within the smoke region outline is marked as the second verification smoke region. The optical flow method includes: Based on the video of the electrical equipment area, extract two consecutive frames of images containing the suspected flame or smoke area, and perform grayscale and Gaussian filtering on the suspected flame or smoke area in the images to obtain a denoised grayscale image. Based on the denoised grayscale image, feature points are selected using the Shi-Tomasi corner detection algorithm. The Shi-Tomasi corner detection algorithm calculates the grayscale changes of each pixel in the denoised grayscale image in different directions, obtaining points with significant grayscale changes in multiple directions as corner points. These corner points are then filtered, removing points with overly smooth edges or insignificant grayscale changes, retaining a representative set of feature points P={p1,p2,…,p...}. n }, where p n Let P be a feature point, and n be the total number of feature points; i Adjacent pixels are set as feature points P. i For each of the feature points P, the points within the neighborhood are... i Optical flow constraint equations are established for points in the neighborhood to obtain feature point P. i The optical flow vector is (u i ,v i ), where (u,v) represents the displacement of the feature point between two consecutive frames containing the suspected flame or smoke area; calculate the optical flow vector of each feature point P to obtain the optical flow vector set V={(u1,v1),(u2,v2),…,(u... n ,v n )}; Calculate the direction of each optical flow vector and perform statistical analysis on the optical flow directions of all feature points; when the optical flow direction shows a diffusion trend, it is judged as a smoke region and marked as the third verification smoke region; Combining the second verification smoke region and the third verification smoke region, the overlapping pixel region of the second verification smoke region and the third verification smoke region is analyzed, and the overlapping pixel region is the smoke region. Based on the flame area, calculate the position of the fire source in three-dimensional space, and mark the position of the fire source in three-dimensional space onto the corresponding position of the three-dimensional model of the electrical equipment area; When the smoke area is extracted but the flame area is not, a projection relationship is established from the two-dimensional image coordinates captured by the high-definition camera to the three-dimensional spatial coordinates based on the installation position, viewing angle, and imaging principle of the high-definition camera. Based on the smoke shape, flow direction, and convergence point detected by the high-definition camera, the smoke shape, flow direction, and convergence point of the two-dimensional image are converted into coordinate information in three-dimensional space according to the projection relationship. Based on the smoke flow direction and convergence point detected by multiple high-definition cameras, the motion vector of the smoke and the range of the convergence point are drawn in the three-dimensional space. Based on the fire source direction obtained by reverse calculation from the smoke flow direction and the position information of the convergence point, a comprehensive analysis is performed to estimate the position of the fire source in three-dimensional space, and the position of the fire source in three-dimensional space is marked on the corresponding position of the three-dimensional model of the electrical equipment area. Once the fire source is marked in the 3D model of the electrical equipment area, the alarm device is immediately triggered, sending the 3D model of the electrical equipment area with the fire source location marked to the user's terminal, and performing an emergency power cut-off.

2. The method for monitoring the operation of electrical equipment based on flame and smoke video analysis according to claim 1, characterized in that: Based on each of the aforementioned key monitoring locations, high-definition cameras are installed to conduct viewing angle tests; The viewing angle test includes: analyzing the viewing angle range covered by each of the high-definition cameras, the size of the monitored area, and the space and layout around the electrical equipment to ensure that the high-definition cameras cover all key areas and that the images of the key areas are clear and without blind spots; If the perspective test reveals blind spots or unclear images in the key area, adjust the installation angle or position of the high-definition camera or increase the number of high-definition cameras.

3. The method for monitoring the operation of electrical equipment based on flame and smoke video analysis according to claim 1, characterized in that: The video of the electrical equipment area is processed into frames of images to form video frames; Gaussian filtering is applied to the video frames to remove high-frequency noise and smooth the image. Using a Gaussian mixture model, multiple Gaussian distributions are established for each pixel's color or brightness value. Based on these multiple Gaussian distributions, background modeling is performed on each pixel, with different Gaussian distributions representing different states in the background. When the difference between the pixel value in the background and the Gaussian distribution in the model exceeds a set threshold, the pixel value will be marked as a suspected flame or smoke area.

4. The method for monitoring the operation of electrical equipment based on flame and smoke video analysis according to claim 1, characterized in that, When identifying flames in a video, the flame region is extracted based on the suspected flame or smoke area, including: Based on the suspected flame or smoke area, color feature analysis is performed to obtain the color information of the flame, and it is marked as the first verified flame area; based on the suspected flame or smoke area, edge detection is performed to obtain the outline information of the flame, and it is marked as the second verified flame area; combining the first verified flame area and the second verified flame area, the flame area is analyzed and extracted.

5. The method for monitoring the operation of electrical equipment based on flame and smoke video analysis according to claim 4, characterized in that: The color feature analysis includes: The suspected flame or smoke area is converted from the RGB color space to the HSV color space. The HSV color space is based on hue, saturation, and lightness. By setting a range of hue, saturation, and lightness, the HSC value of each pixel in the suspected flame or smoke area is compared with the range of hue, saturation, and lightness. If the HSV value of a pixel is within the range of hue, saturation, and lightness, then the pixel is marked as the first verified flame area. The edge detection includes: The suspected flame or smoke area is converted from an RGB color image to a grayscale image; The grayscale image is Gaussian smoothed. The Gaussian smoothing process involves constructing a Gaussian kernel centered on each pixel in the grayscale image and performing a convolution operation between the Gaussian kernel and the pixels of the grayscale image to achieve Gaussian smoothing of the grayscale image and reduce the impact of noise. Calculate the gradient magnitude and gradient direction of each pixel in the Gaussian smoothed grayscale image. The gradient magnitude represents the degree of drastic change in pixel value in the grayscale image, and the gradient direction represents the direction of the fastest change in pixel value. Based on the gradient direction θ1 of pixel s1 in the grayscale image, determine the two adjacent pixels s0 and s2 of the pixel in the grayscale image along the gradient direction, and compare the gradient magnitude G1 of pixel s1 with the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2; when the gradient magnitude G1 of pixel s1 in the grayscale image is greater than the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2, then retain the gradient magnitude G1 of pixel s1 in the grayscale image; when the gradient magnitude G1 of pixel s1 in the grayscale image is less than either the gradient magnitudes G0 and G2 of the two adjacent pixels s0 and s2, then set the gradient magnitude G1 of pixel s1 in the grayscale image to 0. Set a high threshold T h and low threshold T l Used to detect edge pixels; when the gradient magnitude of a pixel in the grayscale image is greater than a high threshold T h When the gradient magnitude of a pixel in the grayscale image is less than a low threshold T, it is determined to be a pixel with a strong edge. l If the gradient magnitude of a pixel in the grayscale image is between a high threshold T and an edge pixel, then the pixel in the grayscale image is determined to be a non-edge pixel. h and low threshold T l When the strong edge pixel is connected to the strong edge pixel, the weak edge pixel adjacent to the strong edge pixel is checked. When the adjacent weak edge pixel is connected to the strong edge pixel, the adjacent weak edge pixel is marked as an edge pixel. The edge connection is completed to obtain clear flame outline information, and the pixel area in the flame outline information is marked as the second verification flame area. Perform an intersection operation on the pixels of the first verification flame region and the second verification flame region; when a pixel is simultaneously a pixel in both the first verification flame region and the second verification flame region, it is determined to be a pixel in the flame region; extract all pixels that are simultaneously in both the first verification flame region and the second verification flame region to form the flame region.

6. The method for monitoring the operation of electrical equipment based on flame and smoke video analysis according to claim 1, characterized in that: The step of calculating the position of the fire source in three-dimensional space based on the flame area, and marking the position of the fire source in three-dimensional space onto the corresponding position in the three-dimensional model of the electrical equipment area, includes: When only one high-definition camera detects the flame area, the location of the fire source in three-dimensional space is calculated based on the installation position and viewing angle of the high-definition camera and the three-dimensional model of the electrical equipment area. When multiple high-definition cameras detect the flame area, the location of the fire source is jointly estimated by utilizing the differences in the viewing angle and position of each camera and through spatial geometric relationships. The location of the fire source in three-dimensional space is marked on the corresponding position of the three-dimensional model of the electrical equipment area.

7. An electrical equipment operation monitoring system based on flame and smoke video analysis, applied to the electrical equipment operation monitoring method based on flame and smoke video analysis as described in any one of claims 1-6, characterized in that, The system includes: a 3D model module, a flame or smoke detection module, a fire source location module, and a fire source alarm module; The 3D model module includes a perspective testing module, a video acquisition module, and a model construction module. The model construction module performs a preliminary assessment of the electrical equipment area and constructs a 3D model of the electrical equipment area. The perspective testing module identifies key monitoring locations based on the model construction module and performs perspective testing by installing high-definition cameras to ensure that each key monitoring area is clearly covered by the high-definition cameras. The video acquisition module acquires video data of the electrical equipment area in real time through the installed high-definition cameras. The flame or smoke detection module includes a flame or smoke extraction module, a flame verification module, and a smoke verification module. The flame or smoke extraction module uses a Gaussian mixture model to analyze the color or brightness changes of each pixel and detect differences from the background to determine if a suspected flame or smoke area exists. The flame verification module extracts the suspected flame or smoke area through color feature analysis and edge detection, and analyzes it in conjunction with a first verified flame area and a second verified flame area to ultimately confirm the accurate location of the flame area. The smoke verification module generates a first smoke verification area through grayscale threshold segmentation, obtains a second smoke verification area through edge detection and contour detection, and further analyzes the motion characteristics of the smoke using optical flow to obtain a third smoke verification area. Combining the second and third smoke verification areas confirms the accurate location of the smoke area. The fire source location module includes a flame fire source location module and a smoke fire source location module; the flame fire source location module calculates the fire source location based on one or more cameras; the smoke fire source location module calculates the fire source location based on the shape, flow direction and convergence point of the smoke. The fire source alarm module displays a 3D model with the fire source location marked to the user through the terminal interface, and provides real-time monitoring, alarm and operation feedback functions.

Images