Fire-fighting equipment detection early warning method and system based on visual camera

Abstract

The invention belongs to the technical field of fire safety, and discloses a fire-fighting equipment detection early warning method and system based on a visual camera, and the method comprises the following steps: obtaining multi-focal-length image frames of a fixed spray head and a fire extinguisher edge region, judging the continuous change of gray scale in the continuous image frames, and carrying out the detection early warning of the fire-fighting equipment based on the visual camera. Comparing the gray level distribution difference under the long focus and the short focus, identifying the dynamic trajectory of the contour line angular point of the fire-fighting equipment association structure, analyzing the delay of the function action response, marking as an abnormal region, and generating a fire-fighting equipment collaborative early warning information set; according to the invention, by extracting and analyzing the multi-focal-length image frames of the edge areas of important facilities such as a fixed spray head and a fire extinguisher, fine changes of the fire-fighting facilities under differentiated focal lengths can be captured, working states and potential faults of the fire-fighting facilities can be judged, functional abnormalities caused by equipment aging or damage can be recognized in advance, early warning can be carried out in real time, and the safety of the fire-fighting facilities can be improved. And a higher guarantee level is brought to fire safety.

Description

Technical Field

[0001] The present invention belongs to the technical field of fire safety, and specifically, relates to a fire-fighting facility detection and early warning method and system based on a vision camera. Background Art

[0002] The technical field of fire safety includes the research and application of fire prevention and control, fire alarm, fire extinguishing equipment and related safety technologies. The core contents of this field include the design and innovation of fire-fighting equipment, fire detection and early warning systems, fire emergency response measures and their automated management technologies, etc.; fire safety technology aims to improve the emergency response speed in case of a fire through effective technical means and reduce casualties and property losses; specific technologies involve fire early warning devices, automatic fire extinguishing systems, fire monitoring and alarm technologies, fire impact assessment, etc.; with the continuous development of technology, fire safety technology is increasingly developing towards the direction of intelligence, automation and precision, and improving the effectiveness and response speed of fire-fighting facilities through image recognition, sensor technology and networked management.

[0003] In the prior art, it is necessary to regularly inspect and maintain fire-fighting facilities to ensure their stable operation. The existing maintenance of fire-fighting facilities is carried out by maintenance personnel to detect each fire-fighting facility one by one, and the detection process is cumbersome and the labor intensity is high. To solve the above technical problems, there is also a method in the prior art that uses a vision camera to collect video images of fire-fighting facilities for detection and early warning.

[0004] The Chinese invention patent with the patent application number: CN202410495687.4 discloses a fire-fighting equipment detection and evaluation system and method based on video processing and deep learning, including the following steps: collecting multi-source data of fire-fighting equipment, extracting the attributes of fire-fighting equipment, the topological relationship of fire-fighting equipment, and the key frame images of fire-fighting equipment based on the multi-source data; constructing a graph structure space based on the multi-source data, constructing and training a multi-channel graph neural network model for the graph structure space; outputting the graph node feature representation and the risk level probability; splicing the node feature representations into a high-dimensional vector, constructing a high-dimensional manifold space, and performing manifold learning and dimensionality reduction. Based on the dimensionality-reduced manifold space, analyzing the state evolution trajectory of the fire-fighting equipment, and preliminarily analyzing the equipment anomaly situation through the state evolution trajectory; constructing a risk assessment model, giving a comprehensive risk score for each fire-fighting equipment, and outputting a hierarchical maintenance plan for the fire-fighting equipment according to the comprehensive risk score.

[0005] The existing fire-fighting facility detection methods of this type monitor fire-fighting facilities by using visual cameras to identify the working status of the facilities and existing hidden dangers, mainly involving the real-time status detection and early warning mechanism of fire-fighting facilities. Image acquisition is carried out through visual cameras, and then the condition of the facilities is analyzed through image processing technology; this early warning method specifically includes dynamically monitoring fire-fighting facilities, automatically identifying whether there are defects or damages in the facilities, and judging whether they are in a normal working state; this early warning method takes computer vision technology as the core, realizes the detection and early warning functions of fire-fighting facilities through the analysis of visual images, and combines image processing technology to timely discover potential safety risks for effective early warning.

[0006] The existing fire safety technologies still have deficiencies in real-time status detection and precise early warning. Traditional methods focus on basic fire detection and alarm systems, which are triggered only when a fire has occurred, and have weak capabilities for preventive maintenance and early fault diagnosis of equipment; for example, conventional fire monitoring systems cannot effectively identify the damage or functional decline of fire-fighting facilities themselves, and can only respond passively until the equipment completely fails. This delay can lead to uncontrollable consequences in emergency situations; the existing technologies also appear insufficient in dealing with the details of image data, unable to perform in-depth image analysis and structural evaluation, restricting their performance in early warning accuracy and response speed; the limitations of this technology are particularly obvious when dealing with complex or concealed fire scenarios, resulting in the reaction of the early warning system not being timely or accurate enough, increasing safety risks. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a fire-fighting facility detection and early warning method and system based on a visual camera, which is used to solve the deficiencies of the existing fire-fighting facility detection methods in real-time status detection and precise early warning, and also have deficiencies in dealing with the details of the collected image data, unable to perform in-depth image analysis and structural evaluation, restricting their performance in early warning accuracy and response speed.

[0008] To solve the above technical problems, the present invention provides the following technical solutions: The fire-fighting facility detection and early warning method based on a visual camera includes the following steps: S1: Obtain multi-focal image frames of the edge regions of fixed sprinkler heads and fire extinguishers, extract edge pixel points and the gray-scale values of the surrounding area groups, judge whether there is continuous gray-scale change in consecutive image frames, mark it as an edge blur area, and obtain an image occlusion area information set; S2: Based on the image occlusion area information set, select the fire extinguisher nozzle area and the sprinkler head water outlet area as the image reverse comparison objects, compare the gray-scale distribution differences in the long-focus and short-focus states, and generate a focal length difference response early warning point set; S3: Invoke the warning point set for focal length difference response, extract the corner points of the contour line of the fire-fighting facility associated structure, measure the pixel distance from the corner points to the visual center reference point, identify the oscillation behavior of the dynamic trajectory of the corner points, and generate an initial judgment point series of structural deformation; S4: Based on the initial judgment point series of structural deformation, extract the boundary contours of the pressure gauge pointer scale area and the emergency release valve pull ring area in consecutive image frames, calculate the closure index and pixel aggregation index in consecutive image frames, compare the time coincidence of the occlusion period and the response elongation interval, mark it as an abnormal area, and generate an occlusion lag correlation monitoring label set.

[0009] The following is a further optimization of the above technical solution by the present invention: The image occlusion area information set includes an edge blur feature map, a blur area coordinate set, and a blur duration index; the warning point set for focal length difference response includes a response offset vector, a focal length difference level, and a contrast gray difference curve; the initial judgment point series of structural deformation includes an oscillation corner point sequence, a pixel offset path, and a deformation amplitude index; the occlusion lag correlation monitoring label set includes an occlusion duration section, a linkage anomaly matching degree, and a response delay statistic item.

[0010] Further optimization: The specific steps for obtaining the image occlusion area information set are as follows: S101: Obtain multi-focal length image frames of the edge areas of fixed sprinklers and fire extinguishers, extract the pixel point sets corresponding to the edge areas in each frame of the image, and invoke the single-pixel gray values within the areas covered by the pixel point sets to obtain the gray distribution set of the edge areas in the consecutive image frame sequence. Classify the gray values according to the pixel coordinate positions to generate an edge gray distribution value set; S102: Based on the edge gray distribution value set, invoke the differential time series gray values under the same edge pixel coordinates, perform a difference operation on each group of gray values to obtain the gray difference sequence between consecutive image frames, and judge whether there is a continuous change in gray within the area according to the change amount of the single-frame gray difference and the number of consecutive changing frames to generate an edge gray fluctuation trend; S103: According to the edge gray fluctuation trend, screen the pixel coordinate groups with the number of consecutive image frames of gray change exceeding the threshold, mark them as edge blur areas, and perform a merging process on the coordinates and coverage ranges of the edge blur areas to obtain the image occlusion area information set.

[0011] Further optimization: The specific steps for obtaining the warning point set for focal length difference response are as follows: S201: Invoke the coordinates of the edge blur areas in the image occlusion area information set, extract the gray value sets at the same positions in the corresponding long-focus and short-focus image frames, and respectively aggregate the gray responses of the pixels within the areas to obtain the gray contrast value set of the occlusion areas; S202: Based on the gray - scale contrast value set of the occluded area, screen the pixel groups located in the fire extinguisher nozzle area and the sprinkler head water outlet area, calculate the gray - scale difference of the same area under two focal lengths, mark the corresponding points as early - warning positions, and combine the focal - length difference and coordinate information to obtain the focal - length difference response early - warning point set; The formula for calculating the gray - scale difference of the same area under two focal lengths is as follows: ; Among them, represents the gray - scale response difference eigenvalue of the area under the focal length and the focal length , represents the total number of pixels included in the area , represents the gray - scale value of the rd pixel in the area under the focal length represents the gray - scale value of the th pixel in the area under the focal length represents the horizontal coordinate value of the th pixel in the area under the focal length represents the horizontal coordinate value of the th pixel in the area under the focal length represents the vertical coordinate value of the th pixel in the area under the focal length represents the vertical coordinate value of the th pixel in the area under the focal length represents the adjustable compensation factor for the horizontal displacement error.

[0012] Further optimization: The specific steps for obtaining the initial judgment point sequence of structural deformation are as follows: S301: Call the image coordinate area in the focal - length difference response early - warning point set, extract the contour line edges of the corresponding fire - fighting facilities in each image coordinate area, locate the image pixel mutation points at the boundary corners through the edge pixel gradient change, and use the positions with angle changes in each group of image pixel mutation points as contour corner points to generate the structural contour corner point coordinate set; S302: Based on the set of structural contour corner coordinates, with the image center position as the visual center reference point, measure the pixel distances between the corner points and the visual center reference point, and arrange the pixel distances in the order of the image frame sequence to obtain the corner pixel distance sequence; S303: According to the corner pixel distance sequence, calculate the pixel displacement change amount of the corner points between consecutive image frames, extract the corner point sequence with the pixel displacement change amount exceeding the set oscillation amplitude threshold, identify the oscillation behavior in the image frame sequence and mark the corresponding frame positions and corner point numbers to generate the initial judgment point sequence of structural deformation.

[0013] Further optimization: The specific steps for obtaining the occlusion lag correlation monitoring tag set are as follows: S401: Invoke the initial judgment point sequence of structural deformation, extract the boundaries of the pressure gauge pointer scale area and the emergency release valve pull ring area in each frame of the image, calculate the gradient jump of the boundary pixels and the pixel distribution density of the contour closed area to obtain the boundary feature response value, and generate the boundary closure index value set; S402: Based on the corresponding region coordinates in the boundary closure index value set, measure the aggregation index of the pixel gray value distribution within the same closed area, and generate the pixel aggregation index sequence by calculating the proportion of the number of pixel points where the pixel gray values in the area converge to the mean value; S403: According to the pixel aggregation index sequence, screen the time periods that meet the occlusion state of consecutive image frames, retrieve the self-check response signals of the corresponding image frames, and refer to the signal trigger frame position and the response action completion frame position to obtain the functional action response delay value; S404: Based on the response elongation time interval marked in the functional action response delay value, compare whether there is a coincidence in frame numbers between the time range and the frame time period corresponding to the occlusion state of consecutive image frames. If both time overlaps are established at the same time, mark the region coordinates as abnormal regions to generate the occlusion lag correlation monitoring tag set.

[0014] Further optimization: The formula for calculating the gradient jump of the boundary pixels and the pixel distribution density of the contour closed area to obtain the boundary feature response value is: ; Where, represents the boundary feature response value, represents the total number of boundary pixels in the real-time frame, represents the gray value jump of the a-th boundary pixel in the gradient direction, represents the pixel distance from the a-th boundary pixel to the centroid point in the neighborhood, represents the pixel density within the closed contour with the a-th pixel as the center area, represents the average value of the pixel density within the contour in the real-time image frame, Represents the local boundary intensity value within the neighborhood of the a-th boundary pixel.

[0015] Further optimization: The method further includes step S5: S5: Invoke the occlusion hysteresis correlation monitoring tag set, compare the image occlusion range, structural oscillation frequency, and response delay duration of the abnormal area, perform classification recognition and level determination, label them as early warning prompts, key attention, and emergency response levels, and generate a fire-fighting facility collaborative early warning information set; The fire-fighting facility collaborative early warning information set includes an early warning level identifier, an abnormal source pointing tag, and a collaborative response situation.

[0016] Further optimization: The specific steps for obtaining the fire-fighting facility collaborative early warning information set are as follows: S501: Invoke the coordinates of the abnormal area marked in the occlusion hysteresis correlation monitoring tag set, extract the image occlusion range of the corresponding area, calculate the occlusion coverage rate by the ratio of the number of occluded pixels to the total number of pixels in the area, retrieve the structural oscillation frequency and functional response delay frame length data at the associated position, and generate a regional feature index group; S502: Based on multiple values in the regional feature index group, set three grading intervals for the image occlusion coverage rate, oscillation frequency, and response delay frame length, respectively divide each index into different risk sections, and perform classification mapping according to the combination of the sections to which the three indexes belong, analyze the risk category label of the abnormal area, and obtain the linkage risk level identifier value; S503: According to the linkage risk level identifier value, label the abnormal area as three grade categories of early warning prompt, key attention, and emergency response respectively, and at the same time integrate the image coordinates, time frame numbers, and corresponding risk factors under each grade to generate a fire-fighting facility collaborative early warning information set.

[0017] The present invention also provides a fire-fighting facility detection and early warning system based on a vision camera for performing the above-mentioned fire-fighting facility detection and early warning method based on a vision camera. The system includes: The image acquisition module acquires multi-focus image frames of the edge areas of fixed sprinklers and fire extinguishers, extracts the edge pixel points of the nozzle edge area and the water outlet hole contour, extracts the gray value sequence of the area around the edge in consecutive image frames, identifies the continuously changing state of the gray level, and obtains an image occlusion area information set; The edge occlusion recognition module, based on the image occlusion area information set, extracts the gray response areas in the long-focus and short-focus image frames, calculates the gray difference, and filters out the nozzle and water outlet hole areas that exceed the gray response threshold, and obtains a focal length difference response early warning point set; The focal length response analysis module calls the focal length difference response warning point set, obtains the corner coordinates of the edge line of the nozzle structure in the corresponding image frame, calculates the pixel distance value sequence from the corner to the center point based on the image visual center point coordinates and each corner coordinate, compares the distance changes of each corner in consecutive image frames, and obtains the initial judgment point column of structural deformation; The structure dynamic monitoring module extracts the boundary contours of the pressure gauge scale pointer and the emergency release valve pull ring area according to the initial judgment point column of structural deformation, calculates the boundary closure value and the pixel aggregation value in consecutive image frames, and obtains the occlusion lag correlation monitoring label set; The linkage warning module calls the occlusion lag correlation monitoring label set, compares the self-check signal trigger period and the structure action response period in the image frame, calculates the coincidence duration of the two, and performs classification and recognition in combination with the occlusion range, oscillation frequency and response delay time to obtain the fire-fighting facility collaborative warning information set.

[0018] The present invention adopts the above technical solutions and has at least the following beneficial effects: 1. By acquiring and analyzing multi-focal length image frames, the present invention realizes more detailed observation and evaluation of the state of fire-fighting facilities. Extracting and analyzing multi-focal length image frames of the edge areas of important facilities such as fixed sprinkler heads and fire extinguishers helps to capture the subtle changes of fire-fighting facilities at different focal lengths, such as the continuous change of gray values, and judge their working states and potential faults.

[0019] 2. In actual operation, the present invention can identify in advance the functional abnormalities caused by equipment aging or damage, give early warnings in real time, and reduce the fire risk caused by equipment failures; the dynamic tracking of structural deformation and the analysis of consecutive image frames of boundary contours further improve the monitoring ability of the integrity of fire-fighting facilities. Through precise image processing and analysis, the response speed and accuracy of fire safety are enhanced, bringing a higher level of guarantee for fire safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of the working process of the present invention; Figure 2 is a system flow chart of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0022] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations; any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or more advantageous than other embodiments or design solutions; rather, the use of the word "example" is intended to present concepts in a specific manner; in addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either of the two can be selected.

[0023] Please refer to Figure 1 , the embodiments of the present invention provide a fire-fighting facility detection and early warning method based on a vision camera. The processing flow of this method specifically includes the following steps: S1: Obtain multi-focal image frames of the edge areas of fixed sprinkler heads and fire extinguishers, extract edge pixel points and the gray-scale values of the surrounding area groups, determine whether there is a continuous change in gray-scale in consecutive image frames, mark it as an edge blur area, and obtain an image occlusion area information set; S2: Based on the image occlusion area information set, extract the gray-scale response areas at corresponding positions in the differential focal length image frames, select the fire extinguisher nozzle area and the sprinkler head water outlet area as the image reverse comparison objects, compare the gray-scale distribution differences in the long-focus and short-focus states, and generate a focal length difference response early warning point set; S3: Call the focal length difference response early warning point set, extract the corner points of the contour lines of the fire-fighting facility associated structures, measure the pixel distances from the corner points to the visual center reference point, identify the oscillation behavior of the dynamic trajectories of the corner points, and generate a preliminary structural deformation point sequence; S4: Based on the preliminary structural deformation point sequence, extract the boundary contours of the pressure gauge pointer scale area and the emergency release valve pull ring area in consecutive image frames, calculate the closure index and the pixel aggregation index in consecutive image frames, determine whether they are lower than the integrity reference value. If the continuous occlusion state is established, then retrieve the self-inspection response signal, analyze the delay situation of the functional action response, and compare the time coincidence of the occlusion period and the response elongation interval. If both exist, mark it as an abnormal area and generate an occlusion lag associated monitoring label set; S5: Call the occlusion lag associated monitoring label set, compare the image occlusion range, the structural oscillation frequency and the response delay duration of the abnormal area, perform classification recognition and level determination, label it as the early warning prompt, key attention and emergency response levels, and generate a fire-fighting facility collaborative early warning information set.

[0024] The image masking area information set includes an edge blur feature map, a blurred area coordinate set, and a blur duration index; the focal length difference response warning point set includes a response offset vector, a focal length difference level, and a contrast gray difference curve; the initial judgment point sequence of structural deformation includes an oscillation corner point sequence, a pixel offset path, and a deformation amplitude index; the masking lag correlation monitoring label set includes a masking duration section, a linkage anomaly matching degree, and a response delay statistic item; the fire-fighting facility collaborative warning information set includes a warning level identifier, an abnormal source pointing label, and a collaborative response situation.

[0025] The steps for obtaining the image masking area information set are specifically as follows: S101: Obtain multi-focal image frames of the edge areas of fixed sprinklers and fire extinguishers, extract the set of pixel points corresponding to the edge areas in each frame of the image, and call the single-pixel gray values within the area covered by the set of pixel points to obtain the gray distribution set of the edge areas in the continuous image frame sequence. Classify the gray values according to the pixel coordinate positions to generate an edge gray distribution value set.

[0026] To obtain multi-focal image frames of the edge areas of fixed sprinklers and fire extinguishers, in actual operation, an image acquisition device needs to be set up. An industrial camera with three-axis adjustment is fixed on the factory building ceiling, and the angle is adjusted to ensure that the camera can completely cover the edge of the object to be detected. Different focal length lenses are selected to achieve multi-focal imaging. For the positions of the sprinklers and fire extinguishers, the detection area ROI is set using image processing software. The image sensor is used to collect and number the image sequences taken at each focal length. In each frame of the image, the edge pixel contour is extracted using an edge detection algorithm. The edge detection intensity parameter is set in the software, and the appropriate edge extraction strategy is determined by comparing the edge sharpness under different parameters. Taking the outer edge of the fire extinguisher bottle mouth as an example, the pixel edge is quickly located by detecting the change in its edge gray value. The edge pixel points within this area are uniformly extracted according to the image frame number and a coordinate index table is constructed. Then, the edge coordinates are mapped to the corresponding images according to the image frame order. The gray value of each pixel position is extracted to form a gray record set with coordinates as the index and the frame sequence as the time dimension. Combining the sliding window method, a 5×5 area around each pixel is set for gray aggregation to reduce noise interference, and the complete edge gray distribution value set is formed by grouping and summarizing according to the positions.

[0027] S102: Based on the edge gray distribution value set, call the differential time series gray values under the same edge pixel coordinates, perform a difference operation on each group of gray values to obtain the gray difference sequence between consecutive image frames, and judge whether there is a continuous change in gray value within the area according to the change amount of the single-frame gray difference and the number of consecutive changing frames, and generate an edge gray fluctuation trend.

[0028] It is necessary to further determine whether there is a continuous grayscale change phenomenon in the area. In actual operation, it is necessary to analyze the changes between continuous image frames from the grayscale values ​​of each group of pixels. By comparing the image frame numbers, compare whether the grayscale values ​​of the same pixel in each pair of adjacent image frames are different, record the change amplitude, and form a grayscale difference list in sequence. Observe the change trend of each group of grayscale difference data. A fixed difference threshold can be set in the software, and a grayscale change reference value can be set manually or automatically, set to 3 grayscale levels, and set a continuous change frame number threshold, such as 5 frames. When making a judgment, if it is found that multiple consecutive grayscale differences are greater than the reference value, it means that there is a continuous change in the area. By labeling and saving the pixels that meet the conditions, recording their continuous change frames and maximum grayscale difference, it not only reflects the change of each pixel in the image sequence, but also records its start change frame and end frame number, which is convenient for subsequent fuzzy recognition processing. This operation can be automated with the help of image analysis software, and grayscale comparison and marking actions can be executed in batches through rule scripts to ensure that the change data of each pixel is completely saved and generate edge grayscale fluctuation trends.

[0029] S103: According to the edge grayscale fluctuation trend, the pixel coordinate groups whose continuous image frames with grayscale changes exceed the threshold are screened and marked as edge blurred areas. The coordinates and coverage of the edge blurred areas are merged to obtain an image masked area information set.

[0030] Screening and classifying the change information is a key step in identifying fuzzy areas. During the operation, the pixel points that have abnormal fluctuations are first determined based on the set continuous image frame number threshold. During the screening process, all pixel grayscale trend data will be traversed. As long as the grayscale difference of a pixel point within the set frame number is higher than the set grayscale reference standard, the point will be marked as a fuzzy candidate point. The candidate points are summarized and the clustering algorithm in the image analysis tool is used to merge the spatially close pixels. Specifically, the distance between each two points is calculated based on the image coordinates, and the points less than a certain distance threshold are classified into an area to generate a fuzzy edge group. The outer boundary coordinates of the group are then uniformly framed to form a continuous rectangular masked area, and the position coordinates and size range of the area are recorded. Each masked area is assigned an independent number, which can be used for subsequent image restoration or preprocessing before image recognition. In the image processing software, the area can be marked by a mask layer so that it does not affect the image analysis and processing of the main target, and the image masked area information set is obtained.

[0031] The specific steps for obtaining the focal length difference response warning point set are as follows: S201: Call the coordinates of the edge blur area in the image masked area information set, extract the grayscale value set at the same position in the corresponding long-focus and short-focus image frames, and aggregate the grayscale responses of the pixels in the area respectively to obtain the masked area grayscale contrast value set.

[0032] In actual processing, the coordinate information of each rectangular area stored in the masked area data set is read, including the coordinates of the upper left corner and the width and height dimensions, and then the areas are mapped one by one to the corresponding short-focus and long-focus image frames. The image frame number must be consistent with the image sequence based on which the masked area is generated to ensure that the contrast value has temporal consistency. The pixel grayscale value in the area is selected in the short-focus image, and the grayscale matrix is ​​constructed according to the pixel coordinates and the local average operation is performed to aggregate the grayscale response in the area. At the same time, the grayscale matrix of the same area is extracted in the long-focus image for the same processing. The short-focus image is characterized by a wide viewing angle and weak edge features, and its grayscale response fluctuation is small, while the long-focus image can reflect more details and the grayscale fluctuation is more intense. By respectively aggregating the pixel grayscale responses of the same masked area in the two focal length images, a set of grayscale contrast value sets with spatial consistency and focal length difference characteristics is obtained. In the image analysis software, each masked area can be set as an operation block, and the masked coordinates and image frame matching extraction area are called cyclically to complete the grayscale aggregation process and uniformly output the grayscale contrast data table of the masked area to obtain the masked area grayscale contrast value set.

[0033] S202: Based on the grayscale contrast value set of the masked area, the pixel group located in the nozzle area of ​​the fire extinguisher and the water outlet area of ​​the sprinkler head is screened, the grayscale difference of the same area under two focal lengths is calculated, the corresponding points are marked as warning positions, and the focal length difference and coordinate information are combined to obtain the focal length difference response warning point set.

[0034] The formula for calculating the grayscale difference of the same area at two focal lengths is as follows: ; in, Representative area In focal length With focal length The grayscale response difference eigenvalue under Representative area The total number of pixels contained in Representative area Middle pixels at focal length The gray value below Representative area Middle pixels at focal length The gray value below Representative area Middle pixels at focal length The horizontal coordinate value under Representative area Middle pixels at focal length The horizontal coordinate value under The representative region The -th pixel in the representative region has a longitudinal coordinate value under the focal length The representative region The -th pixel in the representative region has a longitudinal coordinate value under the focal length represents an adjustable compensation factor for the horizontal displacement error.

[0035] Meaning of parameters and derivation process of formula calculation: For an image region numbered in a certain area, according to the real-time data acquisition result of the video monitoring device, the total number of pixels contained in the area is In the focal length parameter, the focal length of the acquired image is mm and mm, and the pixel gray value and pixel coordinate data are respectively obtained after being acquired by the image gray matrix and the coordinate transformation matrix; For the -th pixel, the following data are given respectively: The 1st pixel: , ; , ; , ; The 2nd pixel: , ; , ; , ; The 3rd pixel: , ; , ; , ; The 4th pixel: , ; , ; , ; Parameter Set to 0.8, and according to the weighted fitting result obtained from the statistical analysis of the change range of the grayscale value with the horizontal pixel displacement error in the previous experimental tests, it fluctuates within the range of 0.5 to 1.0 and increases as the correlation between the focal length difference and the horizontal displacement strengthens; Substitute into the formula item by item and calculate as follows: The first item: ; ; ; Expression value: ; The second item: ; ; ; Expression value: ; The third item: ; ; ; Expression value: ; The fourth item: ; ; ; Expression value: ; Sum the four items, average them, and then take the absolute value: ; This result shows that in the case of 35mm and 50mm focal lengths in Region 3, the grayscale response difference eigenvalue is 6.835, belonging to a medium grayscale change response level. This value will be used to mark the warning level of the pixel set in this region, further classified into the focal length difference response warning point set, and combined with the corresponding coordinate positions for the positioning calculation of the risk area.

[0036] The specific steps for obtaining the initial judgment point sequence of structural deformation are as follows: S301: Call the image coordinate area with concentrated focal length difference response warning points, extract the contour edges of the corresponding fire-fighting facilities in each image coordinate area, locate the image pixel mutation points at the boundary corners through the edge pixel gradient change, and use the positions with angle changes in each group of image pixel mutation points as contour corner points to generate a set of structural contour corner point coordinates.

[0037] In actual operation, identify and load the image frames and their local areas corresponding to each warning point, and then perform contour extraction processing on the target fire-fighting facilities in the area through the image analysis module. For the metal frame of the sprinkler head or the edge structure of the fire extinguisher bottle body, use the edge enhancement algorithm to highlight the pixel positions with significant gray-scale gradient changes, execute the contour line tracking algorithm based on the gradient change to obtain a set of continuous edge pixel points, find the pixel positions with a large mutation amplitude of the gray-scale gradient in the edge line, mark the boundary mutation points according to the local slope change trend of the pixel gray-scale. The mutation points often appear at the structural broken lines or turning points, such as the connection part between the nozzle and the bottle body of the fire extinguisher or the edge of the fixed buckle of the sprinkler head. Further screen the positions with significant angle changes as key corner points according to the angle relationship formed by each group of image pixel mutation points. Use a moving window to traverse the included angle relationship formed by three consecutive points in the edge segment, and determine that the turning point with an included angle less than the set angle value, such as 120 degrees, is a valid contour corner point. The corner points can be highlighted in the original image in the form of cross coordinates through the image marking tool, and at the same time, output a structured data record table, including information such as the corner point position coordinates, the frame number to which it belongs, the area identifier, and the corresponding device number, to generate a set of structural contour corner point coordinates.

[0038] S302: Based on the set of structural contour corner point coordinates, use the image center position as the visual center reference point, measure the pixel distance between the corner points and the visual center reference point, and arrange the pixel distances in the order of the image frame sequence to obtain a sequence of corner point pixel distances.

[0039] To further identify signs of structural deformation, the center point of the image is set as the visual reference center. This center point is the central pixel coordinate position of the image resolution. For example, if the image is 1920×1080, the center point is (960, 540). For each contour corner point, calculate its pixel distance in the image, that is, the Euclidean distance value from the corner point to the center point. Traverse all frames in the image sequence, calculate the distance for the pixel positions corresponding to the same corner point number in consecutive image frames, and arrange them in the order of frame numbers to form a complete pixel distance sequence. This sequence can reflect the relative position fluctuations of the corner points in different image frames. Through this sequence, it can be intuitively presented whether there is a relative displacement phenomenon caused by structural offset or picture vibration for each corner point. For example, the distance between the corner point at the edge of the fire extinguisher bottle mouth and the center point is 315 pixels in the initial frame. If it shows an increasing trend frame by frame in subsequent frames, it indicates that there is a displacement in the image. Then, uniformly number and record the distance sequences of all corner points to obtain the corner point pixel distance sequence.

[0040] S303: According to the corner point pixel distance sequence, calculate the pixel displacement change amount between consecutive image frames for the corner points, extract the corner point sequences whose pixel displacement change amount exceeds the set oscillation amplitude threshold, identify the oscillation behavior in the image frame sequence, and mark the corresponding frame positions and corner point numbers to generate a preliminary judgment point list for structural deformation.

[0041] Analyze whether the pixel distances of each corner point in consecutive image frames change violently. Number each corner point, traverse its distance sequence, and calculate the displacement change amount between adjacent frames, that is, the absolute difference in distance values between two consecutive frames. When this change value exceeds the set oscillation amplitude threshold, such as 10 pixels, it indicates that there is a significant position change for this corner point. The system will extract the corner points with violent changes and their corresponding frame numbers, and mark them as suspected oscillation points. This process is particularly suitable for identifying image change phenomena caused by changes in shooting angles, shaking of facilities, or automatic focusing of the lens. Record the corner point numbers that meet the oscillation criteria, their in-frame positions, and the change amplitudes, and output them as a preliminary judgment point list for structural deformation. The data is presented in a list form, including fields such as frame numbers, corner point coordinates, forward and backward displacement values, and oscillation amplitude levels, which can be used for subsequent deformation confirmation and image stability diagnosis processes. If necessary, the image visualization interface can also be called to synchronously highlight the identified oscillation corner points to form a preliminary judgment point list for structural deformation.

[0042] The specific steps for obtaining the occlusion lag correlation monitoring tag set are as follows: S401: Call the preliminary judgment point list for structural deformation, extract the boundaries of the pressure gauge pointer scale area and the emergency release valve pull ring area in each frame of the image, calculate the gradient jump of the boundary pixels and the pixel distribution density of the contour closed area to obtain the boundary feature response values, and generate a set of boundary closure index values.

[0043] The formula for obtaining the boundary feature response value by calculating the gradient jump of boundary pixels and the pixel distribution density of the contour closed area is as follows: ; where, represents the boundary feature response value, represents the total number of boundary pixels in the real-time frame, represents the gray-level jump value of the a-th boundary pixel in the gradient direction, represents the pixel distance from the a-th boundary pixel to the centroid point in the neighborhood, represents the pixel density within the closed contour centered on the a-th pixel, represents the average value of the pixel density within the contour in the real-time image frame, represents the local boundary intensity value within the neighborhood of the a-th boundary pixel.

[0044] Meaning of parameters and derivation process of formula calculation: Parameter , which represents the total number of boundary pixel points participating in feature extraction in the current image frame. The boundary points are extracted from the infrared image frame with a size of 1280×720 pixels by the Canny edge detection method, and the boundary aggregation area is obtained by combining image partition clustering, and the number of significant gradient change points within the effective contour closed area is counted; Parameter represents the gray-level jump value of the a-th boundary pixel point, with the unit of gray level, which is calculated based on the gray value of the current pixel and the gray difference of its adjacent pixel in the main gradient direction. After using the Sobel operator to extract the main gradient direction of the image, for the a-th boundary pixel point, the gradient jump value is extracted in the following way; For example: the gray value of the 1st boundary pixel point is 186, and the adjacent pixel is 142, then: ; Parameter is the Euclidean distance from the a-th pixel point to the centroid of the area within its closed contour, with the unit of pixel. The centroid coordinates of the area are calculated based on the average value of all boundary point coordinates. For example, if the average value of the boundary point coordinates within the contour is (112.4, 98.3), and the coordinates of the 1st boundary pixel are (110, 100), then: ; Parameter represents the pixel density value within the 9×9 pixel window area centered on the a-th boundary pixel, which is defined as the number of non-background pixels in the area divided by the total number of pixels 81. There are 53 non-background pixels in the 1st pixel area: ; Parameter Represents the average value of the pixel density in all contour areas in the current image frame. The density average of the window area corresponding to the 12 boundary points is calculated. The density values ​​are 0.654, 0.602, 0.688, 0.623, 0.612, 0.671, 0.660, 0.645, 0.679, 0.607, 0.618, and 0.640, respectively. Then: ; parameter is the local boundary strength in the neighborhood of the ath pixel, which is defined as the average value of the pixel gradient modulus in a 5×5 window centered on the current point, in grayscale. The modulus values ​​of all pixels in the area are extracted from the Sobel gradient image and averaged. For example, the sum of the modulus values ​​in the neighborhood of the first pixel is 1272, then: ; Substitute the above values ​​into the formula to calculate the response term of the first boundary pixel: ; For all The same calculation is performed on the boundary pixels, and the 12 boundary response values ​​are: 2.504, 2.384, 2.672, 2.590, 2.467, 2.710, 2.601, 2.533, 2.688, 2.449, 2.512, 2.574; Sum the 12 boundary response values: 2.504+2.384+2.672+2.590+2.467+2.710+2.601+2.533+2.688+2.449+2.512+2.574≈30.694; Calculate the boundary characteristic response value: ; The results show that the characteristic response intensity of the boundary area in the current image frame is 2.558. The higher the value, the more drastic the boundary jump, the higher the contour concentration, and the more significant the boundary closure. It serves as the basic characteristic value for constructing the subsequent boundary closure index value set.

[0045] S402: Based on the coordinates of the corresponding area in the boundary closure index value concentration, measure the aggregation index of the pixel gray value distribution in the same closed area, and generate a pixel aggregation index sequence by calculating the proportion of the number of pixel points in the area whose pixel gray values ​​converge to the mean.

[0046] Further analyze the pixel gray-scale aggregation characteristics within the closed region, extract the gray-scale values of the pixels inside each closed region, calculate the average gray-scale value of the region as a reference benchmark, and then count how many pixels in the region have gray-scale values close to the average value. Adopt a gray-scale convergence criterion, such as ±10 gray levels as the judgment range. If the gray-scale of a certain pixel falls between the average value ±10, it is considered to belong to the aggregated pixels. Calculate the ratio between the number of such aggregated pixels and the total number of pixels in the region to obtain the pixel aggregation index of the closed region. This index reflects whether the pixels inside the region are uniform and whether there are abnormal situations such as light and shadow noise interference. This processing process is particularly sensitive to the gray-scale changes caused by reflection and shadow in the pressure gauge pointer dial. The system executes this calculation operation in a loop for the target closed region of each frame of the image, and sorts out the pixel aggregation indexes of all frames according to the image frame sequence. This sequence serves as an important reference data for subsequent judgment of the correspondence between region occlusion and action response, and supports cross-analysis with the image occlusion state to generate a pixel aggregation index sequence.

[0047] S403: According to the pixel aggregation index sequence, filter out the time periods that meet the occlusion state of consecutive image frames, retrieve the self-check response signals of the corresponding image frames, and refer to the signal trigger frame position and the response action completion frame position to obtain the functional action response delay value.

[0048] Filter out the time periods of image frames that meet the continuous occlusion state. The judgment condition is that the pixel aggregation is continuously lower than the set reference threshold. For example, when the set aggregation is lower than 0.6, it is in the occlusion state. When several consecutive frames meet this condition, mark this frame interval as the occlusion time period. Subsequently, retrieve the device self-check response signal records corresponding to the frame numbers in this time period, and find the signal trigger frame and the action completion frame corresponding to the frame number from the log or signal. Calculate the functional response delay value through the time interval between the two frame numbers. This value reflects the response time difference between the trigger of the detection command and the completion of the action in the image. Set the frame number when the emergency pull ring is triggered to 102 and the frame number when the pull-out action is completed to 112, then the delay value is 10 frames. Uniformly record the response values corresponding to the occlusion state segment to generate a functional action response delay value list. The data table contains fields such as the start and end numbers of each frame segment, the corresponding region, the trigger and completion frames, and the response frame difference, etc., for further comparison of the time consistency between occlusion and response to obtain the functional action response delay value.

[0049] S404: Based on the response elongation time interval marked in the functional action response delay value, compare whether there is a frame number coincidence between the time range and the frame time period corresponding to the continuous image frame occlusion state. If both time overlaps hold simultaneously, mark the region coordinates as an abnormal region and generate an occlusion lag correlation monitoring label set.

[0050] Based on the extended response time interval recorded in the functional action response delay value, it is compared with the image masking time period identified in the previous step for each group of frame numbers to determine whether there is any overlapping behavior in the frame number range. The comparison logic uses the intersection of the frame number start and end value ranges to make a judgment. As long as there is any frame number intersection between the two, it is considered that the masking behavior affects the response action recognition. If the start and end frame numbers of a group of functional response delay segments overlap with the masking state segment by at least one frame, the corresponding image area coordinates are marked as abnormal areas, and the area information, frame time range, response delay value and other information are packaged together to generate abnormal records, which can be displayed in a color overlay on the image monitoring platform or output as a structured log table for operation and maintenance personnel to quickly locate abnormal behaviors and determine the impact of masking on self-test responses. It is used to assist in deciding whether to perform hardware re-inspection or image optimization operations, and output the masking lag associated monitoring label set.

[0051] The specific steps for obtaining the fire protection facility collaborative warning information set are as follows: S501: Call the coordinates of the abnormal area marked by the masking lag associated monitoring label set, extract the image masking range of the corresponding area, calculate the masking coverage rate by the ratio of the number of masked pixels to the total number of pixels in the area, call the structural oscillation frequency and functional response delay frame length data of the associated position, and generate a regional feature indicator group.

[0052] According to the marked image frame number and regional coordinate information, the abnormal area image block is intercepted in the corresponding frame image, and the masking state of each pixel in the area is identified. The identification method is based on the pixel grayscale concentration and regional boundary closure results calculated in the previous order, and the pixels with violent grayscale fluctuations and incomplete boundary closure are judged as masked pixels. By counting the total number of masked pixels in the area and performing a ratio operation with the total number of pixels in the area, the masking coverage value of the area is obtained. The value is between 0 and 1, and the severity of the masking of the corresponding area in the current image frame is recorded. At the same time, the previously generated structural oscillation frequency data and functional response delay frame length data are retrieved, and the number of pixel displacement anomalies in the entire image frame sequence and the frame length delay of the corresponding functional operation response are read respectively. The three data are integrated to form a characteristic indicator group of the area, and a unique number is assigned to each abnormal area. Its image coordinates, frame number, masking coverage, oscillation frequency and response delay value are recorded to generate a regional characteristic indicator group.

[0053] S502: Based on multiple values ​​in the regional characteristic indicator group, three grading intervals are set for image masking coverage, oscillation frequency and response delay frame length, and each indicator is divided into differentiated risk segments. Classification mapping is performed according to the segment combination to which the three indicators belong, and the risk category label of the abnormal area is analyzed to obtain the linkage risk level identification value.

[0054] Each indicator is segmented and classified by setting a fixed grading interval. The specific settings are as follows: image masking coverage is divided into three intervals: low risk (0.0, 0.3), medium risk (0.3, 0.6), and high risk (0.6, 1.0); structural oscillation frequency is divided into low frequency (0, 3), medium frequency (3, 7), and high frequency (7 and above); response delay frame length is divided into normal (0, 5), mild delay (5, 10), and severe delay (above 10). The system maps each indicator value to the corresponding risk level number according to the above interval, and combines the intervals where the three indicators are located to form a unique multi-dimensional risk segment combination code. For example, if an area corresponds to medium risk masking, medium frequency oscillation and mild delay, the combination code is B2. It is matched against the preset classification mapping table to identify the area as a medium risk type. The system traverses all abnormal areas and completes the overall risk category analysis according to the segment combination to which the three indicators belong. The risk category label of each area is output, and the grading processing and response strategy recommendation are performed to obtain the linkage risk level identification value.

[0055] S503: According to the linkage risk level identification value, the abnormal area is marked as three level categories: early warning, key attention and emergency response. At the same time, the image coordinates, time frame number and corresponding risk factors under each level are integrated to generate a fire protection facility collaborative early warning information set.

[0056] The abnormal areas are divided into three levels: early warning, key attention and emergency response. The early warning is a low-level combination code area, the key attention is a medium-level combination code area, and the emergency response is a high-level combination code area. The level of each area is automatically matched according to its identification value, and the image coordinates and corresponding frame number information of all areas are integrated by level classification. At the same time, the specific risk factors involved in each data are extracted, including shielding coverage exceeding the threshold, oscillation frequency abnormality or response delay timeout, etc., and they are combined with the image positioning information to form a structured early warning record. Each record contains fields such as risk level, coordinate position, image frame number and key indicator value, which can be used as input data for image recognition alarm management, and can also be used for visualization platform to overlay hierarchical layers to achieve rapid distinction of abnormalities of different levels and recommendation of response strategies, so as to facilitate front-line operators to deal with related facility hazards in time according to risk levels and generate a collaborative early warning information set for fire protection facilities.

[0057] See also Figure 2 The present invention also provides a firefighting facility detection and early warning system based on a visual camera, which is used to execute the firefighting facility detection and early warning method based on a visual camera. The system includes: The image acquisition module obtains multi - focal image frames of the edge regions of fixed spray nozzles and fire extinguishers, extracts the edge pixel points of the nozzle edge region and the water outlet hole contour, extracts the gray - scale value sequences of the edge surrounding regions in consecutive image frames, identifies the continuously changing gray - scale states, and obtains the image occlusion region information set; Based on the image occlusion region information set, the edge occlusion recognition module extracts the gray - scale response regions in the long - focal and short - focal image frames, calculates the gray - scale difference and screens the nozzle and water outlet hole regions that exceed the gray - scale response threshold, and obtains the focal - length difference response warning point set; The focal - length response analysis module calls the focal - length difference response warning point set, obtains the corner coordinates of the edge line of the nozzle structure in the corresponding image frame, calculates the pixel distance value sequence from the corner points to the center point based on the image visual center point coordinates and each corner point coordinate, compares the distance changes of each corner point in consecutive image frames, and obtains the initial judgment point sequence of structural deformation; According to the initial judgment point sequence of structural deformation, the structure dynamic monitoring module extracts the boundary contours of the pressure gauge scale pointer and the emergency release valve pull - ring region, calculates the boundary closure value and the pixel aggregation value in consecutive image frames, and obtains the occlusion lag - related monitoring label set; The linkage warning module calls the occlusion lag - related monitoring label set, compares the self - inspection signal trigger period and the structure action response period in the image frame, calculates the coincidence duration of the two, and conducts classification recognition in combination with the occlusion range, oscillation frequency and response delay time, and obtains the fire - fighting facility collaborative warning information set.

[0058] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention; therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A firefighting facility detection and early warning method based on a visual camera, characterized in that: Includes the following step: S1: Obtain multi-focal image frames of the edge area of ​​the fixed sprinkler head and the fire extinguisher, extract the grayscale values ​​of the edge pixel points and the surrounding area group, determine whether there is a continuous change in grayscale in the continuous image frames, mark it as the edge fuzzy area, and obtain the image masked area information set; S2: Based on the image masked area information set, the fire extinguisher nozzle area and the sprinkler head water outlet area are selected as the image reverse comparison objects, and the grayscale distribution difference under long focus and short focus states is compared to generate a focal length difference response warning point set; S3: Call the focal difference response warning point set, extract the corner points of the outline of the fire-fighting facility-related structure, measure the pixel distance from the corner point to the visual center reference point, identify the oscillation behavior of the dynamic trajectory of the corner point, and generate the initial judgment point sequence of the structural deformation; S4: Based on the initial judgment point sequence of structural deformation, the boundary contours of the pressure gauge pointer scale area and the emergency release valve pull ring area in the continuous image frames are extracted, the closure index and pixel aggregation index are calculated in the continuous image frames, the time overlap between the masking period and the response extension interval is compared, and the area is marked as abnormal, generating a masking lag associated monitoring label set.

2. The firefighting facility detection and early warning method based on a visual camera according to claim 1 is characterized in that: The image masked area information set includes an edge blur feature map, a blur area coordinate set, and a blur duration index; the focal difference response warning point set includes a response offset vector, a focal difference level, and a contrast grayscale difference curve; the structural deformation preliminary judgment point series includes an oscillation corner point sequence, a pixel offset path, and a deformation amplitude index; the mask lag associated monitoring label set includes a mask duration segment, a linkage anomaly matching degree, and a response delay statistical item.

3. The firefighting facility detection and early warning method based on a visual camera according to claim 1 is characterized in that: The steps of obtaining the image masked area information set are specifically as follows: S101: Obtain multi-focal image frames of edge areas of fixed sprinkler heads and fire extinguishers, extract a set of pixel points corresponding to the edge areas in each frame of the image, and call the grayscale value of a single pixel in the area covered by the pixel point set to obtain a grayscale distribution set of the edge areas in a continuous image frame sequence, classify the grayscale values ​​according to the pixel coordinate positions, and generate an edge grayscale distribution value set; S102: based on the edge grayscale distribution value set, calling the differentiated time series grayscale values ​​under the same edge pixel coordinates, performing a difference operation on each set of grayscale values, obtaining a grayscale difference sequence between consecutive image frames, and judging whether there is a continuous grayscale change in the region according to the change in the single-frame grayscale difference and the number of consecutive change frames, and generating an edge grayscale fluctuation trend; S103: According to the edge grayscale fluctuation trend, the pixel coordinate groups whose continuous image frames with grayscale changes exceed the threshold are screened and marked as edge blurred areas. The coordinates and coverage of the edge blurred areas are merged to obtain an image masked area information set.

4. The firefighting facility detection and early warning method based on a visual camera according to claim 3 is characterized in that: The steps for acquiring the focal length difference response warning point set are specifically as follows: S201: calling the coordinates of the edge blur region in the image masked region information set, extracting the grayscale value set at the same position in the corresponding long-focus and short-focus image frames, and respectively aggregating the grayscale responses of the pixels in the region to obtain a grayscale contrast value set of the masked region; S202: Based on the grayscale contrast value set of the masked area, the pixel group located in the nozzle area of ​​the fire extinguisher and the water outlet area of ​​the sprinkler head is selected, the grayscale difference of the same area under two focal lengths is calculated, the corresponding point is marked as the warning position, and the focal length difference and coordinate information are combined to obtain the focal length difference response warning point set; The formula for calculating the grayscale difference of the same area at two focal lengths is as follows: ； in, Representative area In focal length With focal length The grayscale response difference eigenvalue under Representative area The total number of pixels contained in Representative area Middle pixels at focal length The gray value below Representative area Middle pixels at focal length The gray value below Representative area Middle pixels at focal length The horizontal coordinate value under Representative area Middle pixels at focal length The horizontal coordinate value under Representative area Middle pixels at focal length The vertical coordinate value of Representative area Middle pixels at focal length The vertical coordinate value of Represents an adjustable compensation factor for horizontal displacement error.

5. The firefighting facility detection and early warning method based on visual camera according to claim 4 is characterized in that: The steps for obtaining the structural deformation preliminary judgment point sequence are specifically as follows: S301: calling the image coordinate area in the focus difference response warning point set, extracting the contour line edge of the corresponding fire-fighting facility in each image coordinate area, locating the image pixel mutation point at the boundary corner by the edge pixel gradient change, and taking the position of the angle change in each group of image pixel mutation points as the contour corner point, and generating a structure contour corner point coordinate set; S302: Based on the structure contour corner point coordinate set, the image center position is used as the visual center reference point, the pixel distance between the corner point and the visual center reference point is measured, and the pixel distances are arranged in the order of the image frame sequence to obtain a corner point pixel distance sequence; S303: Calculate the pixel displacement change of the corner points between consecutive image frames based on the corner point pixel distance sequence, extract the corner point sequence whose pixel displacement change exceeds the set oscillation amplitude threshold, identify the oscillation behavior in the image frame sequence and mark the corresponding frame position and corner point number, and generate a preliminary judgment point sequence of structural deformation.

6. The firefighting facility detection and early warning method based on visual cameras according to claim 5 is characterized in that: The steps for obtaining the masked hysteresis associated monitoring label set are specifically as follows: S401: calling the structural deformation initial judgment point sequence, extracting the boundary between the pressure gauge pointer scale area and the emergency release valve pull ring area in each frame image, calculating the gradient jump of the boundary pixels and the pixel distribution density of the contour closed area to obtain the boundary feature response value, and generating a boundary closure index value set; S402: Based on the coordinates of the corresponding area in the boundary closure index value concentration, measure the aggregation index of the pixel gray value distribution in the same closed area, and generate a pixel aggregation index sequence by calculating the proportion of the number of pixel points in the area whose pixel gray values ​​converge to the mean value; S403: According to the pixel concentration index sequence, the time period that satisfies the continuous image frame masking state is selected, the self-test response signal of the corresponding image frame is retrieved, and the function action response delay value is obtained by referring to the signal trigger frame position and the response action completion frame position; S404: Based on the response extension time interval marked in the functional action response delay value, compare whether there is any overlap in frame number between the time range and the frame time period corresponding to the masking state of the continuous image frames. If the time overlap of both is established at the same time, mark the area coordinates as an abnormal area and generate a masking lag associated monitoring label set.

7. The firefighting facility detection and early warning method based on visual cameras according to claim 6 is characterized in that: The formula used to calculate the gradient jump of boundary pixels and the pixel distribution density of the contour closed area to obtain the boundary feature response value is: ； in, represents the boundary characteristic response value, represents the total number of boundary pixels in the real-time frame, Represents the grayscale jump value of the a-th boundary pixel in the gradient direction, Represents the pixel distance from the ath boundary pixel to the centroid point in the neighborhood, Represents the pixel density within the closed contour with the ath pixel as the center area, Represents the average value of the pixel density within the contour in the real-time image frame, Represents the local boundary intensity value within the neighborhood of the ath boundary pixel.

8. The firefighting facility detection and early warning method based on visual camera according to claim 1 is characterized in that: The method also Including S5 step: S5: Call the masking hysteresis associated monitoring label set, compare the image masking range, structural vibration frequency and response delay time of the abnormal area, perform classification and level determination, mark it as warning prompt, key attention and emergency response level, and generate a fire protection facility collaborative warning information set; The fire-fighting facility collaborative warning information set includes a warning level identifier, an abnormal source pointing label, and a collaborative response situation.

9. The firefighting facility detection and early warning method based on visual cameras according to claim 8 is characterized in that: The steps for obtaining the fire protection facility collaborative warning information set are specifically as follows: S501: calling the coordinates of the abnormal area marked by the masking hysteresis associated monitoring tag set, extracting the image masking range of the corresponding area, calculating the masking coverage rate by the ratio of the number of masked pixels to the total number of pixels in the area, calling the structural oscillation frequency and function response delay frame length data of the associated position, and generating a regional feature indicator group; S502: Based on multiple values ​​in the regional characteristic indicator group, three classification intervals of image mask coverage, oscillation frequency and response delay frame length are set, each indicator is divided into a differentiated risk segment, and classification mapping is performed according to the segment combination of the three indicators, and the risk category label of the abnormal area is analyzed to obtain the linkage risk level identification value; S503: According to the linkage risk level identification value, the abnormal area is marked as three level categories: early warning, key attention and emergency response. At the same time, the image coordinates, time frame number and corresponding risk factors under each level are integrated to generate a fire protection facility collaborative early warning information set.

10. Firefighting facility detection and early warning system based on visual camera, characterized by: A system for executing the firefighting facility detection and early warning method based on a visual camera according to any one of claims 1 to 9, comprising: The image acquisition module obtains multi-focal length image frames of the edge areas of fixed sprinkler heads and fire extinguishers, extracts edge pixel points of the nozzle edge area and the water outlet contour, extracts the grayscale value sequence of the edge peripheral area in continuous image frames, identifies the grayscale continuous change state, and obtains the image masked area information set; The edge masking recognition module extracts the grayscale response area in the long-focus and short-focus image frames based on the image masking area information set, calculates the grayscale difference, and screens the nozzle and water outlet areas that exceed the grayscale response threshold to obtain the focal length difference response warning point set; The focal response analysis module calls the focal difference response warning point set to obtain the corner point coordinates of the nozzle structure edge line in the corresponding image frame. Based on the coordinates of the image visual center point and the coordinates of each corner point, the pixel distance value sequence from the corner point to the center point is calculated. The distance change of each corner point in the continuous image frames is compared to obtain the initial judgment point sequence of the structural deformation. The structural dynamic monitoring module extracts the boundary contours of the pressure gauge scale pointer and the emergency release valve pull ring area based on the initial judgment point list of the structural deformation, calculates the boundary closure value and pixel aggregation value in the continuous image frames, and obtains the masked lag correlation monitoring label set; The linkage warning module calls the shielding lag associated monitoring label set, compares the self-test signal triggering period and the structural action response period in the image frame, calculates the overlapping duration of the two, and classifies and identifies them based on the shielding range, oscillation frequency and response delay time to obtain the fire-fighting facility collaborative warning information set.

Images