A smoke and fire detection and early warning method and system based on AI video analysis

Abstract

The present invention provides a method and system for detecting and warning fireworks based on AI video analysis, including: docking a fireworks warning supervision platform with a video device, and extracting video frames from the real-time video stream in the video device based on the docking result; transmitting one image frame to a video analysis model based on a communication interface for preliminary fireworks detection based on the video extraction result, and retransmitting a target number of image frames to the video analysis model based on a communication interface for re-inspection when fireworks are present; based on the re-inspection result, when fireworks are present, displaying a real-time alarm page based on the fireworks warning supervision platform, and synchronously sending a fireworks warning notification to a management terminal. The accuracy and precision of fireworks detection are improved, the accuracy of fireworks warning is guaranteed, and repeated false alarms are avoided.

Description

Technical Field

[0001] The present invention relates to the field of image processing and monitoring technology, and in particular to a method and system for detecting and warning fireworks based on AI video analysis. Background Art

[0002] At present, fire prevention is an important issue of safety protection in social production and life, so it is necessary to pay attention to the monitoring of fireworks to ensure safety;

[0003] Traditional image processing methods are to identify and detect some features in the image, such as shape, color and texture, etc. However, traditional image processing mostly analyzes the collected on-site images and cannot perform real-time analysis. At the same time, during the analysis, once a fire-like situation is detected in an image, a corresponding alarm operation will be performed, resulting in poor real-time and accuracy of fire-like warnings, which affects the accuracy of fire-like detection results and causes repeated false alarms.

[0004] Therefore, in order to overcome the above-mentioned defects, the present invention provides a smoke and fire detection and early warning method and system based on AI video analysis. Summary of the invention

[0005] The present invention provides a method and system for detecting and warning fireworks based on AI video analysis, which are used to connect a video device with a fireworks warning platform, thereby facilitating the transmission of the real-time video stream of the monitoring area collected by the video device to the fireworks warning platform for real-time analysis; secondly, the real-time video stream is sampled, and a frame of the image is transmitted to a video analysis model for preliminary fireworks detection, thereby improving the efficiency of preliminary fireworks detection; at the same time, when the preliminary fireworks detection determines that fireworks exist, a set of image frames of target data is locked from the real-time video stream for re-inspection, thereby improving the accuracy and precision of fireworks detection; finally, when it is determined that fireworks exist according to the re-inspection result, a fireworks warning notification is issued, thereby ensuring the accuracy of fireworks warning and avoiding repeated false alarms.

[0006] The present invention provides a smoke and fire detection and early warning method based on AI video analysis, comprising:

[0007] S1: Connect the fireworks warning and monitoring platform with the video device, and extract frames from the real-time video stream in the video device based on the connection result;

[0008] S2: Based on the video frame extraction result, one image frame is transmitted to the video analysis model through the communication interface each time for preliminary fireworks detection, and when fireworks are present, a target number of image frame sets are transmitted back to the video analysis model through the communication interface for re-inspection;

[0009] S3: When it is determined based on the re-inspection result that fireworks exist, a real-time alarm page is displayed based on the fireworks warning supervision platform, and a fireworks warning notification is simultaneously sent to the management terminal.

[0010] Preferably, a method for detecting and warning fireworks based on AI video analysis, in S1, a fireworks warning monitoring platform is connected with a video device, including:

[0011] Obtain terminal parameters of different video devices, and determine service standards and communication protocols of different video devices based on the terminal parameters;

[0012] Different video devices are classified based on service standards, and based on the classification results, parameter compatibility adaptation is performed on the preset gateway device according to the communication protocol, and different types of video devices are connected to the preset gateway device based on parameter compatibility adaptation;

[0013] At the same time, the data interface standards of the fireworks warning and supervision platform are unified, and the data interface after the interface standard is unified is connected with the preset gateway device after parameter compatibility adaptation to complete the connection between the fireworks warning and supervision platform and the video equipment.

[0014] Preferably, a method for detecting and warning fireworks based on AI video analysis, in S1, extracting video frames from a real-time video stream in a video device based on a docking result, includes:

[0015] Reading a real-time video stream in the video device based on the docking result, and determining an image frame included in the real-time video stream based on the reading result;

[0016] Extracting corresponding image content from the real-time video stream frame by frame based on the image frame, obtaining each frame of image data based on the image content, and obtaining a frame image sequence based on each frame of image data;

[0017] A polling frame extraction mechanism for a frame image sequence is determined based on a video frame extraction index, and video frame extraction is performed on a real-time video stream in a video device based on the polling frame extraction mechanism.

[0018] Preferably, a method for detecting and warning fireworks based on AI video analysis, in S2, based on the video frame extraction result, one image frame is transmitted to the video analysis model based on the communication interface each time for preliminary fireworks detection, and when fireworks exist, a target number of image frame sets are transmitted to the video analysis model based on the communication interface for re-inspection, including:

[0019] Obtain an image frame corresponding to the video frame extraction result, and determine a data transmission rule based on the communication protocol of the communication interface;

[0020] Extracting semantics from the data transmission rules, determining a syntax execution command in the data transmission rules based on the extracted semantics, and converting the format of an image frame based on the syntax execution command to obtain transmittable image data;

[0021] The transmittable image data is transmitted to the video analysis model based on the communication interface, and a frame of the image is divided into M sampled small images based on the transmission result;

[0022] The three-primary-color grayscale average values ​​of each sampled image in the color space are extracted based on the RGB color channels, and the three-primary-color grayscale average values ​​are weighted and summarized to obtain a color histogram corresponding to a frame of the image.

[0023] Converting the color histogram into a low-dimensional color histogram, quantizing each color component in the low-dimensional color histogram to a normalized range, and obtaining the color proportion in an image frame based on the quantization result;

[0024] Determine the degree of membership of the color proportion relative to the reference fireworks color feature, and when the degree of membership is higher than a preset membership threshold, determine that fireworks exist;

[0025] Based on the determination result, a target number of image frame sets are transmitted to the video analysis model via a communication interface for re-inspection.

[0026] Preferably, a method for detecting and warning fireworks based on AI video analysis, based on the determination result, retransmits a target number of image frame sets to a video analysis model based on a communication interface for re-inspection, including:

[0027] Based on the determination result, a source position of an image frame corresponding to the video frame extraction result is locked, and based on the source position, a known number of forward and backward video frames are locked in the real-time video stream with the image frame corresponding to the video frame extraction result as the center to obtain a target number of image frame sets;

[0028] Retransmitting a target number of image frames to a video analysis model based on a communication interface, and classifying the target number of image frames bit by bit based on the retransmission result to obtain inter-frame difference parameters;

[0029] Determine the motion region between the target number of image frame sets based on the inter-frame difference parameter, and determine the spatial span of the preliminary fireworks detection result in the image frame set based on the motion region;

[0030] When the spatial span is greater than the preset requirement, the fireworks area in each image frame in the image frame set is calibrated based on the preliminary fireworks detection result, and the edge of the fireworks in the fireworks area is monitored based on the calibration result to obtain the fireworks outline;

[0031] The morphological features of the fireworks are determined based on the fireworks contours, and a re-inspection of a target number of image frame sets is completed based on the fireworks features.

[0032] The present invention provides a smoke and fire detection and early warning system based on AI video analysis, comprising:

[0033] The video frame extraction module is used to connect the fireworks warning and supervision platform with the video device, and extract the video frames of the real-time video stream in the video device based on the connection result;

[0034] A fireworks detection module is used to transmit one image frame at a time to the video analysis model based on the communication interface for preliminary fireworks detection based on the video frame extraction result, and to retransmit a target number of image frames to the video analysis model based on the communication interface for re-inspection when fireworks are present;

[0035] The early warning module is used to determine the presence of fireworks based on the re-inspection results, display a real-time alarm page based on the fireworks early warning supervision platform, and simultaneously send a fireworks early warning notification to the management terminal.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. By connecting the video equipment with the fireworks warning platform, it is convenient to transmit the real-time video stream of the monitoring area collected by the video equipment to the fireworks warning platform for real-time analysis. Secondly, the real-time video stream is sampled and the obtained image frames are transmitted to the video analysis model for preliminary fireworks detection, which improves the efficiency of preliminary fireworks detection. At the same time, when the preliminary fireworks detection determines that there are fireworks, the image frame set of the target data is locked from the real-time video stream for re-inspection, which improves the accuracy and precision of fireworks detection. Finally, when it is determined that there are fireworks based on the re-inspection results, a fireworks warning notification is issued, which ensures the accuracy of fireworks warning and avoids repeated false alarms.

[0038] 2. By parsing the communication protocol of the communication interface, the data transmission rules corresponding to the communication interface can be accurately and effectively determined. Secondly, the data transmission rules are semantically extracted, and the syntax execution commands in the data transmission rules are locked according to the extraction results, and the format of each image frame is converted through the syntax execution command, so as to facilitate the accurate and reliable transmission of the image frames obtained by video extraction to the video analysis model. Finally, by segmenting the transmitted image frames and analyzing the color histogram, the color proportion of each color in the image frame is locked according to the color histogram, and the locked color proportion is compared with the benchmark fireworks color feature to achieve accurate and reliable preliminary detection of fireworks, thereby ensuring the accuracy of the preliminary detection of fireworks. At the same time, when it is determined that fireworks exist, re-inspection is carried out in time to ensure the timeliness and reliability of fireworks detection.

[0039] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0040] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0042] Figure 1 This is a flow chart of a method for smoke and fire detection and early warning based on AI video analysis in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart of step 1 in a method for smoke and fire detection and early warning based on AI video analysis in an embodiment of the present invention;

[0044] Figure 3 The figure is a structural diagram of a smoke and fire detection and early warning system based on AI video analysis in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0046] Embodiment 1:

[0047] This embodiment provides a smoke and fire detection and early warning method based on AI video analysis, such as Figure 1 As shown, including:

[0048] S1: Connect the fireworks warning and monitoring platform with the video device, and extract frames from the real-time video stream in the video device based on the connection result;

[0049] S2: Based on the video frame extraction result, one image frame is transmitted to the video analysis model through the communication interface each time for preliminary fireworks detection, and when fireworks are present, a target number of image frame sets are transmitted back to the video analysis model through the communication interface for re-inspection;

[0050] S3: When it is determined based on the re-inspection result that fireworks exist, a real-time alarm page is displayed based on the fireworks warning supervision platform, and a fireworks warning notification is simultaneously sent to the management terminal.

[0051] In this embodiment, video equipment is set up in the monitoring area to collect real-time video of the monitoring area, so as to ensure that the monitoring video is transmitted back in real time for real-time analysis. The video equipment includes national standard equipment and non-national standard equipment, both of which can be connected to the fireworks warning supervision platform, among which the fireworks warning supervision platform is set in advance.

[0052] In this embodiment, the real-time video stream refers to real-time video data in a corresponding area monitored by a video device.

[0053] In this embodiment, video frame extraction refers to extracting a frame of image from a real-time video stream to perform preliminary smoke and fire detection.

[0054] In this embodiment, the video analysis model is built in advance and is used to interpret the received image frames to achieve accurate and effective judgment on whether there are fireworks.

[0055] In this embodiment, preliminary fireworks detection refers to roughly analyzing the image frames obtained by video frame extraction through a video analysis model to determine whether there are fireworks. That is, it cannot accurately determine whether there are fireworks, and further analysis of the real-time video stream is required.

[0056] In this embodiment, the target number is set in advance, and is generally set to 8 frames.

[0057] In this embodiment, re-checking refers to performing fireworks detection again on a target number of image frame sets through a video analysis model, so as to accurately determine whether fireworks actually exist.

[0058] The beneficial effects of the above technical solution are: by connecting the video equipment with the fireworks warning platform, it is convenient to transmit the real-time video stream of the monitoring area collected by the video equipment to the fireworks warning platform for real-time analysis; secondly, the real-time video stream is sampled, and the obtained image frames are transmitted to the video analysis model for preliminary fireworks detection, thereby improving the efficiency of preliminary fireworks detection; at the same time, when the preliminary fireworks detection determines that fireworks exist, the image frame set of target data is locked from the real-time video stream for re-inspection, thereby improving the accuracy and precision of fireworks detection; finally, when it is determined that fireworks exist according to the re-inspection results, a fireworks warning notification is issued, thereby ensuring the accuracy of fireworks warning and avoiding repeated false alarms.

[0059] Embodiment 2:

[0060] Based on Example 1, this example provides a smoke and fire detection and warning method based on AI video analysis, such as Figure 2 As shown, in S1, the fireworks warning monitoring platform is connected with the video equipment, including:

[0061] S101: Acquire terminal parameters of different video devices, and determine service standards and communication protocols of different video devices based on the terminal parameters;

[0062] S102: Different video devices are classified based on the service standard, and parameter compatibility adaptation is performed on the preset gateway device according to the communication protocol based on the difference classification result, and different types of video devices are connected to the preset gateway device based on the parameter compatibility adaptation;

[0063] S103: At the same time, the data interface of the fireworks warning and supervision platform is unified in interface standard, and the data interface after the interface standard is unified is connected with the preset gateway device after parameter compatibility adaptation, so as to complete the connection between the fireworks warning and supervision platform and the video device.

[0064] In this embodiment, different video devices refer to national standard devices (cameras, platforms, NVRs, etc.) and non-national standard devices (onvi, rtsp, rtmp, live broadcast devices, etc.).

[0065] In this embodiment, the terminal parameters refer to parameters such as device type parameters and operating power of different video devices.

[0066] In this embodiment, the service standard refers to the working modes of different video devices, namely, national standards and non-national standards.

[0067] In this embodiment, the communication protocol refers to the communication method adopted by different video devices when performing data transmission.

[0068] In this embodiment, the preset gateway device is set in advance and is used as an intermediary to connect different video devices and the fireworks warning supervision platform.

[0069] In this embodiment, the interface standard unification refers to performing unified operations on data transmitted from different video devices to the fireworks warning and supervision platform, thereby facilitating transmission and management, that is, using a unified standard to receive data in different formats.

[0070] The beneficial effects of the above technical solution are: by classifying different video devices according to their terminal parameters, it is convenient to perform parameter compatibility adaptation on the preset gateway device according to different types of video devices; secondly, according to the difference classification results, different categories of video devices are connected to the preset gateway device, so as to facilitate the connection of different video devices with the fireworks and fire warning supervision platform through the preset gateway device; finally, after the interface standard of the data interface of the fireworks and fire warning supervision platform is unified, different video devices can be effectively connected with the fireworks and fire warning supervision platform through the preset gateway device, so as to facilitate the reception and analysis of real-time video stream data monitored by different video devices through the fireworks and fire warning supervision platform, thereby ensuring the real-time nature of the fireworks and fire analysis.

[0071] Embodiment 3:

[0072] Based on Example 1, this embodiment provides a method for detecting and warning fireworks based on AI video analysis. In S1, a fireworks warning monitoring platform is connected to a video device, including:

[0073] Based on the docking results, the terminal monitoring areas of different video devices are extracted, and the identity labels of the video streams of different video devices in the fireworks early warning supervision platform are determined based on the environmental characteristics of the terminal monitoring areas;

[0074] Based on the terminal quantity of video devices and the coverage area of ​​the corresponding terminal monitoring area, the cache space in the fireworks early warning and supervision platform is individually divided to obtain a personalized cache space, and the identity tag is encapsulated with the corresponding personalized cache space.

[0075] In this embodiment, the environmental features refer to the buildings and distribution of objects contained in the terminal monitoring area.

[0076] In this embodiment, the identity tag refers to a marking tag that can characterize the terminal monitoring area corresponding to different video streams.

[0077] In this embodiment, personalized division refers to splitting the cache space in the fireworks warning supervision platform according to the number of video devices and the data volume of each terminal monitoring area and the monitored video stream, so as to facilitate caching the video streams monitored by different video devices in the corresponding cache space, wherein the personalized cache space is the result obtained after personalized division of the cache space.

[0078] The beneficial effects of the above technical solution are: by determining the terminal monitoring areas corresponding to different video devices, and determining the identity tags corresponding to different video streams according to the environmental characteristics of the terminal monitoring areas, secondly, according to the terminal quantity of the video device and the coverage area of ​​the corresponding terminal monitoring area, the cache space in the fireworks warning supervision platform is individually divided, and finally, the obtained identity tags are encapsulated with the corresponding personalized cache space, so as to facilitate the effective storage of video streams monitored by different video devices, and also facilitate timely and effective analysis of the fireworks conditions in different terminal monitoring areas.

[0079] Embodiment 4:

[0080] Based on Example 1, this embodiment provides a method for detecting and warning fireworks based on AI video analysis. In S1, based on the docking result, video frames are extracted from the real-time video stream in the video device, including:

[0081] Reading a real-time video stream in the video device based on the docking result, and determining an image frame included in the real-time video stream based on the reading result;

[0082] Extracting corresponding image content from the real-time video stream frame by frame based on the image frame, obtaining each frame of image data based on the image content, and obtaining a frame image sequence based on each frame of image data;

[0083] A polling frame extraction mechanism for a frame image sequence is determined based on a video frame extraction index, and video frame extraction is performed on a real-time video stream in a video device based on the polling frame extraction mechanism.

[0084] In this embodiment, the image frame refers to a plurality of static images contained in the real-time video stream.

[0085] In this embodiment, each frame of image data refers to information such as a specific recording object contained in each image frame.

[0086] In this embodiment, the frame image sequence refers to the result obtained by splitting the real-time video stream into multiple continuous image frames.

[0087] In this embodiment, the video frame extraction index is known in advance and is used to characterize the standard to be followed when performing video frame extraction. For example, it may be to randomly extract frames each time or to extract frames 30 seconds after the first frame.

[0088] In this embodiment, the polling frame extraction mechanism refers to a rule for performing video frame extraction on a real-time video stream formulated according to a video frame extraction indicator.

[0089] The beneficial effects of the above technical solution are: by reading the real-time video stream in the video device and determining the image frames contained in the real-time video stream, secondly, extracting the image content in each image frame to achieve accurate and effective determination of the frame image sequence, and finally, according to the established polling frame extraction mechanism, video frame extraction is achieved for the real-time video stream, thereby facilitating the preliminary detection of fireworks and improving the efficiency of the preliminary detection of fireworks and fire.

[0090] Embodiment 5:

[0091] On the basis of Example 4, this embodiment provides a method for detecting and warning fireworks based on AI video analysis, and obtains a frame image sequence based on each frame of image data, including:

[0092] Obtain each frame of image data extracted frame by frame, and divide each frame of image data into uniform blocks based on a reference division size;

[0093] Based on the multi-scale evaluation index, the preset training images are trained to build a support vector regression model, and the image data of each block is analyzed based on the support vector regression model to obtain the feature vector of each image block;

[0094] Determine a quality evaluation value of each image block based on the feature vector, determine the image quality loss degree of each frame image based on the quality evaluation value of each image block, and determine the frame image whose image quality loss degree is greater than a preset threshold as a distorted frame image;

[0095] Similar structural features are determined based on adjacent frame image data, and the distorted frame image is repaired based on the similar structural features.

[0096] In this embodiment, the reference division size is set in advance and is used to divide the frame image data.

[0097] In this embodiment, the multi-scale evaluation index is set in advance and is a basis for evaluating the quality of frame images. It is not unique and may be, for example, clarity.

[0098] In this embodiment, the preset training image is known in advance.

[0099] In this embodiment, a support vector regression model is constructed after training a preset training image according to a multi-scale evaluation index, and is used to extract feature information of image data, wherein the feature vector is the image feature information contained in the image block obtained after analyzing the image block through the support vector regression model.

[0100] In this embodiment, the quality evaluation value refers to a parameter that can characterize the quality of the image block. A higher evaluation value indicates a better quality of the image block.

[0101] In this embodiment, the preset threshold is set in advance and is a standard used to measure whether the image block meets the minimum requirement and can be modified.

[0102] In this embodiment, similar structural features refer to similarities between adjacent frames of image data, including recorded objects and image components.

[0103] The beneficial effects of the above technical solution are: by dividing the frame image data into uniform blocks and training the preset training images through multi-scale evaluation indicators, the support vector regression model can be accurately and effectively constructed; secondly, the image data of each block is analyzed through the constructed support vector regression model to accurately and effectively determine the quality evaluation value of each image block; finally, the distorted frame image existing in the frame image is locked through the quality evaluation value, and the distorted frame image is repaired through similar structural features in adjacent frame image data, thereby ensuring the accuracy and reliability of the obtained frame image, and providing convenience and guarantee for improving the accuracy of fireworks detection.

[0104] Embodiment 6:

[0105] Based on Example 1, this embodiment provides a method for detecting and warning fireworks based on AI video analysis. In S2, based on the video frame extraction result, one image frame is transmitted to the video analysis model based on the communication interface each time for preliminary fireworks detection, and when fireworks exist, a target number of image frame sets are transmitted to the video analysis model based on the communication interface for re-inspection, including:

[0106] Obtain an image frame corresponding to the video frame extraction result, and determine a data transmission rule based on the communication protocol of the communication interface;

[0107] Extracting semantics from the data transmission rules, determining a syntax execution command in the data transmission rules based on the extracted semantics, and converting the format of an image frame based on the syntax execution command to obtain transmittable image data;

[0108] The transmittable image data is transmitted to the video analysis model based on the communication interface, and a frame of the image is divided into M sampled small images based on the transmission result;

[0109] The three-primary-color grayscale average values ​​of each sampled image in the color space are extracted based on the RGB color channels, and the three-primary-color grayscale average values ​​are weighted and summarized to obtain a color histogram corresponding to one image frame.

[0110] Converting the color histogram into a low-dimensional color histogram, quantizing each color component in the low-dimensional color histogram to a normalized range, and obtaining the color proportion in an image frame based on the quantization result;

[0111] Determine the degree of membership of the color proportion relative to the reference fireworks color feature, and when the degree of membership is higher than a preset membership threshold, determine that fireworks exist;

[0112] Based on the determination result, a target number of image frame sets are transmitted to the video analysis model via a communication interface for re-inspection.

[0113] In this embodiment, the communication protocol refers to a data transmission method corresponding to the communication interface, wherein the data transmission rule is the data transmission method, which is determined according to the communication protocol of the communication interface.

[0114] In this embodiment, the syntax execution command refers to specific requirement parameters corresponding to data transmission in the data transmission rule determined after semantic analysis of the data transmission rule, including the format of the data, etc.

[0115] In this embodiment, the transmittable image data refers to the result obtained by converting the format of one image frame through a syntax execution command.

[0116] In this embodiment, the sampling thumbnails refer to a plurality of image blocks obtained by segmenting the image frames extracted from the video.

[0117] In this embodiment, the RGB color channel refers to the three primary colors of red, green and blue. The grayscale average values ​​of the three primary colors of each sampled small image are determined through the RGB color channel, thereby determining the color histogram corresponding to a frame of the image.

[0118] In this embodiment, the color histogram is used to characterize the existence state and distribution of the image colors in the sampled thumbnails, so as to facilitate determining whether fireworks exist according to the color histogram, wherein fireworks include smoke and fire.

[0119] In this embodiment, the low-dimensional color histogram refers to a color histogram that is subjected to dimensional transformation, in order to more accurately and intuitively determine the color conditions contained in the image.

[0120] In this embodiment, each color component refers to the content of each color in an image frame.

[0121] In this embodiment, the reference fireworks color feature is known in advance and is used to characterize the color situation corresponding to the fireworks.

[0122] In this embodiment, the degree of membership is used to characterize the possibility that fireworks are contained in an image frame determined according to the color proportion. The higher the degree of membership, the more likely it is that fireworks exist.

[0123] In this embodiment, the preset membership threshold is set in advance.

[0124] In this embodiment, the normalization range is set in advance and is used to quantify each color component to a standard, thereby facilitating determination of the proportion of different colors.

[0125] The beneficial effects of the above technical solution are: by parsing the communication protocol of the communication interface, the data transmission rules corresponding to the communication interface are accurately and effectively determined; secondly, the data transmission rules are semantically extracted, and the syntax execution commands in the data transmission rules are locked according to the extraction results, and the format of a frame of image frames is converted through the syntax execution commands, so as to facilitate the accurate and reliable transmission of the frame of image frames obtained by video extraction to the video analysis model; finally, by segmenting the transmitted frame of image frames and analyzing the color histogram, the color proportion of each color in the frame of image frames is locked according to the color histogram, and the locked color proportion is compared with the reference fireworks color feature, so as to realize accurate and reliable preliminary detection of fireworks, thereby ensuring the accuracy of the preliminary detection of fireworks; at the same time, when it is determined that fireworks exist, re-inspection is carried out in time to ensure the timeliness and reliability of fireworks detection.

[0126] Embodiment 7:

[0127] Based on Example 6, this embodiment provides a method for detecting and warning fireworks based on AI video analysis, which transmits a target number of image frame sets to a video analysis model for re-inspection based on a communication interface based on the determination result, including:

[0128] Based on the determination result, a source position of an image frame corresponding to the video frame extraction result is locked, and based on the source position, a known number of forward and backward video frames are locked in the real-time video stream with the image frame corresponding to the video frame extraction result as the center to obtain a target number of image frame sets;

[0129] Retransmitting a target number of image frames to a video analysis model based on a communication interface, and classifying the target number of image frames bit by bit based on the retransmission result to obtain inter-frame difference parameters;

[0130] Determine the motion region between the target number of image frame sets based on the inter-frame difference parameter, and determine the spatial span of the preliminary fireworks detection result in the image frame set based on the motion region;

[0131] When the spatial span is greater than the preset requirement, the fireworks area in each image frame in the image frame set is calibrated based on the preliminary fireworks detection result, and the edge of the fireworks in the fireworks area is monitored based on the calibration result to obtain the fireworks outline;

[0132] The morphological features of the fireworks are determined based on the fireworks contours, and a re-inspection of a target number of image frame sets is completed based on the fireworks features.

[0133] In this embodiment, the source position refers to the specific position of an image frame corresponding to the video frame extraction result in the original video stream.

[0134] In this embodiment, performing a known number of forward and backward video frame locking based on the source position in the real-time video stream with an image frame corresponding to the video frame extraction result as the center means selecting a corresponding number of multiple image frames in the real-time video stream according to the image frame corresponding to the video frame extraction result, including image frames before and after the video frame extraction result.

[0135] In this embodiment, the target number is the same as the known number and is set in advance.

[0136] In this embodiment, the inter-frame difference parameter refers to the image difference between adjacent image frames in a target number of image frames.

[0137] In this embodiment, the motion area refers to a monitoring area corresponding to a target number of image frame sets.

[0138] In this embodiment, the spatial span refers to the proportion of the preliminary fireworks detection results in the image frame set, that is, how many image frames contain the preliminary fireworks detection results.

[0139] In this embodiment, the preset requirements are set in advance.

[0140] In this embodiment, the fireworks outline refers to the shape of the fireworks in the image frame.

[0141] In this embodiment, the fireworks shape feature refers to the specific shape of the fireworks determined based on the fireworks outline.

[0142] In this embodiment, completing the re-inspection of the target number of image frame sets based on the fireworks features means comparing the obtained fireworks morphological features with the standard fireworks morphology. If the two meet a preset similarity standard, it is determined that fireworks do exist.

[0143] The beneficial effect of the above technical solution is: by locking the target number of image frames according to the source position of an image frame corresponding to the video frame extraction result, and transmitting the locked target number of image frames to the video analysis model for analysis, the spatial span of the preliminary fireworks detection result in the image frame set and the fireworks contour are locked, and finally, the fireworks morphological features contained in the image frame set are determined according to the fireworks contour, so as to realize accurate and reliable re-inspection of fireworks according to the fireworks morphological features, thereby ensuring the accuracy and reliability of the re-inspection and avoiding repeated false alarms.

[0144] Embodiment 8:

[0145] Based on Example 1, this example provides a method for detecting and warning fireworks based on AI video analysis. In S3, when fireworks are determined to exist based on the re-inspection result, a real-time alarm page is displayed based on the fireworks warning supervision platform, including:

[0146] Obtaining a re-inspection result, and when the re-inspection result determines that fireworks are present, determining a video surveillance area corresponding to the real-time video stream based on the image frame set;

[0147] Extracting environmental features of the video surveillance area, and determining the location of fireworks based on the environmental features, and determining the severity level of fireworks based on the re-inspection results, and generating fireworks warning information based on the location of fireworks and the severity level of fireworks;

[0148] The fireworks warning information is fed back to the fireworks warning supervision platform, and a fireworks warning page is generated on the fireworks warning supervision platform based on the fireworks warning information for fireworks warning display.

[0149] In this embodiment, the fireworks position refers to the specific location where the fireworks occur.

[0150] The beneficial effects of the above technical solution are: by determining the video surveillance area where fireworks occur according to the re-inspection results, and accurately and effectively locking the location of the fireworks according to the environmental characteristics of the video surveillance area, secondly, determining the severity level of the fireworks according to the re-inspection results, and generating corresponding fireworks warning information according to the severity level of the fireworks and the location of the fireworks, and finally, feeding back the fireworks warning information to the fireworks warning supervision platform to achieve accurate and effective display of the fireworks warning.

[0151] Embodiment 9:

[0152] Based on Example 1, this example provides a method for detecting and warning fireworks based on AI video analysis. In S3, a fireworks warning notification is synchronously sent to a management terminal, including:

[0153] Obtain the communication address of the management terminal and the terminal address of the fireworks warning and supervision platform, and generate a distributed transmission link based on the communication address and the terminal address;

[0154] Based on the distributed transmission link, the fire and smoke warning information is synchronously sent to the management terminal for fire and smoke warning notification.

[0155] The beneficial effect of the above technical solution is: by building a distributed transmission link based on the communication address of the management terminal and the terminal address of the fireworks warning supervision platform, and synchronously sending the fireworks warning information to the management terminal through the distributed transmission link for fireworks warning notification, it is convenient for the management terminal to understand whether fireworks occur in time, thereby improving the timeliness of fireworks detection.

[0156] Embodiment 10:

[0157] This embodiment provides a smoke and fire detection and warning system based on AI video analysis, such as Figure 3 As shown, including:

[0158] The video frame extraction module is used to connect the fireworks warning and supervision platform with the video device, and extract the video frames of the real-time video stream in the video device based on the connection result;

[0159] A fireworks detection module is used to transmit one image frame at a time to the video analysis model based on the communication interface for preliminary fireworks detection based on the video frame extraction result, and to retransmit a target number of image frames to the video analysis model based on the communication interface for re-inspection when fireworks are present;

[0160] The early warning module is used to determine the presence of fireworks based on the re-inspection results, display a real-time alarm page based on the fireworks early warning supervision platform, and simultaneously send a fireworks early warning notification to the management terminal.

[0161] The beneficial effects of the above technical solution are: by connecting the video equipment with the fireworks warning platform, it is convenient to transmit the real-time video stream of the monitoring area collected by the video equipment to the fireworks warning platform for real-time analysis; secondly, the real-time video stream is sampled, and the obtained image frames are transmitted to the video analysis model for preliminary fireworks detection, thereby improving the efficiency of preliminary fireworks detection; at the same time, when the preliminary fireworks detection determines that fireworks exist, the image frame set of target data is locked from the real-time video stream for re-inspection, thereby improving the accuracy and precision of fireworks detection; finally, when it is determined that fireworks exist according to the re-inspection results, a fireworks warning notification is issued, thereby ensuring the accuracy of fireworks warning and avoiding repeated false alarms.

[0162] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for fire detection and early warning based on AI video analysis, characterized in that: include: S1: Connect the fireworks warning and monitoring platform with the video device, and extract frames from the real-time video stream in the video device based on the connection result; S2: Based on the video frame extraction result, one image frame is transmitted to the video analysis model through the communication interface each time for preliminary fireworks detection, and when fireworks are present, a target number of image frame sets are transmitted back to the video analysis model through the communication interface for re-inspection; S3: When it is determined based on the re-inspection results that fireworks exist, a real-time alarm page is displayed based on the fireworks warning supervision platform, and a fireworks warning notification is simultaneously sent to the management terminal; In S2, based on the video frame extraction result, one image frame is transmitted to the video analysis model based on the communication interface each time for preliminary fireworks detection, and when fireworks exist, a target number of image frame sets are transmitted back to the video analysis model based on the communication interface for re-inspection, including: Obtain an image frame corresponding to the video frame extraction result, and determine a data transmission rule based on the communication protocol of the communication interface; Extracting semantics from the data transmission rules, determining a syntax execution command in the data transmission rules based on the extracted semantics, and converting the format of an image frame based on the syntax execution command to obtain transmittable image data; The transmittable image data is transmitted to the video analysis model based on the communication interface, and a frame of the image is divided into M sampled small images based on the transmission result; The three-primary-color grayscale average values ​​of each sampled image in the color space are extracted based on the RGB color channels, and the three-primary-color grayscale average values ​​are weighted and summarized to obtain a color histogram corresponding to one image frame. Converting the color histogram into a low-dimensional color histogram, quantizing each color component in the low-dimensional color histogram to a normalized range, and obtaining the color proportion in an image frame based on the quantization result; Determine the degree of membership of the color proportion relative to the reference fireworks color feature, and when the degree of membership is higher than a preset membership threshold, determine that fireworks exist; Based on the determination result, a target number of image frame sets are transmitted to the video analysis model via a communication interface for re-inspection.

2. The method for smoke and fire detection and early warning based on AI video analysis according to claim 1, characterized in that: In S1, the fireworks warning monitoring platform is connected with the video equipment, including: Obtain terminal parameters of different video devices, and determine service standards and communication protocols of different video devices based on the terminal parameters; Different video devices are classified based on service standards, and based on the classification results, parameter compatibility adaptation is performed on the preset gateway device according to the communication protocol, and different types of video devices are connected to the preset gateway device based on parameter compatibility adaptation; At the same time, the data interface standards of the fireworks warning and supervision platform are unified, and the data interface after the interface standard is unified is connected with the preset gateway device after parameter compatibility adaptation to complete the connection between the fireworks warning and supervision platform and the video equipment.

3. The method for smoke and fire detection and early warning based on AI video analysis according to claim 1, characterized in that: In S1, based on the docking result, video frames are extracted from the real-time video stream in the video device, including: Reading a real-time video stream in the video device based on the docking result, and determining an image frame included in the real-time video stream based on the reading result; Extracting corresponding image content from the real-time video stream frame by frame based on the image frame, obtaining each frame of image data based on the image content, and obtaining a frame image sequence based on each frame of image data; A polling frame extraction mechanism for a frame image sequence is determined based on a video frame extraction index, and video frame extraction is performed on a real-time video stream in a video device based on the polling frame extraction mechanism.

4. The method for smoke and fire detection and early warning based on AI video analysis according to claim 1, characterized in that: Based on the determination result, the target number of image frame sets are transmitted to the video analysis model through the communication interface for re-inspection, including: Based on the determination result, a source position of an image frame corresponding to the video frame extraction result is locked, and based on the source position, a known number of forward and backward video frames are locked in the real-time video stream with the image frame corresponding to the video frame extraction result as the center to obtain a target number of image frame sets; Retransmitting a target number of image frames to a video analysis model based on a communication interface, and classifying the target number of image frames bit by bit based on the retransmission result to obtain inter-frame difference parameters; Determine the motion region between the target number of image frame sets based on the inter-frame difference parameter, and determine the spatial span of the preliminary fireworks detection result in the image frame set based on the motion region; When the spatial span is greater than the preset requirement, the fireworks area in each image frame in the image frame set is calibrated based on the preliminary fireworks detection result, and the edge of the fireworks in the fireworks area is monitored based on the calibration result to obtain the fireworks outline; The morphological features of the fireworks are determined based on the fireworks contours, and a re-inspection of a target number of image frame sets is completed based on the fireworks features.

5. A fire detection and warning system based on AI video analysis, characterized in that: include: The video frame extraction module is used to connect the fireworks warning and supervision platform with the video device, and extract the video frames of the real-time video stream in the video device based on the connection result; A fireworks detection module is used to transmit one image frame at a time to the video analysis model based on the communication interface for preliminary fireworks detection based on the video frame extraction result, and to retransmit a target number of image frames to the video analysis model based on the communication interface for re-inspection when fireworks are present; The early warning module is used to determine the presence of fireworks based on the re-inspection results, display a real-time alarm page based on the fireworks early warning supervision platform, and simultaneously send a fireworks early warning notification to the management terminal; Among them, the smoke and fire detection module includes: Obtain an image frame corresponding to the video frame extraction result, and determine a data transmission rule based on the communication protocol of the communication interface; Extracting semantics from the data transmission rules, determining a syntax execution command in the data transmission rules based on the extracted semantics, and converting the format of an image frame based on the syntax execution command to obtain transmittable image data; The transmittable image data is transmitted to the video analysis model based on the communication interface, and a frame of the image is divided into M sampled small images based on the transmission result; The three-primary-color grayscale average values ​​of each sampled image in the color space are extracted based on the RGB color channels, and the three-primary-color grayscale average values ​​are weighted and summarized to obtain a color histogram corresponding to one image frame. Converting the color histogram into a low-dimensional color histogram, quantizing each color component in the low-dimensional color histogram to a normalized range, and obtaining the color proportion in an image frame based on the quantization result; Determine the degree of membership of the color proportion relative to the reference fireworks color feature, and when the degree of membership is higher than a preset membership threshold, determine that fireworks exist; Based on the determination result, a target number of image frame sets are transmitted to the video analysis model via a communication interface for re-inspection.

Images