Artificial intelligence fireworks analysis method and system based on video images

Abstract

The present invention provides an artificial intelligence fireworks analysis method based on video images, including the first step of video transcoding; the second step of video data preprocessing; the third step of presetting an image recognition model; the fourth step of outputting an image recognition result; and the fifth step of regional determination of image recognition result information. Fireworks recognition is based on an artificial intelligence algorithm, using image data as an analysis object to identify fireworks in an image; the model can further improve the recognition rate and eliminate false alarms through retraining; thus, the system's video merging mode can achieve optimal use of computing power hardware resources, greatly saving hardware costs.

Description

Technical Field

[0001] The present invention relates to the technical field of fireworks recognition, and in particular to an artificial intelligence fireworks analysis method and system based on video images. Background Art

[0002] Existing fire alarm systems are divided into two categories: detectors, such as smoke and temperature sensors, which detect smoke concentration and temperature at close range; and infrared video systems, which use infrared cameras to detect the temperature of the target location at a long distance.

[0003] Detectors have a time delay in sensing smoke and fire. For example, smoke detectors only trigger an alarm when the smoke concentration reaches a certain threshold, which may have occurred some time before the smoke or fire started. Furthermore, detectors have a limited coverage area, detecting only a 10-square-meter area. They can only be used indoors and cannot be used in open outdoor spaces.

[0004] Infrared cameras are specialized equipment that are expensive and require installation and construction, which brings a lot of engineering workload. Infrared cameras identify by temperature and have a high recognition rate for flames, but cannot identify smoke in the early stages of a fire. Infrared cameras are prone to false alarms for high-temperature objects. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide an artificial intelligence fireworks analysis method and system based on video images to solve the problems raised in the above background technology. The present invention solves the problems in the existing technology by establishing fireworks recognition on the basis of artificial intelligence algorithms, taking image data as the analysis object, and identifying fireworks in images; and further improving the recognition rate through sample training, eliminating false alarms.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an artificial intelligence fireworks analysis method based on video images, comprising the following steps:

[0007] Step 1: Video transcoding

[0008] Acquire video data from multiple video sources and transcode it into RTMP format to ensure data compatibility between different types of acquisition devices. At the same time, compress the video resolution to the recognition model resolution.

[0009] The second step is video data preprocessing

[0010] S2-1, performing frame sampling on the video data to reduce the frame rate of each frame image in the video data;

[0011] S2-2, performing frame identification on each image frame, adding a picture identification code ID to each image frame, converting the picture identification code ID into binary, generating a sequence of pictures corresponding to the picture identification code ID, and superimposing the generated sequence of pictures onto the original frame position;

[0012] S2-3, frame merging of the video data after frame identification

[0013] Extract one frame of image from multiple video data at a preset sampling time point and put it into the frame merging queue to serve as 1 to n frames of merged video data, where n is the number of video data sources, and send the extracted source video frames to the next sampling time point for output;

[0014] S2-4, return to step S2-1, loop;

[0015] Step 3: Preset image recognition model

[0016] S3-1, obtaining a sampled image training sample, wherein the sampled image training sample is annotated with at least first training feature information and second training feature information, wherein the first training feature information is used to represent smoke and fire information in the video image, and the second training feature information is used to represent no smoke and fire information in the video image;

[0017] S3-2, training a convolutional neural network using the first training feature information and the second training feature information to obtain an image recognition model;

[0018] Step 4: Output image recognition results

[0019] S4-1, based on the image recognition model, taking each frame of the image in the picture identification code ID as an analysis object, extracting the image feature information;

[0020] S4-2, comparing the image feature information with the training feature information in the acquired sample image training sample to obtain a result correspondence;

[0021] S4-3, outputting image recognition result information based on the extraction and recognition process of S4-1.

[0022] Furthermore, the method further includes the following steps:

[0023] Step 5: Regional determination of image recognition result information

[0024] The image recognition result information is delineated in an identification area to ensure that the image recognition result information is within the range of the external monitoring image. The specific implementation method is: the image recognition result information is delineated in an area based on the artificial intelligence fireworks analysis result of the video image.

[0025] Furthermore, based on the fourth step, it is also necessary to judge the uniqueness of the image recognition result information obtained to ensure that the image recognition model outputs correct information and reduces the false alarm rate. The specific implementation method is: when an alarm occurs, the authenticity of the alarm is manually confirmed. If it is confirmed to be a false alarm, it is added to the negative sample set, and re-training is carried out after a certain amount of negative samples are collected.

[0026] Furthermore, in the third step, the image recognition result information includes flame information and smoke information, wherein:

[0027] If the image recognition result information reaches the alarm threshold set by the external system, storage information of this image recognition result information is generated, and the storage information includes image information, video image storage path information, device terminal information of the collected image information, alarm time information and alarm type information.

[0028] Furthermore, the sampled image training samples are also annotated with third training feature information and fourth training feature information, wherein:

[0029] The third training feature information is used to characterize the location information of the area where fire or smoke occurs in the video image, and the fourth training feature information is used to characterize the type information of the fire or smoke occurs in the video image, and

[0030] If the first training feature information and the second training feature information are used to train the convolutional neural network, the loss values ​​of the first training feature information and the second training feature information are calculated respectively through a preset loss function, so that the image recognition model can automatically adjust its internal parameters by learning the sampled image training samples, thereby improving the recognition rate. The specific implementation method includes: extracting features from the original image according to different block sizes; fusing the extracted feature information; convolving the fused information; and calculating the loss function.

[0031] Furthermore, the present invention proposes an artificial intelligence fireworks analysis system based on video images.

[0032] It includes a video data acquisition module, a video data processing module, a training module and an output module, wherein:

[0033] A video data acquisition module is used to acquire target video data from multiple video sources;

[0034] A video data processing module is used to perform format transcoding on the target video data and simultaneously complete preprocessing of the image carried in the target video to obtain the target video data image after frame merging;

[0035] A training module obtains sampled image training samples and is used to train the convolutional neural network using training feature information in the sampled image training samples to obtain an image recognition model;

[0036] The output module is used to input the target video data image into the image recognition model to obtain image recognition result information of the target video data.

[0037] Furthermore, the fire analysis system proposed in the present invention further includes a video data occurrence area determination module, a time determination module, a storage module, and an alarm information push module, wherein:

[0038] A video data generation area determination module is used to monitor and output the area location information of the image recognition result information on an external terminal monitoring device;

[0039] A time determination module is used to determine the correctness and uniqueness of the image recognition result information output by the training module to ensure that the image recognition model outputs correct information and reduce the false alarm rate of the output module;

[0040] The storage module is used to store the image recognition result information output by the training module and convert it into storage information for use by the sampling image training samples of the training module;

[0041] The warning information push module is used to build a third-party execution software to push warning information that reaches the warning threshold in the image recognition result information to the preset contacts of the system to complete the warning prompt.

[0042] Furthermore, the video data processing module includes a video data transcoding unit, a video data frame sampling processing unit, a video data frame identification processing unit and a video data frame merging processing unit, wherein:

[0043] The video data transcoding processing unit is used to transcode the target video data from the acquired multiple video sources into RTMP format to ensure data compatibility between different models of acquisition devices;

[0044] The video data frame sampling processing unit is used to perform frame sampling on the transcoded target video data to reduce the frame rate of each frame image in the target video data;

[0045] The video data frame identification processing unit is used to perform frame identification on each frame of the image to ensure that the system can identify the video data corresponding to the frame image and processed by the frame identification;

[0046] The video data frame merging processing unit is used to perform frame merging on the video data after frame identification, so as to improve the computing power redundancy of the system hardware device.

[0047] As a third aspect of the present invention, an electronic device for artificial intelligence fireworks analysis based on video images is proposed, comprising at least one processor and a memory that establishes an information interaction channel with at least one of the processors, wherein:

[0048] Each of the memories stores a computer program executable by at least one of the processors;

[0049] The computer program is executed by at least one of the processors, so that the at least one processor executes the video-image-based artificial intelligence fireworks analysis method and / or the video-image-based artificial intelligence fireworks analysis system.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] In the present invention, fireworks recognition is based on an artificial intelligence algorithm, using image data as the analysis object to identify fireworks in images; the model can further improve the recognition rate and eliminate false alarms through retraining; thereby, the system's video merging mode can achieve optimal use of computing power hardware resources, significantly saving hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0053] Figure 1 Schematic diagram of the implementation process of the video image-based artificial intelligence fireworks analysis method proposed in one embodiment of the present invention;

[0054] Figure 2 Schematic diagram of the process of video merging in the video image-based artificial intelligence fireworks analysis method proposed in one embodiment of the present invention;

[0055] Figure 3 Schematic diagram of parsing the original video ID when merging videos in one embodiment of the present invention;

[0056] Figure 4 This is a diagram of an implementation scenario for fireworks recognition based on an image recognition model proposed in one embodiment of the present invention;

[0057] Figure 5 This is an implementation scenario for performing region determination on image recognition result information proposed in one embodiment of the present invention. Figure 2 .

[0058] Figure 6This is a schematic diagram of the principle of automatically adjusting the internal parameters of an image recognition model by learning from sampled image training samples according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the present invention and should not be regarded as the entire invention or as a limitation or restriction of the technical solution of the present invention.

[0060] like Figure 1-6 As shown, as an embodiment of the present invention, the present invention provides a technical solution: an artificial intelligence fireworks analysis method based on video images, comprising the following steps:

[0061] Step 1: Video transcoding

[0062] Acquire video data from multiple video sources and transcode it into RTMP format to ensure data compatibility between different types of acquisition devices. At the same time, compress the video resolution to the recognition model resolution.

[0063] The second step is video data preprocessing

[0064] S2-1. Frame sampling is performed on the video data to reduce the frame rate of each frame in the video data. It should be noted that the general method for frame sampling of video data is full-frame analysis. By using frame skipping sampling and merging, the number of frames analyzed per video channel per unit time is reduced, so that the GPU computing resources originally required to analyze one video channel can be used to analyze multiple channels.

[0065] S2-2, performing frame identification on each image frame, adding a picture identification code ID to each frame image, converting the picture identification code ID into binary, generating a sequence of pictures corresponding to the picture identification code ID, and superimposing the generated sequence of pictures onto the original frame position. It should be understood that the original frame position here refers to the video data in the frame sampling stage, which is compressed to the same resolution after sampling and then superimposed with the identification ID;

[0066] S2-3, frame merging of the video data after frame identification

[0067] By presetting the sampling rate, one frame of image is extracted from the multi-channel video data at the sampling time point and placed into the frame merging queue to serve as 1 to n frames of the merged video data, where n is the number of video data sources, and the extracted source video frames are sent to the next sampling time point for output. It can be understood that in the specific implementation, after extracting one frame from each of the three channels of video, the source video frames between the next sampling point are discarded. Then the source video frames discarded in this process refer to the video frames that have not been extracted from the three channels of video, which enter the next sampling point for re-extraction.

[0068] Based on the technical concept of the second step above, it can be understood that, on the one hand, the commonly used video frame rate is 25fps, which causes the redundancy of computing power for each channel of 5fps. On the other hand, since the change between two frames is very small in actual use (in the case of 25fps, the interval is only 40ms), if each frame image is recognized, it will greatly waste computing power resources. Therefore, the sampling method is used to reduce the frame rate. However, the reduction in frame rate will further lead to redundancy of computing power. Therefore, the present invention adopts the frame merging method to merge multiple video frames into one video stream for calculation, so as to ensure the full utilization of hardware resources.

[0069] like Figure 2 As shown, taking the merging of three videos as an example, the sampling rate is 1 / 3. At the sampling time point, one frame is extracted from each of the three videos and placed in the queue as frames 1-3 of the merged video. The source video frames between the next sampling point are discarded, and this cycle repeats. At the same time, since the last recognized video stream is formed by the merging of multiple original streams, when the AI ​​recognition alarm occurs, the system cannot distinguish the original video ID corresponding to the frame. It is necessary to add a system-recognizable identifier to each frame of image data, and the data structure of the video code stream cannot be changed. The system implements this function by adding an image identification code to the peripheral position of the image. It is understandable that the peripheral position can be adjusted in the specific implementation. When the present invention is implemented, a color block with a height of 20 pixels and a width of 20 pixels is used as a color block starting from the first pixel in the upper left corner of the image. Black represents 1 and white represents 0. Six color blocks are arranged in sequence to express 0-31. If the number of channels needs to be expanded later, the number of color blocks can be increased. With each additional color block, the number of IDs that can be expressed doubles.

[0070] like Figure 3 As shown, when the present invention is specifically implementing the identification of the original video ID, the video ID is converted into binary and converted into a corresponding black and white block sequence image. During frame sampling, the image is superimposed on a fixed position of the original frame. When the AI ​​recognition alarm occurs, the original video ID is reversely parsed according to this rule, black = 1, white = 0, the parsing result is 101101, and the decimal result is 45.

[0071] S2-4, return to step S2-1, loop;

[0072] The third step is to preset the image recognition model. It can be understood that the image recognition model is based on the YOLO algorithm. It takes each frame of the input video code as the analysis object, extracts the objects in the image, and obtains the corresponding relationship with the result. The output results include two categories: flame and smoke. It should be noted that when modeling, a large number of pictures of smoke and fire (positive samples) and pictures of no smoke and fire (negative samples) are first collected, and the regional location and type of fire / smoke are marked. The labeled picture data is input into the model for training. The model automatically adjusts the internal parameters by learning from the samples, thereby achieving the effect of improving the recognition rate. Its specific implementation method is as follows:

[0073] S3-1. Obtain a sampled image training sample. The image training sample is annotated with at least first training feature information and second training feature information. The first training feature information is used to represent smoke and fire information in the video image, and the second training feature information is used to represent the absence of smoke and fire information in the video image. It is understood that the smoke and fire information needs to include the type of smoke and fire, as well as the location of the smoke and fire area in the image (defined in pixels or relative percentage).

[0074] S3-2, using the first training feature information and the second training feature information to train the convolutional neural network to obtain an image recognition model. It can be understood that the original YOLO algorithm is a generalized target recognition algorithm. Therefore, the present invention combines the characteristics of the fireworks recognition dataset and replaces the optimization function from SGD to Adam, thereby having a high probability of improving the model training effect. At the same time, it switches from single thread to multi-thread, thereby improving the concurrent computing capability.

[0075] Step 4: Output image recognition results

[0076] S4-1, based on the image recognition model, each frame of the image in the picture identification code ID is used as the analysis object to extract image feature information. Extracting image feature information is to identify the probability of the corresponding identification object (smoke, fire) appearing in the image through model calculation;

[0077] S4-2, comparing the image feature information with the training feature information in the acquired sample image training sample to obtain a result correspondence;

[0078] S4-3: Output image recognition result information based on the extraction and recognition process of S4-1.

[0079] In one embodiment of the present invention, the present invention further comprises the following steps:

[0080] Step 5: Regional determination of image recognition result information

[0081] The image recognition result information is delineated in the recognition area to ensure that the image recognition result information is within the range of the external monitoring image. The specific implementation method is as follows: the image recognition result information is delineated based on the artificial intelligence fireworks analysis results of the video image. It can be understood that based on the area determination, for scenes where only a specific area in the image needs to be monitored, the recognition area can be delineated on the image. When the system recognizes a fireworks alarm, it will further determine the area. If it is outside the set area, the result will be ignored.

[0082] At the same time, in order to solve the problem of false alarms in image recognition result information, the present invention summarizes the recognition conditions of multiple consecutive frames to reduce the false alarm rate. For example, if more than 5 frames appear in 10 consecutive seconds, an alarm will be triggered, which can effectively filter out occasional false alarms.

[0083] Based on the fourth step, it is also necessary to judge the uniqueness of the image recognition result information obtained to ensure that the image recognition model outputs correct information and reduces the false alarm rate. The specific implementation method is: when an alarm occurs, the authenticity of the alarm is manually confirmed. If it is confirmed to be a false alarm, it is added to the negative sample set, and re-training is carried out after a certain amount of negative samples are collected.

[0084] It should be noted that in the third step, the image recognition result information includes flame information and smoke information, among which,

[0085] If the image recognition result information reaches the alarm threshold set by the external system, the storage information of this image recognition result information will be generated. The storage information includes image information, video image storage path information, device terminal information of the collected image information, alarm time information and alarm type information. In the specific implementation, when the system recognizes a fireworks alarm, the system determines that the alarm is triggered, and the corresponding image of the alarm will be saved. The storage path and the alarm camera ID, time, and alarm type will be written to the database for viewing and subsequent training. At the same time, the push function will be triggered after the alarm, and the preset contacts will be notified as soon as possible through the system platform, SMS, voice call, etc.

[0086] In one embodiment of the present invention, the sampled image training sample is further annotated with third training feature information and fourth training feature information, wherein:

[0087] The third training feature information is used to characterize the location information of the area where fire or smoke occurs in the video image, and the fourth training feature information is used to characterize the type information of the fire or smoke occurring in the video image, and

[0088] If the first training feature information and the second training feature information are used to train the convolutional neural network, the loss values ​​of the first training feature information and the second training feature information are calculated respectively through a preset loss function, so that the image recognition model can automatically adjust its internal parameters by learning the sampled image training samples, thereby improving the recognition rate. The specific implementation method includes: 1) extracting features from the original image according to different block sizes; 2) fusing the extracted feature information; 3) convolving the fused information; 4) calculating the loss function.

[0089] It is understandable that, on the one hand, there is a delay in the perception of smoke and fire by detector equipment. Taking smoke detection as an example, the alarm will only be triggered when the smoke concentration reaches a certain threshold. At this time, some time may have passed since the smoke and fire started. The video smoke and fire analysis proposed in the present invention can identify the fire in the early stage of the fire, just when the smoke and fire start. At the same time, due to the limited coverage range of detector equipment, it can only detect the surrounding area of ​​10 square meters. The video smoke and fire analysis can cover an area of ​​thousands of square meters within the visible range of the camera, so it is suitable for use in indoor and outdoor scenes.

[0090] On the other hand, considering that infrared cameras are special equipment with high prices, they need to be installed and constructed, which brings a lot of engineering workload; external cameras recognize flames through temperature, and have a high recognition rate for flames, but cannot recognize smoke in the early stages of a fire; and infrared cameras are prone to false alarms for high-temperature objects. Therefore, some buyers build image recognition models and analyze video fireworks based on ordinary videos. The cameras are inexpensive and can use existing cameras without the need for reinstallation. They can recognize both smoke and fire and will not react to non-flame high-temperature objects, which is suitable for the existing market.

[0091] As another embodiment of the present invention, the method of identifying fireworks information based on the YOLO algorithm established by the image recognition model can be replaced by a CNN convolutional neural network model to achieve the optimal use of computing power hardware resources and significantly save hardware costs.

[0092] As a second aspect of the present invention, an artificial intelligence fireworks analysis system based on video images is proposed, which includes a video data acquisition module, a video data processing module, a training module and an output module, wherein:

[0093] A video data acquisition module is used to acquire target video data from multiple video sources;

[0094] The video data processing module is used to perform format transcoding on the target video data and simultaneously complete preprocessing of the image carried in the target video to obtain the target video data image after frame merging;

[0095] A training module obtains sampled image training samples and is used to train the convolutional neural network using training feature information in the sampled image training samples to obtain an image recognition model;

[0096] The output module is used to input the target video data image into the image recognition model to obtain the image recognition result information of the target video data.

[0097] It should be noted that the system proposed in the present invention also includes a video data generation area determination module, a time determination module, a storage module and an alarm information push module, wherein:

[0098] The video data generation area determination module is used to monitor the regional location information of the output image recognition result information on the external terminal monitoring device;

[0099] The time determination module is used to determine the correctness and uniqueness of the image recognition result information output by the training module to ensure that the image recognition model outputs correct information and reduce the false alarm rate of the output module;

[0100] The storage module is used to store the image recognition result information output by the training module and convert it into storage information for use by the sampling image training samples of the training module;

[0101] The warning information push module is used to build third-party execution software to push warning information that reaches the warning threshold in the image recognition result information to the system's preset contacts to complete the warning prompt.

[0102] At the same time, the video data processing module includes a video data transcoding unit, a video data frame sampling processing unit, a video data frame identification processing unit and a video data frame merging processing unit, wherein,

[0103] The video data transcoding processing unit is used to transcode the target video data from the acquired multiple video sources into RTMP format to ensure data compatibility between different models of acquisition devices;

[0104] The video data frame sampling processing unit is used to perform frame sampling on the transcoded target video data to reduce the frame rate of each frame image in the target video data;

[0105] The video data frame identification processing unit is used to perform frame identification on each frame image to ensure that the system can identify the video data corresponding to the frame image after frame identification processing;

[0106] The video data frame merging processing unit is used to merge the video data after frame identification to improve the computing power redundancy of the system hardware equipment.

[0107] As a third aspect of the present invention, an electronic device for artificial intelligence fireworks analysis based on video images is proposed, comprising at least one processor and a memory that establishes an information interaction channel with the at least one processor, wherein:

[0108] Each memory stores a computer program executable by at least one processor;

[0109] The computer program is executed by at least one processor, so that the at least one processor executes the video image-based artificial intelligence fireworks analysis method and / or the video image-based artificial intelligence fireworks analysis system.

[0110] The technical scope of the present invention is not limited to the contents of the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. An artificial intelligence fireworks analysis method based on video images, characterized by: The following steps are involved: The first step is video transcoding Obtain video data from multiple video sources and transcode them into RTMP format to ensure data compatibility between different types of acquisition devices; The second step is video data preprocessing S2-1, performing frame sampling on the video data to reduce the frame rate of each frame image in the video data; S2-2, performing frame identification on each frame of the image, adding a picture identification code ID to each frame of the image, converting the picture identification code ID into binary, generating a sequence of pictures corresponding to the picture identification code ID, and superimposing the generated sequence of pictures to the original frame position; S2-3, frame merging of the video data after frame identification By using a preset sampling rate, one frame of image is extracted from multiple video data at a sampling time point and put into a frame merging queue to serve as 1 to n frames of merged video data, where n is the number of video data sources, and the extracted source video frames are sent to the next sampling time point for output; S2-4, return to step S2-1, and repeat in sequence; Step 3: Preset image recognition model S3-1, obtaining a sampled image training sample, wherein the image training sample is annotated with at least first training feature information and second training feature information, wherein the first training feature information is used to characterize information about smoke and fire in a video image, and the second training feature information is used to characterize information about no smoke and fire in a video image; S3-2, training a convolutional neural network using the first training feature information and the second training feature information to obtain an image recognition model; Step 4: Output image recognition results S4-1, based on the image recognition model, taking each frame of the image in the picture identification code ID as an analysis object, extracting the image feature information; S4-2, comparing the image feature information with the training feature information in the acquired sampled image training sample to obtain a result correspondence; S4-3, outputting image recognition result information according to the extraction and recognition process of S4-1.

2. The artificial intelligence fireworks analysis method based on video images according to claim 1 is characterized in that: The following steps are also included: Step 5: Regional determination of image recognition result information The image recognition result information is delineated in an identification area to ensure that the image recognition result information is within the range of the external monitoring image. The specific implementation method is: the image recognition result information is delineated in an area based on the artificial intelligence fireworks analysis result of the video image.

3. The artificial intelligence fireworks analysis method based on video images according to claim 1 is characterized in that: Based on the fourth step, it is also necessary to judge the uniqueness of the image recognition result information obtained to ensure that the image recognition model outputs correct information and reduce the false alarm rate. The specific implementation method is: when an alarm occurs, the authenticity of the alarm is manually confirmed. If it is confirmed to be a false alarm, it is added to the negative sample set, and re-training is carried out after a certain amount of negative samples are collected.

4. The artificial intelligence fireworks analysis method based on video images according to claim 1 is characterized in that: In the third step, the image recognition result information includes flame information and smoke information, wherein: If the image recognition result information reaches the alarm threshold set by the external system, storage information of the image recognition result information is generated, and the storage information includes image information, video image storage path information, device terminal information of the collected image information, alarm time information and alarm type information.

5. The artificial intelligence fireworks analysis method based on video images according to claim 1 is characterized in that: The sampled image training sample is also annotated with third training feature information and fourth training feature information, wherein: The third training feature information is used to characterize the location information of the area where fire or smoke occurs in the video image, the fourth training feature information is used to characterize the type information of the fire or smoke occurring in the video image, and, If the first training feature information and the second training feature information are used to train the convolutional neural network, the loss values ​​of the first training feature information and the second training feature information are calculated respectively through a preset loss function, so that the image recognition model can automatically adjust its internal parameters by learning the sampled image training samples, thereby improving the recognition rate. The specific implementation method includes: extracting features from the original image according to different block sizes; fusing the extracted feature information; convolving the fused information; and calculating the loss function.

6. A system for the artificial intelligence fireworks analysis method based on video images according to any one of claims 1 to 5, characterized in that: It includes a video data acquisition module, a video data processing module, a training module and an output module, wherein: A video data acquisition module is used to acquire target video data from multiple video sources; A video data processing module is used to perform format transcoding on the target video data and synchronously complete preprocessing of the image carried in the target video to obtain the target video data image after frame merging; A training module obtains sampled image training samples, and is used to train the convolutional neural network through the training feature information in the sampled image training samples to obtain an image recognition model; The output module is used to input the target video data image into the image recognition model to obtain the image recognition result information of the target video data.

7. The artificial intelligence fireworks analysis system based on video images according to claim 6 is characterized by: It also includes a video data occurrence area determination module, a time determination module, a storage module and a warning information push module, wherein: A video data generation area determination module, used for monitoring and outputting the area location information of the image recognition result information on an external terminal monitoring device; A time determination module is used to determine the correctness and uniqueness of the image recognition result information output by the training module, so as to ensure that the image recognition model outputs correct information and reduce the false alarm rate of the output module; The storage module is used to store the image recognition result information output by the training module and convert it into storage information for use by the sampled image training samples of the training module; The warning information push module is used to build a third-party execution software to push the warning information that reaches the warning threshold in the image recognition result information to the preset contact of the system to complete the warning prompt.

8. The artificial intelligence fireworks analysis system based on video images according to claim 6 is characterized by: The video data processing module includes a video data transcoding processing unit, a video data frame sampling processing unit, a video data frame identification processing unit and a video data frame merging processing unit, wherein: The video data transcoding processing unit is used to transcode the target video data in the acquired multiple video sources into RTMP format to ensure data compatibility between different models of acquisition devices; The video data frame sampling processing unit is used to perform frame sampling on the transcoded target video data to reduce the frame rate of each frame image in the target video data; The video data frame identification processing unit is used to perform frame identification on each frame of the image to ensure that the system can identify the video data corresponding to the frame of the image and processed by the frame identification; The video data frame merging processing unit is used to merge frames of the video data after frame identification, so as to improve the computing power redundancy of the system hardware device.

9. An electronic device for artificial intelligence fireworks analysis based on video images, characterized in that: The device comprises at least one processor and a memory for establishing an information interaction channel with at least one of the processors, wherein: Each of the memories stores a computer program executable by at least one of the processors; The computer program is executed by at least one of the processors so that at least one of the processors executes the video-image-based artificial intelligence fireworks analysis method according to any one of claims 1 to 5 and / or the video-image-based artificial intelligence fireworks analysis system according to any one of claims 6 to 8.

Images