Fire accident video analysis method, device, equipment, storage medium and product

Abstract

The invention relates to the technical field of fire accident analysis, in particular to a fire accident video analysis method, device and equipment, a storage medium and a product. According to the method, fire feature data and personnel and vehicle data in fire scene video data are extracted; classifying and identifying the fire feature data to obtain flame information, smoke information and light and shadow information; performing super-resolution reconstruction and recognition on a personnel behavior image in the personnel and vehicle data to obtain personnel feature information; performing motion blur removal processing on a vehicle image in the personnel and vehicle data to obtain license plate feature information; and finally, based on the feature information, generating a fire accident video analysis report. According to the method, multiple video analysis technologies are integrated, the fire feature data in the fire scene video data are automatically extracted, and the efficiency and accuracy of data extraction and recognition are effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of fire accident analysis, and in particular to a fire accident video analysis method, device, equipment, storage medium and product. Background Art

[0002] With the acceleration of urbanization and the increase in building density, the frequency and complexity of fire accidents are increasing. Rapid response and accurate analysis of fire accidents have become urgent problems for fire departments and related units. Traditional fire accident analysis mainly relies on manual viewing of surveillance videos to play back and judge the on-site situation. The method is cumbersome and inefficient. It not only consumes a lot of human resources, but is also easily interfered by human factors, resulting in incomplete information extraction or inaccurate judgment. In addition, the complex environment of the fire scene, such as light changes, smoke, and the rapid movement of people and vehicles, further increases the difficulty and error rate of manual analysis. In recent years, with the rapid development of artificial intelligence and computer vision technology, automated analysis methods based on video data have gradually become a research hotspot. The performance of deep learning models in image and video recognition has been significantly improved. They can automatically extract a variety of feature information from fire scene videos, such as flames, smoke, light and shadow changes, faces and license plates, and perform efficient classification and recognition. However, most of the existing fire video analysis technologies only detect a single type of feature information, lack comprehensive analysis of multiple feature data, and cannot generate fire accident analysis reports comprehensively and accurately, which leads to poor results in fire scene video analysis. Therefore, how to improve the effect of fire video analysis has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The main purpose of this application is to provide a fire accident video analysis method, device, equipment, storage medium and product, aiming to solve the technical problem of how to improve the fire video analysis effect.

[0004] To achieve the above objectives, the present application provides a fire accident video analysis method, the method comprising the following steps:

[0005] Extracting fire feature data and personnel and vehicle data from the fire scene video data;

[0006] Classifying and identifying the fire characteristic data to obtain flame information, smoke information, and light and shadow information;

[0007] Performing super-resolution reconstruction and recognition on the personnel behavior images in the personnel vehicle data to obtain personnel feature information;

[0008] Performing a motion blur removal process on the vehicle image in the personnel vehicle data to obtain vehicle feature information;

[0009] A fire accident video analysis report is generated based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information.

[0010] In one embodiment, before the step of extracting the fire feature data and the personnel and vehicle data from the fire scene video data, the method further includes:

[0011] Based on the topology of the local storage of on-site monitoring, identify each video input channel;

[0012] Determine the target video input channel according to the device type, resolution, frame rate and video encoding format corresponding to each of the video input channels;

[0013] Based on the target video input channel, obtaining the target monitoring video;

[0014] The target monitoring video is optimized to obtain the fire scene video data.

[0015] In one embodiment, the step of optimizing the target monitoring video to obtain the fire scene video data includes:

[0016] Converting the target surveillance video into a preset standard format;

[0017] Based on a preset data recovery method, the lost frames of the target surveillance video after format conversion are recovered to obtain multiple restored surveillance videos;

[0018] Performing image enhancement processing on the multiple restored surveillance videos to obtain multiple enhanced videos;

[0019] Performing time stamp synchronization on the multiple enhanced videos;

[0020] An integrity check is performed on the enhanced video after time stamp synchronization, and the video that passes the integrity check is used as the fire scene video data.

[0021] In one embodiment, the step of extracting fire feature data and personnel and vehicle data from the fire scene video data includes:

[0022] Preprocessing the fire scene video to obtain a target frame set, wherein the preprocessing includes one or more of video frame decomposition, image enhancement, and region of interest detection;

[0023] Based on a preset fire feature detection model, flame detection, smoke detection, and light and shadow detection are performed on each frame in the target frame set to obtain the fire feature data;

[0024] Based on a preset personnel and vehicle detection model, personnel detection and vehicle detection are performed on each frame in the target frame set to obtain the personnel and vehicle data.

[0025] In one embodiment, the step of performing super-resolution reconstruction and recognition on the personnel behavior image in the personnel vehicle data to obtain personnel feature information includes:

[0026] Extracting a personnel behavior image from the personnel vehicle data, and performing quality evaluation on the extracted personnel behavior image;

[0027] Eliminate the facial images whose quality evaluation values ​​are lower than the preset facial image quality standard evaluation values ​​to obtain quality facial images;

[0028] Standardizing the size, color space, brightness and contrast of the quality facial image to obtain a standard quality facial image;

[0029] Based on a preset super-resolution reconstruction model, reconstructing and enhancing the standard quality face image to obtain a reconstructed face image;

[0030] Based on a preset face recognition algorithm, feature extraction is performed on the reconstructed face image to obtain the person feature information.

[0031] In one embodiment, the step of performing de-motion blurring processing on the vehicle image in the personnel vehicle data to obtain vehicle feature information includes:

[0032] Extracting the vehicle image from the personnel vehicle data, and performing the quality evaluation on the extracted vehicle image;

[0033] Eliminate the vehicle images whose quality evaluation values ​​are lower than the preset vehicle image quality standard evaluation values ​​to obtain quality vehicle images;

[0034] Performing the standardization process on the size, color space, brightness and contrast of the quality vehicle image to obtain a standard quality vehicle image;

[0035] dividing the standard quality vehicle image into a plurality of license plate sub-images, and determining fuzzy information of each license plate sub-image;

[0036] Based on the fuzzy information, each of the license plate sub-images is input into a preset deblurring model to obtain the vehicle feature information.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a fire accident video analysis device, the fire accident video analysis device comprising:

[0038] A data extraction module, used to extract fire feature data and personnel and vehicle data from the fire scene video data;

[0039] An information recognition module is used to classify and recognize the fire characteristic data to obtain flame information, smoke information, and light and shadow information;

[0040] A personnel identification module is used to perform super-resolution reconstruction and identification on the personnel behavior images in the personnel vehicle data to obtain personnel feature information;

[0041] A vehicle identification module, used for performing a motion blur removal process on the vehicle image in the personnel vehicle data to obtain vehicle feature information;

[0042] The target module is used to generate a fire accident video analysis report based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information.

[0043] In addition, to achieve the above-mentioned objectives, the present application also proposes a fire accident video analysis device, which includes: a memory, a processor, and a fire accident video analysis program stored on the memory and executable on the processor, wherein the fire accident video analysis program is configured to implement the steps of the fire accident video analysis method described above.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a fire accident video analysis program is stored. When the fire accident video analysis program is executed by a processor, the steps of the fire accident video analysis method described above are implemented.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the fire accident video analysis method as described above.

[0046] This application extracts fire feature data and personnel and vehicle data from the fire scene video data; then classifies and identifies the fire feature data to obtain flame information, smoke information, and light and shadow information; super-resolution reconstruction and identification of personnel behavior images in personnel and vehicle data to obtain personnel feature information; de-motion blurring of vehicle images in personnel and vehicle data to obtain vehicle feature information; and finally, based on the flame information, smoke information, light and shadow information, personnel feature information, and vehicle feature information, a fire accident video analysis report is generated. This application integrates a variety of video analysis technologies to automatically extract fire feature data, personnel feature information, and vehicle feature information from the fire scene video data, and classifies, identifies, super-resolution reconstructs, and de-motion blurring these information, effectively improving the efficiency and accuracy of data extraction and identification. The classification and identification of fire feature data can accurately assess the starting position and spread direction of the fire; super-resolution reconstruction improves the accuracy of face recognition in complex environments; and license plate de-motion blurring enhances the recognition ability of fast-moving or blurred license plates. Ultimately, the fire accident video analysis report generated based on the above information can reflect the actual situation at the fire scene, provide strong decision-making support for fire emergency response, cause investigation and responsibility determination, thereby improving the effect of fire video analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the first embodiment of the fire accident video analysis method of the present application;

[0048] Figure 2 This is a schematic diagram of a sub-process in the second embodiment of the fire accident video analysis method of this application;

[0049] Figure 3 This is a schematic diagram of a sub-process in the third embodiment of the fire accident video analysis method of the present application;

[0050] Figure 4 This is a schematic diagram of the module structure of a fire accident video analysis device according to an embodiment of the present application;

[0051] Figure 5 Schematic diagram of the equipment structure of the hardware operating environment involved in the fire accident video analysis method in the embodiment of the present application.

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] It should be noted that with the acceleration of urbanization and the increase in building density, the frequency and complexity of fire accidents are constantly increasing. Rapid response and accurate analysis of fire accidents have become urgent problems for fire departments and related units. Traditional fire accident analysis mainly relies on manual viewing of surveillance videos to play back and judge the on-site situation. The method is cumbersome and inefficient. It not only consumes a lot of human resources, but is also easily interfered by human factors, resulting in incomplete information extraction or inaccurate judgment. In addition, the complex environment of the fire scene, such as light changes, smoke, and the rapid movement of people and vehicles, further increases the difficulty and error rate of manual analysis. In recent years, with the rapid development of artificial intelligence and computer vision technology, automated analysis methods based on video data have gradually become a research hotspot. The performance of deep learning models in image and video recognition has been significantly improved. They can automatically extract a variety of feature information from fire scene videos, such as flames, smoke, light and shadow changes, faces and license plates, and perform efficient classification and recognition. However, most existing fire video analysis technologies only detect a single type of feature information, lack comprehensive analysis of multiple feature data, and are unable to generate comprehensive and accurate fire accident analysis reports, which in turn leads to poor results in fire scene video analysis. Therefore, how to improve the effect of fire video analysis has become a technical problem that needs to be solved urgently.

[0056] The main solution of this application is: by extracting fire feature data and personnel and vehicle data from the fire scene video data; then classifying and identifying the fire feature data to obtain flame information, smoke information, and light and shadow information; super-resolution reconstructing and identifying personnel behavior images in the personnel and vehicle data to obtain personnel feature information; de-motion blurring the vehicle images in the personnel and vehicle data to obtain vehicle feature information; finally, generating a fire accident video analysis report based on the flame information, smoke information, light and shadow information, personnel feature information, and vehicle feature information.

[0057] This application integrates multiple video analysis technologies to automatically extract fire feature data and personnel and vehicle feature information from fire scene video data, and classifies, identifies, super-resolution reconstructs, and de-motion blurs this information, effectively improving the efficiency and accuracy of data extraction and identification. The classification and identification of fire feature data can accurately assess the starting location and spread direction of the fire; super-resolution reconstruction improves the accuracy of face recognition in complex environments; and license plate de-motion blur enhances the recognition of fast-moving or blurred license plates. Ultimately, the fire accident video analysis report generated based on the above information can reflect the actual situation at the fire scene, provide strong decision-making support for fire emergency response, cause investigation, and responsibility determination, thereby improving the effectiveness of fire video analysis.

[0058] It should be noted that the execution subject of the method of this embodiment can be a computing service device with data processing, network communication and program running functions, or it can be the above-mentioned fire accident video analysis device with the same or similar functions. This embodiment and the following embodiments will be described by taking the fire accident video analysis device as an example.

[0059] Based on this, the first embodiment of the fire accident video analysis method of this application is proposed, please refer to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the fire accident video analysis method of the present application.

[0060] In this embodiment, the fire accident video analysis method includes the following steps:

[0061] S1: extracting fire feature data and personnel and vehicle data from the fire scene video data;

[0062] It should be noted that fire scene video data refers to the original video data collected at the scene of the fire accident through various monitoring equipment (such as surveillance cameras, mobile phone cameras, driving recorders, etc.). These data contain images, sounds and other relevant information during the fire, and are an important basis for fire accident analysis. Fire feature data refers to specific fire-related information extracted from the fire scene video, including but not limited to flames, smoke, light and shadow changes, and other key visual features that show the state and development of the fire. These data are used to analyze the starting point of the fire, the path of spread, and its intensity. Personnel and vehicle data refers to the facial images and vehicle image data that appear in the fire scene video, which are used to identify the identities of relevant personnel and vehicle information at the fire scene to help determine the activities of personnel and vehicles at the accident scene.

[0063] Specifically, fire feature data and personnel and vehicle data are extracted from the raw video data collected at the fire scene. When extracting fire feature data, the device automatically detects key features such as flames, smoke, and light and shadow changes in the video frame through a pre-trained deep learning model, and records information such as the appearance time, location, shape, and change trend of these features. At the same time, a feature filtering algorithm is used to remove non-fire-related visual information to ensure that the extracted data accurately reflects the actual situation of the fire.

[0064] Furthermore, to extract face and license plate data, a face detection algorithm (such as MTCNN or YOLO model based on deep learning) is used to scan the video frame by frame to identify the appearance of faces and extract their image information; at the same time, a license plate recognition algorithm (such as YOLO or SSD, etc.) is used to detect and extract the image of the license plate area. In order to improve the accuracy of recognition, the device can combine multi-frame analysis methods to verify the reliability of the recognition results by comparing the changes in the positions of faces and license plates in consecutive frames, ensuring that the identity and information of each detected target are clear and unambiguous.

[0065] By automatically extracting fire feature data and personnel and vehicle data from fire scene videos, the efficiency and accuracy of fire video analysis are greatly improved. This method uses deep learning algorithms and feature detection technology to quickly and accurately identify key fire features (such as flames, smoke, and light and shadow changes) and specific information about personnel and vehicles in the video, reducing the workload and subjective judgment errors of manual video viewing and analysis. In addition, the use of multi-frame analysis technology improves the robustness of recognition, effectively avoiding single-frame misjudgment and missed detection, and provides a high-quality data foundation for subsequent fire development assessment, personnel identity confirmation, and vehicle tracking. The automation and intelligent characteristics of this method make it more suitable for dealing with complex and changeable fire scene situations, supporting rapid and accurate emergency response and accident investigation.

[0066] S2: Classify and identify the fire characteristic data to obtain flame information, smoke information, and light and shadow information;

[0067] It should be noted that flame information refers to the characteristic data related to flames in the video, including the color, shape, brightness, area, position and changing trend of flames, etc., which is mainly used to determine the location of the fire source and the spread of the fire. Smoke information refers to the characteristic data related to smoke in the video, including the color, density, diffusion speed, direction, area coverage, etc. of smoke, which is used to evaluate the impact and diffusion path of smoke generated by the fire. Light and shadow information refers to the abnormal change characteristics of light and images in the video caused by factors such as flames and smoke, including the flickering of light sources, changes in light and shadow reflections, etc., which can help determine the starting time of the fire and the state of the fire under specific conditions.

[0068] Specifically, in the process of classifying and identifying fire feature data, first, the original data related to flames, smoke, and light and shadow are obtained from the extracted fire scene video data, and these data are input into the trained deep learning model. For the identification of flame information, the model uses the color, shape, brightness and other characteristics of the flame for classification and identification, and identifies the accurate location, area and change trend of the flame; for the identification of smoke information, the model locates and identifies the specific location, range and concentration change of the smoke by analyzing the grayscale change, texture characteristics and regional diffusion; the identification of light and shadow information detects the light intensity change and reflection characteristics in the video frame to detect the abnormal light and shadow caused by flames and smoke.

[0069] Furthermore, in order to improve the accuracy of classification and recognition, the computer equipment will use time series analysis technology to conduct a comprehensive analysis of the data of multiple video frames to eliminate misjudgment and noise interference in single-frame detection. By tracking the time evolution of flames, smoke, and light and shadow, the system can establish a timeline of dynamic changes at the fire scene, accurately assess the starting location, spread speed, and impact range of the fire, and thus form a complete data set of flame, smoke, and light and shadow information.

[0070] By classifying and identifying fire characteristic data, this step can significantly improve the accuracy and efficiency of fire analysis. The precise extraction and identification of flame, smoke, and light and shadow information enables the system to grasp the dynamic changes of the fire scene in real time, identify the starting location and spread path of the fire, and provide a more accurate basis for fire emergency decision-making. In addition, with the application of deep learning and time series analysis technology, the subjective misjudgment and delay problems in traditional manual analysis are avoided, and the response speed and reliability of fire monitoring are effectively improved. This comprehensive data classification and identification method provides important decision-making support and information basis for rapid emergency response, cause investigation and subsequent processing at the fire scene.

[0071] S3: Performing super-resolution reconstruction and recognition on the personnel behavior image in the personnel and vehicle data to obtain personnel feature information;

[0072] It should be noted that the behavior image of a person refers to the image of the face area identified and extracted from the video. Usually, due to the limitations of video quality or on-site conditions, there may be problems such as blur and low resolution. Super-resolution reconstruction is a technology that uses deep learning or image processing technology to convert low-resolution images into high-resolution images. By filling in detailed information, the image is made clearer and more accurate, which facilitates further identification and analysis. Personnel feature information refers to the high-definition face data obtained by super-resolution reconstruction of face images, which is a set of feature information used for personnel identification and confirmation.

[0073] Specifically, the extracted low-resolution human behavior images are input into the pre-trained super-resolution reconstruction model. The model processes the input face images through a series of convolutional neural network layers, extracts their features, and gradually restores and reconstructs higher-resolution images using the high-resolution image feature library learned in the model. The model compares the differences between the original low-resolution image and the generated high-resolution image, performs iterative optimization, and generates clearer and more detailed face images.

[0074] Furthermore, the reconstructed high-resolution human behavior images are analyzed using feature extraction algorithms to extract key facial feature points (such as the position and contour of the eyes, nose, and mouth) and generate facial feature vectors. These feature vectors can be used for subsequent face recognition and comparison to help identify and confirm the identities of relevant personnel at the fire scene.

[0075] By reconstructing facial images with super-resolution, the clarity and recognizability of facial images in low-quality videos are significantly improved, solving the problem of difficult facial recognition caused by low video resolution, insufficient light or blurred images. This technology enables accurate identification and confirmation of people's identities in complex environments such as fires, providing more reliable data support for personnel screening and subsequent investigations at the fire scene. In addition, the super-resolution reconstruction method can quickly process a large number of low-resolution images, reducing the time of the recognition process and improving the efficiency and accuracy of fire emergency response.

[0076] S4: performing de-motion blurring processing on the vehicle image in the personnel vehicle data to obtain vehicle feature information;

[0077] It should be noted that personnel and vehicle data refers to visual information data including facial images and vehicle images extracted from the fire scene video, which is used to identify and confirm the information of relevant personnel and vehicles at the scene. Vehicle images refer to vehicle license plate area images identified and extracted from the video. These images may be blurred or difficult to identify due to the rapid movement of the vehicle or the shaking of the camera equipment. De-motion blurring is an image restoration technology used to eliminate image blur caused by camera shaking or object movement, restore detailed information in the image, and make the blurred image clear and recognizable. Vehicle feature information refers to the clear license plate data obtained by de-motion blurring the vehicle image, including the license plate number, characters and related vehicle information, which is used for vehicle identification and tracking.

[0078] Specifically, the extracted blurred vehicle image is input into a de-motion blurring processing model, which uses deep learning algorithms or traditional algorithms based on image restoration to detect and estimate the motion blur features in the vehicle image. By analyzing the blur kernel of the image (such as the direction and speed of motion), the model can generate a deconvolution or inverse filter to remove the blur effect. This process uses multiple convolutional layers to gradually restore the details in the vehicle image, optimize the clarity of each pixel, and ultimately generate a higher resolution vehicle image.

[0079] Furthermore, optical character recognition (OCR) technology is applied to perform character recognition on the deblurred vehicle image. The OCR algorithm accurately converts the letters and numbers in the license plate into text information based on the clear license plate character shape. At the same time, combined with the format rules and regional features of the license plate, the correctness of the recognition result is further verified to ensure the accurate extraction of vehicle feature information.

[0080] By de-motion blurring the vehicle image, the image blur caused by the rapid movement of the vehicle or the shaking of the camera equipment is effectively eliminated, making the license plate characters clearly identifiable. This processing method significantly improves the accuracy and reliability of license plate recognition, especially in low-light, complex background or fast-moving scenes, and can ensure accurate vehicle feature information. The application of de-motion blurring technology solves the problems of recognition errors and omissions caused by image blur in traditional license plate recognition, and provides more reliable evidence support for vehicle tracking, responsibility division and accident investigation at the fire scene. At the same time, the technology can process a large number of images in a short time, improve recognition efficiency, and facilitate rapid response and decision-making.

[0081] S5: Generate a fire accident video analysis report based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information.

[0082] Specifically, a dynamic change model of the fire accident scene is established by combining flame information, smoke information and light and shadow information. The starting point and spread path of the fire are determined by analyzing the color, area, position and change trend of the flame; the spread of the fire and its impact on the surrounding environment are evaluated by combining the diffusion speed, concentration and coverage of the smoke; the time and development process of the fire are further accurately located by using light and shadow change information (such as light source flickering and shadow movement), forming a comprehensive analysis of the fire scene.

[0083] Furthermore, the personnel and vehicle characteristic information is used to identify and track the personnel and vehicles at the fire scene. The extracted high-resolution personnel characteristic information is matched with the personnel information in the known database to determine the identity and behavior pattern of the relevant personnel at the fire scene; at the same time, the vehicle characteristic information is identified and analyzed to track the vehicles at the scene and their activity paths. All information is integrated and associated to generate a fire accident video analysis report that includes the cause of the fire, the spread process, and the activities of personnel and vehicles.

[0084] Through comprehensive analysis based on flame information, smoke information, light and shadow information, personnel characteristics information and vehicle characteristics information, the generated fire accident video analysis report can comprehensively and accurately reflect the occurrence, development and impact of the fire accident. The report can not only provide the starting point of the fire, the spread path and the possible cause of the fire, but also identify the relevant personnel and vehicle information on the scene, providing strong evidence support for the determination of accident responsibility and subsequent investigation. In addition, by automatically generating analysis reports, the efficiency and accuracy of fire accident analysis are improved, the workload and subjective errors of manual participation are reduced, and the emergency response speed and scientific decision-making are significantly improved.

[0085] This embodiment extracts fire feature data and personnel and vehicle data from the fire scene video data; then classifies and identifies the fire feature data to obtain flame information, smoke information, and light and shadow information; super-resolution reconstruction and identification of personnel behavior images in the personnel and vehicle data to obtain personnel feature information; de-motion blurring of vehicle images in the personnel and vehicle data to obtain vehicle feature information; and finally generates a fire accident video analysis report based on flame information, smoke information, light and shadow information, personnel feature information, and vehicle feature information. This embodiment integrates multiple video analysis technologies to automatically extract fire feature data, personnel feature information, and vehicle feature information from the fire scene video data, and classifies, identifies, super-resolution reconstructs, and de-motion blurring these information, effectively improving the efficiency and accuracy of data extraction and identification. The classification and identification of fire feature data can accurately assess the starting position and spread direction of the fire; super-resolution reconstruction improves the accuracy of face recognition in complex environments; and license plate de-motion blurring enhances the recognition ability of fast-moving or blurred license plates. Ultimately, the fire accident video analysis report generated based on the above information can reflect the actual situation at the fire scene, provide strong decision-making support for fire emergency response, cause investigation and responsibility determination, thereby improving the effect of fire video analysis.

[0086] Based on the above first embodiment, a second embodiment of the fire accident video analysis method of the present application is proposed. Figure 2 , Figure 2 This is a schematic diagram of a sub-process in the second embodiment of the fire accident video analysis method of the present application.

[0087] like Figure 2 As shown, in this embodiment, before step S1, the following steps are also included:

[0088] S1a: Identify each video input channel based on the topology of the local storage of on-site monitoring;

[0089] S1b: determining a target video input channel according to the device type, resolution, frame rate and video encoding format corresponding to each of the video input channels;

[0090] S1c: acquiring a target surveillance video based on the target video input channel;

[0091] S1d: Optimize the target monitoring video to obtain the fire scene video data.

[0092] It should be noted that the topology of the monitoring network refers to the layout and connection relationship of each video monitoring device at the fire scene, including the physical location of the device, the network connection method, and the data transmission path, which is used to effectively manage and schedule video resources. The video input channel refers to the communication path used to transmit monitoring video data. Each channel corresponds to a specific monitoring device or camera, which can independently collect, transmit and store monitoring videos. The device type, resolution, frame rate, and video encoding format refer to the specific type of each monitoring device (such as fixed camera, mobile camera, etc.), the clarity of the video image (such as 1080p, 4K), the number of frames displayed per second (such as 30fps, 60fps), and the compression encoding format of the video data (such as H.264, H.265), etc. These parameters determine the quality and transmission performance of the video data. The target video input channel refers to the monitoring video channel that meets the predetermined conditions (such as video quality and device stability) after screening and optimization. These channels are selected to obtain the most useful video data. The target monitoring video refers to the fire scene video data collected through the target video input channel, which is used for further analysis after processing and optimization. Optimization processing refers to a series of enhancement and adjustment operations on the collected video data, such as denoising, contrast adjustment, deblurring, etc., to improve video quality and analysis effect.

[0093] Specifically, according to the topological structure of the fire scene monitoring network, identify each video input channel, clarify the monitoring equipment connected to each channel, its physical location and network connection status. Combined with the monitoring layout of the site, extract key information such as the device type (such as fixed or mobile camera), resolution, frame rate and video encoding format corresponding to each video input channel to fully understand the performance characteristics and video data quality of each channel. Based on the above identified information, perform channel screening and priority sorting to determine the target video input channel. The screening process mainly considers the clarity (high resolution), smoothness (high frame rate) and compression efficiency (appropriate encoding format) of the video, as well as the reliability of the equipment (such as low failure rate and good network connection). After selecting the target video input channel, obtain the target monitoring video data from it and store the data in the designated storage system.

[0094] Furthermore, after acquiring the target surveillance videos, these videos are optimized to improve their effectiveness in fire analysis. The optimization process includes denoising the video to remove random noise in the picture; adjusting the contrast and brightness of the video to improve the clarity and recognizability of the picture; using a deblurring algorithm to process blurred videos caused by equipment shaking or insufficient light to restore image details. The fire scene video data obtained after optimization has higher quality and reliability, and is suitable for subsequent fire feature extraction and analysis.

[0095] By identifying the video input channel based on the topology of the on-site monitoring network and selecting the best target video input channel based on parameters such as device type, resolution, frame rate and encoding format, it helps to select the best quality and most reliable video source and ensure high-quality acquisition of video data. The optimization processing of the target monitoring video further improves the clarity and availability of the video, enhances the accuracy and effectiveness of the video data at the fire scene, and provides more accurate and efficient basic data support for subsequent fire feature analysis, personnel identification and accident investigation. This optimization method significantly improves the overall effect of fire video analysis and the speed of emergency response.

[0096] In this embodiment, step S1d includes:

[0097] S1d1: Convert the format of the target surveillance video into a preset standard format;

[0098] S1d2: Based on a preset data recovery method, the lost frames of the target surveillance video after format conversion are recovered to obtain multiple restored surveillance videos;

[0099] S1d3: performing image enhancement processing on the multiple restored surveillance videos to obtain multiple enhanced videos;

[0100] S1d4: performing time stamp synchronization on the multiple enhanced videos;

[0101] S1d5: Perform integrity check on the enhanced video after time stamp synchronization, and use the video that passes the integrity check as the fire scene video data.

[0102] It should be noted that the target surveillance video refers to the surveillance video data that has been screened and optimized and further processed for fire scene analysis. The preset standard format refers to the unified video format (such as MP4, AVI, etc.) pre-set in the fire video analysis system to ensure that all video data can be processed and analyzed under the same standard. The data recovery method refers to the method used to recover the video data lost due to transmission interruption, equipment failure, etc., including interpolation algorithms, machine learning models, etc., for reconstructing and recovering lost image frames. The restored surveillance video refers to the video data of the lost frames supplemented after data recovery processing to restore the integrity of the video. Image enhancement processing refers to the optimization processing of video images, such as denoising, contrast enhancement, edge sharpening, etc., to improve the clarity and recognizability of the video. Timestamp synchronization refers to the calibration of time stamps of multiple video data to ensure that videos from different channels are accurately aligned on the time axis for subsequent analysis and comparison. Integrity verification refers to the verification of processed video data to ensure the integrity and consistency of the data and exclude data loss, damage or errors.

[0103] Specifically, the format of the target surveillance video is converted into a preset standard format so that it can be uniformly processed and analyzed in the fire video analysis system. During the format conversion process, the video data is re-encoded and compressed according to the requirements of the preset standard format (such as resolution, encoding format, frame rate, etc.) to ensure that the video data is compatible with the system's analysis software and algorithms. For the video data after format conversion, the preset data recovery method is used to recover the lost frames. During the data recovery process, an interpolation algorithm or a machine learning model is used to estimate and supplement the frames lost due to transmission interruption, equipment failure, etc., to generate multiple restored surveillance videos. Subsequently, the restored video data is subjected to image enhancement processing, and the video quality is improved through denoising, contrast adjustment, edge sharpening and other technologies, and the clarity and detail restoration are improved to obtain multiple enhanced videos.

[0104] Furthermore, the enhanced video is time-stamped and synchronized to ensure that multiple video data from different channels are accurately aligned on the time axis. The time-stamp synchronization process includes adjusting and matching the start time and inter-frame time of the video to ensure that all videos are synchronously analyzed on the same time basis. After synchronization, the enhanced video after time-stamp calibration is integrity checked to ensure that the video data is not lost, damaged or tampered with by comparing the hash value or checksum of the check data. Only videos that pass the integrity check are deemed qualified and used as the final fire scene video data for subsequent analysis.

[0105] By converting the target surveillance video format into a preset standard format, the compatibility and uniformity of the video data are guaranteed, making the subsequent processing and analysis process smoother and more efficient. The use of data recovery methods to recover lost frames effectively solves the problem of video data loss caused by transmission interruption or equipment failure, ensuring the integrity of the video. Image enhancement processing improves the clarity and recognizability of the video, providing high-quality basic data for the accurate extraction of fire features. Timestamp synchronization and integrity verification further ensure the precise alignment of multi-channel video data on the timeline and the integrity and consistency of the data, providing reliable data guarantee for the accurate analysis and judgment of fire accidents. Overall, this series of steps significantly improves the accuracy, reliability and efficiency of fire video analysis.

[0106] This embodiment extracts fire feature data and personnel and vehicle data from the fire scene video data; then classifies and identifies the fire feature data to obtain flame information, smoke information, and light and shadow information; super-resolution reconstruction and identification of personnel behavior images in the personnel and vehicle data to obtain personnel feature information; de-motion blurring of vehicle images in the personnel and vehicle data to obtain vehicle feature information; and finally generates a fire accident video analysis report based on flame information, smoke information, light and shadow information, personnel feature information, and vehicle feature information. This embodiment integrates multiple video analysis technologies to automatically extract fire feature data, personnel feature information, and vehicle feature information from the fire scene video data, and classifies, identifies, super-resolution reconstructs, and de-motion blurring these information, effectively improving the efficiency and accuracy of data extraction and identification. The classification and identification of fire feature data can accurately assess the starting position and spread direction of the fire; super-resolution reconstruction improves the accuracy of face recognition in complex environments; and license plate de-motion blurring enhances the recognition ability of fast-moving or blurred license plates. Ultimately, the fire accident video analysis report generated based on the above information can reflect the actual situation at the fire scene, provide strong decision-making support for fire emergency response, cause investigation and responsibility determination, thereby improving the effect of fire video analysis.

[0107] Based on the above second embodiment, a third embodiment of the fire accident video analysis method of the present application is proposed. Figure 3 , Figure 3 This is a schematic diagram of a sub-process in the third embodiment of the fire accident video analysis method of the present application.

[0108] In this embodiment, step S1 includes:

[0109] S11: preprocessing the fire scene video to obtain a target frame set, wherein the preprocessing includes one or more of video frame decomposition, image enhancement, and region of interest detection;

[0110] S12: Based on a preset fire feature detection model, flame detection, smoke detection, and light and shadow detection are performed on each frame in the target frame set to obtain the fire feature data;

[0111] S13: Based on a preset person and vehicle detection model, perform person detection and vehicle detection on each frame in the target frame set to obtain the person and vehicle data.

[0112] It should be noted that the fire scene video refers to the original video data captured by the on-site monitoring equipment when the fire occurs, which contains the fire development process and the information of related personnel and vehicles. Preprocessing refers to a series of preliminary processing operations performed on the fire scene video before further analysis, including video frame decomposition, image enhancement, and region of interest detection, etc., to improve the accuracy and efficiency of subsequent detection. The target frame set refers to the set of video frames after preprocessing, which contains all the screened and enhanced image frames for fire feature detection and face and license plate detection. Video frame decomposition refers to decomposing the video stream into a series of independent static image frames for frame-by-frame analysis and processing. Image enhancement refers to improving the quality and clarity of video frames through image processing technology, such as denoising, contrast adjustment, brightness optimization, etc. Region of interest detection refers to identifying and marking areas in the image that may contain fire features (such as flames, smoke) or important information such as faces and license plates, so as to narrow the scope of analysis and improve detection efficiency. The fire feature detection model refers to a deep learning model specially trained to identify fire scene features (such as flames, smoke, light and shadow changes, etc.). The face and license plate detection model refers to a deep learning model specially trained to recognize faces and license plates, which can quickly and accurately locate and recognize facial and vehicle feature information in video frames.

[0113] Specifically, the fire scene video is preprocessed to generate a target frame set. The preprocessing steps include video frame decomposition, which decomposes the continuous video stream into a series of static image frames for frame-by-frame analysis; image enhancement processing is performed on each decomposed frame to improve the clarity and recognizability of the image through methods such as denoising, contrast adjustment, and brightness optimization; combined with the region of interest detection technology, the areas in the image that may contain important information such as flames, smoke, faces, license plates, etc. are identified and marked, thereby narrowing the analysis scope, reducing the amount of calculation and the error rate. After these preprocessing steps, a quality-optimized target frame set is obtained, which is convenient for subsequent feature detection and analysis.

[0114] Furthermore, based on the preset fire feature detection model, flame detection, smoke detection and light and shadow detection are performed on each frame in the target frame set. The fire feature detection model uses a deep learning algorithm to automatically analyze the color, brightness, shape, texture and dynamic change information in each frame of the image to identify and classify the flame, smoke and light and shadow change characteristics of the fire scene, and generate detailed fire feature data. At the same time, based on the preset face and license plate detection model, face and license plate detection are performed on each frame in the target frame set. The model uses advanced algorithms such as convolutional neural networks to quickly locate and identify the face and license plate areas in the image, and extracts relevant data to obtain personnel and vehicle data containing personnel and vehicle information.

[0115] By preprocessing the fire scene video (such as video frame decomposition, image enhancement, and region of interest detection), the clarity and analyzability of the video data are effectively improved, the interference of noise and irrelevant information is reduced, and the subsequent fire feature and face and license plate detection are more accurate and efficient. The preset fire feature detection model is used for flame, smoke, and light and shadow detection, which can quickly identify the main characteristics of the fire and provide an important basis for the cause analysis and development prediction of the fire. At the same time, the face and license plate detection model is used to accurately identify the personnel and vehicle information at the fire scene, ensuring comprehensive monitoring and rapid response to the on-site conditions. Overall, this multi-level, multi-step detection method significantly improves the accuracy and efficiency of fire accident analysis and provides reliable data support for emergency response and accident investigation.

[0116] Based on the above second embodiment, in this embodiment, step S3 includes:

[0117] S31: extracting a person behavior image from the person vehicle data, and performing quality evaluation on the extracted person behavior image;

[0118] S32: eliminating the face images corresponding to the quality evaluation values ​​lower than the preset face image quality standard evaluation values, and obtaining quality face images;

[0119] S33: Standardizing the size, color space, brightness and contrast of the quality facial image to obtain a standard quality facial image;

[0120] S34: reconstructing and enhancing the standard quality face image based on a preset super-resolution reconstruction model to obtain a reconstructed face image;

[0121] S35: Based on a preset face recognition algorithm, extract features from the reconstructed face image to obtain the person feature information.

[0122] It should be noted that personnel and vehicle data refers to visual information data containing facial images and vehicle images extracted from the fire scene video, which is used to identify and confirm the information of relevant personnel and vehicles at the scene. Facial images refer to images of the facial area identified and extracted from the video, which are used for further analysis and identification. Quality evaluation refers to the quality assessment of the extracted facial images, which is to judge the quality of the images by measuring parameters such as image clarity, contrast, and noise level to ensure that the quality of facial images used for subsequent processing and identification meets the standards. Standardization processing refers to the unified processing of the size, color space, brightness, and contrast of facial images so that the images meet the preset analysis standards. Super-resolution reconstruction model refers to a model that uses deep learning or image processing technology to convert low-resolution images into high-resolution images, making the images easier to identify and analyze by enhancing details and clarity. Face recognition algorithm refers to an algorithm used to extract facial feature vectors from images. By comparing these feature vectors, the identity of the person in the image can be identified and confirmed.

[0123] Specifically, all face images are extracted from the personnel and vehicle data, and the quality of these images is evaluated. The quality evaluation process measures the image clarity, contrast, noise level, illumination uniformity and other parameters, scores or rates each face image, and removes low-quality images whose quality evaluation values ​​are lower than the preset face image quality standard evaluation value, to obtain a set of images that meet the quality standards, namely, quality face images.

[0124] Furthermore, these quality facial images are standardized, and their size, color space, brightness and contrast are adjusted to a unified standard to ensure that all images are input into the subsequent analysis model under the same conditions. Next, the standard quality facial images are reconstructed and enhanced using a preset super-resolution reconstruction model, and the image resolution and details are gradually enlarged through a deep learning model to make the facial images clearer and easier to recognize. Finally, based on the preset face recognition algorithm, feature extraction is performed on the reconstructed and enhanced high-resolution facial images to generate personal feature information for personal identification.

[0125] By evaluating and screening the quality of the extracted facial images, the high quality of the facial data used for subsequent analysis is ensured, thereby reducing recognition errors caused by poor image quality. Standardization processing ensures that all images are analyzed under uniform conditions, avoiding model instability caused by large differences in image features. Using a super-resolution reconstruction model to enhance facial images can convert low-resolution or blurred facial images into high-resolution images, improving the accuracy and robustness of face recognition. Ultimately, the high-quality facial feature information extracted by the face recognition algorithm can more reliably confirm and track the identity of people, effectively improving the accuracy and efficiency of identifying people at the fire scene, and providing strong data support for fire accident investigations and responsibility determination.

[0126] Based on the above second embodiment, in this embodiment, step S4 includes:

[0127] S41: extracting the vehicle image from the personnel vehicle data, and performing the quality evaluation on the extracted vehicle image;

[0128] S42: eliminating vehicle images corresponding to quality evaluation values ​​lower than a preset vehicle image quality standard evaluation value, to obtain quality vehicle images;

[0129] S43: performing the standardization process on the size, color space, brightness and contrast of the quality vehicle image to obtain a standard quality vehicle image;

[0130] S44: dividing the standard quality vehicle image into a plurality of license plate sub-images, and determining fuzzy information of each license plate sub-image;

[0131] S45: Based on the blur information, input each of the license plate sub-images into a preset deblurring model to obtain the vehicle feature information.

[0132] It should be noted that the vehicle image refers to the license plate area image identified and extracted from the video, which is used for further license plate recognition and analysis. Quality evaluation refers to the quality assessment of the extracted vehicle image, which determines the quality of the image by measuring parameters such as image clarity, contrast, and noise level to ensure that the quality of the vehicle image used for subsequent processing and recognition meets the standards. Standardization processing refers to the unified processing of the size, color space, brightness, and contrast of the vehicle image so that the image meets the preset analysis standards. License plate sub-image refers to the further division of the standardized vehicle image into multiple smaller image blocks, which facilitates the analysis and processing of the blur degree of each local area. Blur information refers to the blurred feature information in the license plate sub-image, including the degree, direction, and range of blur, reflecting the state of image clarity. Deblur model refers to a deep learning model or image processing algorithm used to eliminate image blur, which improves the clarity and recognizability of the image by restoring image details. Vehicle feature information refers to the clear license plate data containing the license plate number, characters, and related vehicle information after deblurring.

[0133] Specifically, vehicle images are extracted from the personnel vehicle data, and the quality of each extracted vehicle image is evaluated. The quality evaluation process determines the quality of the vehicle image by analyzing parameters such as image clarity, contrast, and noise level; images with quality evaluation values ​​lower than the preset vehicle image quality standard evaluation value are eliminated to ensure that only high-quality images are used for subsequent processing, thereby obtaining quality vehicle images.

[0134] Furthermore, the quality vehicle image is standardized, including adjusting the image size, color space, brightness and contrast to meet the preset analysis standards, and generating a standard quality vehicle image. Then, the standard quality vehicle image is further segmented into multiple license plate sub-images (i.e., smaller areas of the vehicle image), and the blur information (such as blur degree and direction) of each license plate sub-image is calculated and determined. Based on this blur information, the license plate sub-image is input into a preset deblurring model, and the model removes blur through algorithms such as inverse convolution and iterative optimization, restores the details of the vehicle image, and finally obtains clear and recognizable vehicle feature information.

[0135] By evaluating and screening the quality of the extracted vehicle images, the vehicle images used for subsequent deblurring processing are ensured to have high quality, reducing the interference of low-quality images on license plate recognition. Standardization processing further unifies the features of the images and avoids recognition errors caused by large image differences. Segmenting the vehicle image into multiple sub-images and deblurring based on the fuzzy information helps to restore the image details more accurately, especially under complex backgrounds or lighting conditions, which can effectively improve the accuracy of license plate recognition. The clear vehicle feature information finally obtained provides reliable data support for vehicle tracking and responsibility identification at the fire scene, which helps to quickly and accurately analyze the situation at the fire scene and the relevant responsible persons.

[0136] This embodiment extracts fire feature data and personnel and vehicle data from the fire scene video data; then classifies and identifies the fire feature data to obtain flame information, smoke information, and light and shadow information; super-resolution reconstruction and identification of personnel behavior images in the personnel and vehicle data to obtain personnel feature information; de-motion blurring of vehicle images in the personnel and vehicle data to obtain vehicle feature information; and finally generates a fire accident video analysis report based on flame information, smoke information, light and shadow information, personnel feature information, and vehicle feature information. This embodiment integrates multiple video analysis technologies to automatically extract fire feature data, personnel feature information, and vehicle feature information from the fire scene video data, and classifies, identifies, super-resolution reconstructs, and de-motion blurring these information, effectively improving the efficiency and accuracy of data extraction and identification. The classification and identification of fire feature data can accurately assess the starting position and spread direction of the fire; super-resolution reconstruction improves the accuracy of face recognition in complex environments; and license plate de-motion blurring enhances the recognition ability of fast-moving or blurred license plates. Ultimately, the fire accident video analysis report generated based on the above information can reflect the actual situation at the fire scene, provide strong decision-making support for fire emergency response, cause investigation and responsibility determination, thereby improving the effect of fire video analysis.

[0137] The present application also provides a fire accident video analysis device, please refer to Figure 4 , Figure 4 This is a schematic diagram of the module structure of a fire accident video analysis device according to an embodiment of the present application, wherein the fire accident video analysis device comprises:

[0138] A data extraction module 401 is used to extract fire feature data and personnel and vehicle data from the fire scene video data;

[0139] An information identification module 402 is used to classify and identify the fire characteristic data to obtain flame information, smoke information, and light and shadow information;

[0140] A personnel identification module 403 is used to perform super-resolution reconstruction on the personnel behavior image in the personnel vehicle data to obtain personnel feature information;

[0141] The vehicle identification module 404 is used to perform a motion blur removal process on the vehicle image in the personnel vehicle data to obtain vehicle feature information;

[0142] The target module 405 is used to generate a fire accident video analysis report based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information.

[0143] The fire accident video analysis device provided in the embodiment of the present application adopts the fire accident video analysis method in the above embodiment, which can solve the technical problem of how to improve the fire video analysis effect. Compared with the prior art, the beneficial effects of the fire accident video analysis device provided in the embodiment of the present application are the same as the beneficial effects of the fire accident video analysis method provided in the above embodiment, and the other technical features in the fire accident video analysis device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0144] The present application provides a fire accident video analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the fire accident video analysis method in the above-mentioned embodiment.

[0145] Reference below Figure 5 , which shows a schematic diagram of the structure of a fire accident video analysis device suitable for implementing the embodiment of the present application. The fire accident video analysis device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The fire accident video analysis device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0146] like Figure 5As shown, the fire accident video analysis device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the fire accident video analysis device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the fire accident video analysis device to communicate with other devices wirelessly or by wire to exchange data. Although the fire accident video analysis device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0147] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0148] The fire accident video analysis device provided by the present application adopts the fire accident video analysis method in the above embodiment, which can solve the technical problem of how to improve the fire video analysis effect. Compared with the prior art, the beneficial effects of the fire accident video analysis device provided by the present application are the same as the beneficial effects of the fire accident video analysis method provided by the above embodiment, and the other technical features in the fire accident video analysis device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0149] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0150] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0151] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the fire accident video analysis method in the above-mentioned embodiment.

[0152] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0153] The computer-readable storage medium may be included in the fire accident video analysis device; or may exist independently without being assembled into the fire accident video analysis device.

[0154] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the fire accident video analysis device, the fire accident video analysis device: extracts the fire feature data and personnel vehicle data in the fire scene video data; classifies and identifies the fire feature data to obtain flame information, smoke information and light and shadow information; performs super-resolution reconstruction and identification on the personnel behavior image in the personnel vehicle data to obtain personnel feature information; performs de-motion blurring on the vehicle image in the personnel vehicle data to obtain vehicle feature information; generates a fire accident video analysis report based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information. The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0155] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0156] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0157] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned fire accident video analysis method, and can solve the technical problem of how to improve the fire video analysis effect. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the fire accident video analysis method provided in the above-mentioned embodiment, and will not be repeated here.

[0158] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the fire accident video analysis method as described above when executed by a processor.

[0159] The computer program product provided in this application can solve the technical problem of how to improve the fire video analysis effect. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the fire accident video analysis method provided in the above embodiment, which will not be repeated here.

[0160] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A fire accident video analysis method, characterized in that: The method comprises: Extracting fire feature data and personnel and vehicle data from the fire scene video data; Classifying and identifying the fire characteristic data to obtain flame information, smoke information, and light and shadow information; Performing super-resolution reconstruction and recognition on the personnel behavior image in the personnel vehicle data to obtain personnel feature information; Performing a motion blur removal process on the vehicle image in the personnel vehicle data to obtain vehicle feature information; A fire accident video analysis report is generated based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information.

2. The method according to claim 1, characterized in that Before the step of extracting the fire feature data and the personnel and vehicle data from the fire scene video data, the method further includes: Based on the topology of the local storage of on-site monitoring, identify each video input channel; Determine the target video input channel according to the device type, resolution, frame rate and video encoding format corresponding to each of the video input channels; Based on the target video input channel, obtaining the target monitoring video; The target monitoring video is optimized to obtain the fire scene video data.

3. The method according to claim 2, characterized in that The step of optimizing the target monitoring video to obtain the fire scene video data comprises: Converting the target surveillance video into a preset standard format; Based on a preset data recovery method, the lost frames of the target surveillance video after format conversion are recovered to obtain multiple restored surveillance videos; Performing image enhancement processing on the multiple restored surveillance videos to obtain multiple enhanced videos; Performing time stamp synchronization on the multiple enhanced videos; An integrity check is performed on the enhanced video after time stamp synchronization, and the video that passes the integrity check is used as the fire scene video data.

4. The method according to claim 1, characterized in that The step of extracting fire feature data and personnel and vehicle data from the fire scene video data comprises: Preprocessing the fire scene video to obtain a target frame set, wherein the preprocessing includes one or more of video frame decomposition, image enhancement, and region of interest detection; Based on a preset fire feature detection model, flame detection, smoke detection, and light and shadow detection are performed on each frame in the target frame set to obtain the fire feature data; Based on a preset personnel and vehicle detection model, personnel detection and vehicle detection are performed on each frame in the target frame set to obtain the personnel and vehicle data.

5. The method according to claim 1, characterized in that The step of performing super-resolution reconstruction and recognition on the personnel behavior image in the personnel vehicle data to obtain personnel feature information includes: Extracting a personnel behavior image from the personnel vehicle data, and performing quality evaluation on the extracted personnel behavior image; Eliminate the facial images whose quality evaluation values ​​are lower than the preset facial image quality standard evaluation values ​​to obtain quality facial images; Standardizing the size, color space, brightness and contrast of the quality facial image to obtain a standard quality facial image; Based on a preset super-resolution reconstruction model, reconstructing and enhancing the standard quality face image to obtain a reconstructed face image; Based on a preset face recognition algorithm, feature extraction is performed on the reconstructed face image to obtain the person feature information.

6. The method according to claim 5, characterized in that The step of performing de-motion blurring processing on the vehicle image in the personnel vehicle data to obtain vehicle feature information comprises: Extracting the vehicle image from the personnel vehicle data, and performing the quality evaluation on the extracted vehicle image; Eliminate the vehicle images whose quality evaluation values ​​are lower than the preset vehicle image quality standard evaluation values ​​to obtain quality vehicle images; Performing the standardization process on the size, color space, brightness and contrast of the quality vehicle image to obtain a standard quality vehicle image; dividing the standard quality vehicle image into a plurality of license plate sub-images, and determining fuzzy information of each license plate sub-image; Based on the fuzzy information, each of the license plate sub-images is input into a preset deblurring model to obtain the vehicle feature information.

7. A fire accident video analysis device, characterized in that: The device comprises: A data extraction module, used to extract fire feature data and personnel and vehicle data from the fire scene video data; An information recognition module is used to classify and recognize the fire characteristic data to obtain flame information, smoke information, and light and shadow information; A personnel identification module is used to perform super-resolution reconstruction and identification on the personnel behavior images in the personnel vehicle data to obtain personnel feature information; A vehicle identification module, used for performing a motion blur removal process on the vehicle image in the personnel vehicle data to obtain vehicle feature information; The target module is used to generate a fire accident video analysis report based on the flame information, the smoke information, the light and shadow information, the personnel feature information and the vehicle feature information.

8. A computer device, characterized in that: The device comprises: a memory, a processor, and a fire accident video analysis program stored in the memory and executable on the processor, wherein the fire accident video analysis program is configured to implement the steps of the fire accident video analysis method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a fire accident video analysis program, which, when executed by a processor, implements the steps of the fire accident video analysis method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the fire accident video analysis method according to any one of claims 1 to 6 are implemented.

Images