Early warning information generation method and device based on fire-fighting video

Abstract

The invention relates to an early warning information generation method and device based on a fire-fighting video, and belongs to the technical field of tunnel detection.The method comprises the steps that the tunnel video collected by video detection equipment in a tunnel is obtained, and the tunnel video is marked and extracted to obtain the fire-fighting video; performing image extraction on the fire-fighting video according to a preset frame rate to obtain a target image; when a traffic accident is detected in the target image, accident detection is carried out on the target image, and an accident type is determined; determining an accident position according to the accident type, the video detection device and the target image, and generating early warning information according to the accident position; according to the invention, when a traffic accident occurs, the accident type and the accident position can be obtained by carrying out image extraction and detection on the fire-fighting video, so that early warning information can be generated to carry out early warning on other vehicles in the tunnel, and potential safety hazards in the tunnel are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel detection, and in particular to a method and device for generating early warning information based on firefighting videos. Background Art

[0002] With the growth of railway construction in my country, the proportion of tunnels in railway construction is also increasing. With the growth of demand, tunnels are getting longer and longer. The tunnel space is small. Once an accident occurs, it is difficult to rescue. Especially when a vehicle catches fire, a lot of smoke and toxic gases will be generated, which brings huge safety hazards. In order to deal with the tunnel, various equipment can be set up in the tunnel, such as fiber grating, air purification equipment, fire extinguishing equipment and alarm equipment, etc., to create a smart tunnel. These devices can be used to obtain tunnel data and monitor the tunnel in real time.

[0003] In the prior art, fiber Bragg gratings are used to collect temperature information of vehicles traveling in tunnels, and the temperature information is detected. When the temperature exceeds a certain threshold, an early warning is performed. However, when the temperature exceeds a certain threshold, a fire may not necessarily occur, and car accidents have nothing to do with temperature. Therefore, fiber Bragg gratings cannot accurately detect accidents in tunnels, resulting in huge safety hazards in tunnels.

[0004] Therefore, it is urgent to propose a method and device for generating early warning information based on fire videos to solve the technical problem that the existing technology cannot accurately detect safety accidents in tunnels, resulting in huge safety hazards in tunnels. Summary of the invention

[0005] In view of this, it is necessary to provide a method and device for generating early warning information based on fire videos to solve the technical problem that the existing technology cannot accurately detect safety accidents in tunnels, resulting in huge safety hazards in tunnels.

[0006] In order to solve the above problems, the present invention provides a method for generating early warning information based on fire video, comprising: Obtaining a tunnel video collected by a video detection device in the tunnel, annotating and extracting the tunnel video, and obtaining a firefighting video; Extracting images from the firefighting video according to a preset frame rate to obtain a target image; When a traffic accident is detected in the target image, performing accident detection on the target image to determine the type of the accident; The accident location is determined according to the accident type, the video detection device and the target image, and warning information is generated according to the accident location.

[0007] In a possible implementation, extracting an image from the firefighting video according to a preset frame rate to obtain a target image includes: Extracting images from the firefighting video according to the preset frame rate to obtain an image set; Screening the images in the image set to obtain an initial image; The initial image is preprocessed to obtain a target image.

[0008] In a possible implementation, extracting images from the firefighting video according to the preset frame rate to obtain an image set includes: Extracting images from the firefighting video according to the preset frame rate to obtain continuous image frames; Resizing each image frame in the continuous image frames according to a preset size to obtain all images; Based on all the images, an image set is obtained.

[0009] In a possible implementation, after extracting the image from the firefighting video according to the preset frame rate to obtain the target image, the method further includes: Performing target detection on the target image to obtain a detection result; Determine whether a traffic accident exists based on the detection result.

[0010] In a possible implementation, determining the accident location according to the accident type, the video detection device, and the target image includes: When the accident type is a vehicle accident, determining the location information of the video detection device; The accident location is determined according to the detection result of the target image and the location information.

[0011] In a possible implementation, the determining the accident location according to the accident type, the video detection device, and the target image further includes: When the accident type is a vehicle fire, obtaining smoke detection data of a smoke sensor according to a video detection device; The smoke detection data is calculated and processed according to a preset smoke recognition algorithm to obtain a smoke detection result; The accident location is determined according to the target image and the smoke detection result.

[0012] In a possible implementation, the step of acquiring smoke detection data of a smoke sensor according to a video detection device includes: Determining location information of the video detection device; The smoke detection data of the smoke sensor at the corresponding position is acquired according to the position information.

[0013] In a possible implementation manner, performing target detection on the target image to obtain a detection result includes: Set up a traffic accident detection model; The target image is input into the traffic accident detection model to perform target detection and obtain a detection result.

[0014] In a possible implementation, the step of annotating and extracting the tunnel video to obtain a firefighting video includes: Set up feature annotation datasets for various types of fire accidents in tunnels; Extracting annotations from the tunnel video according to the feature annotation data set to obtain an annotated video; The labeled video is cropped to obtain a firefighting video.

[0015] On the other hand, the present invention also provides a device for generating early warning information based on firefighting video, comprising: A video acquisition module is used to acquire the tunnel video collected by the video detection equipment in the tunnel, annotate and extract the tunnel video, and obtain the firefighting video; An image extraction module is used to extract images from the firefighting video according to a preset frame rate to obtain a target image; An accident determination module, configured to perform accident detection on the target image and determine the type of accident when a traffic accident is detected in the target image; The accident location module is used to determine the accident location according to the accident type, the video detection equipment and the target image, and to generate warning information according to the accident location.

[0016] The beneficial effects of the present invention are: obtaining a tunnel video collected by a video detection device in a tunnel, annotating and extracting the tunnel video to obtain a fire video; performing image extraction on the fire video according to a preset frame rate to obtain a target image; when a traffic accident is detected in the target image, performing accident detection on the target image to determine the type of accident; determining the location of the accident according to the type of accident, the video detection device and the target image, and generating early warning information according to the accident location; when a traffic accident occurs, the present invention can obtain the type and location of the accident by performing image extraction and detection on the fire video, thereby generating early warning information to warn other vehicles in the tunnel, thereby reducing safety hazards in the tunnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic flow chart of an embodiment of a method for generating early warning information based on fire video provided by the present invention; Figure 2 For the present invention Figure 1A schematic flow chart of an embodiment of step S104; Figure 3 For the present invention Figure 1 A schematic flow chart of an embodiment of step S104; Figure 4 A schematic diagram of the structure of an embodiment of a device for generating early warning information based on fire video provided by the present invention; Figure 5 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0019] like Figure 1 As shown, a specific embodiment of the present invention discloses a method for generating early warning information based on fire video, comprising: S101, obtaining a tunnel video collected by a video detection device in the tunnel, and performing annotation extraction on the tunnel video to obtain a firefighting video; S102, extracting images from the firefighting video according to a preset frame rate to obtain a target image; S103, when a traffic accident is detected in the target image, performing accident detection on the target image to determine the type of the accident; S104: Determine the accident location according to the accident type, video detection equipment and target image, and generate warning information according to the accident location.

[0020] In a specific embodiment of the present invention, video detection equipment can be set at intervals in the tunnel, so that the tunnel can be monitored in real time by the video detection equipment, and the tunnel video collected by the video detection equipment can be obtained, so that the tunnel video can be annotated and extracted to obtain a fire video, and then the fire video can be extracted according to the preset frame rate to obtain a target image, wherein the preset frame rate can be set according to actual conditions, and the embodiment of the present invention is not limited here. Then the target image can be detected to determine whether a traffic accident is detected in the target image. If so, the target image can be detected by image recognition technology to determine the type of accident, wherein the accident type can include vehicle fire, vehicle accident, etc. Vehicle accidents are generally divided into rear-end accidents, side collision accidents, frontal collision accidents, etc. Then the accident location where the traffic accident occurs can be determined according to the accident type, the video detection equipment and the target image, and then early warning information can be generated according to the accident location.

[0021] Compared with the prior art, the present embodiment provides a method for obtaining a tunnel video collected by a video detection device in a tunnel, annotating and extracting the tunnel video to obtain a fire video; performing image extraction on the fire video according to a preset frame rate to obtain a target image; when a traffic accident is detected in the target image, performing accident detection on the target image to determine the type of accident; determining the location of the accident according to the type of accident, the video detection device and the target image, and generating early warning information according to the location of the accident; when a traffic accident occurs, the present invention can obtain the type and location of the accident by performing image extraction and detection on the fire video, thereby generating early warning information to warn other vehicles in the tunnel, thereby reducing safety hazards in the tunnel.

[0022] In some embodiments of the present invention, step S101 includes: Set up feature annotation datasets for various types of fire accidents in tunnels; Annotation extraction is performed on the tunnel video according to the feature annotation data set to obtain annotated video; The labeled video is cropped to obtain the firefighting video.

[0023] In a specific embodiment of the present invention, a feature annotation data set of various types of fire accidents in the tunnel can be set according to big data or the work experience of the staff. The feature annotation data set includes annotation features of various types of fire accidents that may occur in the tunnel. For example, if a traffic accident occurs, there are signs of a traffic accident, such as a change in the motion state of the vehicle, a change in the relative speed between the vehicles, a change in the relative position relationship of the vehicles, etc. Flames can also be identified to identify vehicle fires, and then the tunnel video can be annotated and extracted according to the feature annotation data set to obtain an annotated video; a neural network model can be set, and the neural network model can be trained by the feature annotation data set, and then the tunnel video can be input into the trained neural network model for annotation extraction, so that a labeled video with complete annotation can be obtained, and then the annotated video can be cropped according to the annotated content in the annotated video, and useless videos can be deleted, so that a fire video can be obtained.

[0024] In some embodiments of the present invention, Figure 2 As shown, step S102 includes: S201, extracting images from the firefighting video according to a preset frame rate to obtain an image set; S202, screening the images in the image set to obtain an initial image; S203: pre-process the initial image to obtain a target image.

[0025] In a specific embodiment of the present invention, the firefighting video may be subjected to image extraction according to a preset frame rate to obtain an image set. Specifically, in some embodiments of the present invention, step S201 includes: Extract images from the firefighting video according to a preset frame rate to obtain continuous image frames; Resizing each image frame in the continuous image frames according to a preset size to obtain all images; Based on all the images, an image set is obtained.

[0026] In a specific embodiment of the present invention, a preset frame rate can be set. For example, the preset frame rate is 10 frames per second, and continuous image frames of the fire video can be extracted at the preset frame rate of 10 frames per second. A preset size can also be set to scale each frame in the continuous image frames to a standard size of the preset size, so that all images after size adjustment can be obtained, and then all images can be formed into an image set, and then the images in the image set can be screened, and images in which no abnormalities are detected can be deleted. The images can be screened through a neural network, and the specific screening process and method can be set according to actual conditions, and the embodiment of the present invention is not limited here. Therefore, if an initial image for detecting anomalies can be obtained, the initial image can be multiple, and in the subsequent process, the processing process of each initial image is the same, and the initial image can be preprocessed to obtain a target image, wherein the preprocessing can include denoising, contrast enhancement, etc. The specific preprocessing process can be set according to actual conditions, and the embodiment of the present invention is not limited here.

[0027] In some embodiments of the present invention, after step S102, the method further includes: Perform target detection on the target image and obtain the detection result; Determine whether there is a traffic accident based on the detection results.

[0028] In a specific embodiment of the present invention, after obtaining the target image, target detection can be performed on the target image to obtain the detection result, because if a traffic accident occurs and there are signs of a traffic accident, such as a change in the motion state of the vehicle, a change in the relative speed between the vehicles, a change in the relative position relationship of the vehicles, etc., the flame can be identified by a flame recognition camera, and a neural network can also be provided to identify the vehicle fire, which can be detected by image recognition technology. Therefore, after obtaining the detection result, it is determined whether there is a traffic accident based on the detection result. If not, the next or next video detection device's consumer video detection is performed. If yes, it indicates that a traffic accident has occurred.

[0029] In some embodiments of the present invention, performing target detection on a target image to obtain a detection result includes: Set up a traffic accident detection model; The target image is input into the traffic accident detection model for target detection to obtain the detection result.

[0030] In a specific embodiment of the present invention, a traffic accident detection model can also be set up. The staff can set a training set, a test set, and a verification set based on historical data. The traffic accident detection model is trained and verified by the training set, the test set, and the verification set. Thus, a trained traffic accident detection model can be obtained. Then, the target image can be input into the traffic accident detection model for target detection. The traffic accident detection model can output the detection results.

[0031] Furthermore, when a traffic accident is detected in the target image, the target image can be subjected to accident detection through image recognition technology to determine the type of accident; the accident types may include vehicle fires, vehicle accidents, etc., and vehicle accidents are generally divided into rear-end collisions, side collisions, head-on collisions, etc.

[0032] In some embodiments of the present invention, step S104 includes: When the accident type is a vehicle accident, determining the location information of the video detection device; The accident location is determined based on the detection results and location information of the target image.

[0033] In a specific embodiment of the present invention, when the accident type is a vehicle accident, that is, one of the accidents such as a rear-end collision, a sideswipe accident, or a head-on collision, the position information can be determined based on the installation position of the video detection device, so that the target image can be identified through image recognition technology to determine the position of the traffic accident relative to the image, so that the position information and the position in the identified image can be determined based on the installation position of the video detection device to determine the accident location.

[0034] In some embodiments of the present invention, Figure 3 As shown, step S104 also includes: S301, when the accident type is vehicle fire, obtaining smoke detection data of the smoke sensor according to the video detection device; S302, calculating and processing the smoke detection data according to a preset smoke recognition algorithm to obtain a smoke detection result; S303: Determine the accident location based on the target image and smoke detection results.

[0035] In some embodiments of the present invention, step S301 includes: Determining location information of the video detection device; The smoke detection data of the smoke sensor at the corresponding position is acquired according to the position information.

[0036] In a specific embodiment of the present invention, when the accident type is a vehicle fire, the location information can be determined according to the installation location of the video detection device, so that the smoke detection data of the smoke sensor at the location corresponding to the location information can be obtained, and then the smoke detection data is calculated and processed by a preset smoke recognition algorithm to obtain a smoke detection result. The smoke recognition algorithm can be Faster R-CNN, k-means clustering image segmentation, and YOLOv7 algorithms, which can be specifically set according to actual conditions, and the embodiment of the present invention is not limited here. The target image can also be identified by image recognition technology to determine the position of the traffic accident relative to the image, and then the accident location can be determined according to the smoke detection result and the position of the image.

[0037] Furthermore, after obtaining the accident location, early warning information can be generated according to the accident location. For example, the early warning information of "a vehicle rear-end collision occurred 500 meters from the tunnel entrance" can be sent to the tunnel control terminal to remind the staff. The vehicles in the tunnel can also be warned through the alarm equipment. The early warning methods can include broadcast reminders, SMS notifications, etc.

[0038] In order to better implement the method for generating early warning information based on firefighting video in the embodiment of the present invention, on the basis of the method for generating early warning information based on firefighting video, the embodiment of the present invention also provides a device for generating early warning information based on firefighting video, such as Figure 4 As shown, the warning information generating device 400 based on the fire video includes: The video acquisition module 401 is used to acquire the tunnel video collected by the video detection equipment in the tunnel, and perform annotation extraction on the tunnel video to obtain the fire fighting video; An image extraction module 402 is used to extract images from the firefighting video according to a preset frame rate to obtain a target image; The accident determination module 403 is used to perform accident detection on the target image and determine the type of accident when a traffic accident is detected in the target image; The accident location module 404 is used to determine the accident location according to the accident type, video detection equipment and target image, and generate warning information according to the accident location.

[0039] The fire video-based warning information generating device 400 provided in the above embodiment can implement the technical solution described in the above embodiment of the fire video-based warning information generating method. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the fire video-based warning information generating method, which will not be repeated here.

[0040] like Figure 5As shown, the present invention also provides an electronic device 500. The electronic device 500 includes a processor 501, a memory 502 and a display 503. Figure 5 Only some components of the electronic device 500 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0041] In some embodiments, the memory 502 may be an internal storage unit of the electronic device 500, such as a hard disk or memory of the electronic device 500. In other embodiments, the memory 502 may also be an external storage device of the electronic device 500, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 500.

[0042] Furthermore, the memory 502 may include both an internal storage unit of the electronic device 500 and an external storage device. The memory 502 is used to store application software installed in the electronic device 500 and various data.

[0043] In some embodiments, the processor 501 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 502, such as the fire video-based warning information generation method of the present invention.

[0044] In some embodiments, the display 503 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 503 is used to display information of the electronic device 500 and to display a visual user interface. The components 501-503 of the electronic device 500 communicate with each other through a system bus.

[0045] In some embodiments of the present invention, when the processor 501 executes the fire video-based warning information generation program in the memory 502, the following steps may be implemented: Obtaining a tunnel video collected by a video detection device in the tunnel, annotating and extracting the tunnel video, and obtaining a firefighting video; Extract images from firefighting videos according to a preset frame rate to obtain target images; When a traffic accident is detected in the target image, the target image is subjected to accident detection to determine the type of the accident; The accident location is determined based on the accident type, video detection equipment and target image, and early warning information is generated based on the accident location.

[0046] It should be understood that: when the processor 501 executes the fire video-based warning information generation program in the memory 502, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0047] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 500 mentioned, and the electronic device 500 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 500 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0048] Accordingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the steps or functions of the method for generating warning information based on fire videos provided in the above-mentioned method embodiments.

[0049] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0050] The above is a detailed introduction to the method and device for generating warning information based on fire videos provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for generating early warning information based on fire video, characterized in that: include: Obtaining a tunnel video collected by a video detection device in the tunnel, annotating and extracting the tunnel video, and obtaining a firefighting video; Extracting images from the firefighting video according to a preset frame rate to obtain a target image; When a traffic accident is detected in the target image, performing accident detection on the target image to determine the type of the accident; The accident location is determined according to the accident type, the video detection device and the target image, and warning information is generated according to the accident location.

2. The method for generating early warning information based on fire video according to claim 1, characterized in that: The step of extracting an image from the firefighting video according to a preset frame rate to obtain a target image includes: Extracting images from the firefighting video according to the preset frame rate to obtain an image set; Screening the images in the image set to obtain an initial image; The initial image is preprocessed to obtain a target image.

3. The method for generating early warning information based on fire video according to claim 2, characterized in that: The step of extracting images from the firefighting video according to the preset frame rate to obtain an image set includes: Extracting images from the firefighting video according to the preset frame rate to obtain continuous image frames; Resizing each image frame in the continuous image frames according to a preset size to obtain all images; Based on all the images, an image set is obtained.

4. The method for generating early warning information based on fire video according to claim 1, characterized in that: After extracting the image from the firefighting video according to the preset frame rate to obtain the target image, the method further includes: Performing target detection on the target image to obtain a detection result; Determine whether a traffic accident exists based on the detection result.

5. The method for generating early warning information based on fire video according to claim 4, characterized in that: The determining the accident location according to the accident type, the video detection device and the target image includes: When the accident type is a vehicle accident, determining the location information of the video detection device; The accident location is determined based on the detection result of the target image and the location information.

6. The method for generating early warning information based on fire video according to claim 1, characterized in that: The determining of the accident location according to the accident type, the video detection device and the target image further includes: When the accident type is a vehicle fire, obtaining smoke detection data of a smoke sensor according to a video detection device; The smoke detection data is calculated and processed according to a preset smoke recognition algorithm to obtain a smoke detection result; The accident location is determined according to the target image and the smoke detection result.

7. The method for generating early warning information based on fire video according to claim 6, characterized in that: The step of obtaining smoke detection data of a smoke sensor according to a video detection device includes: Determining location information of the video detection device; The smoke detection data of the smoke sensor at the corresponding position is acquired according to the position information.

8. The method for generating early warning information based on fire video according to claim 4, characterized in that: The performing target detection on the target image to obtain a detection result includes: Set up a traffic accident detection model; The target image is input into the traffic accident detection model to perform target detection and obtain a detection result.

9. The method for generating early warning information based on fire video according to claim 1, characterized in that: The step of labeling and extracting the tunnel video to obtain a firefighting video includes: Set up feature annotation datasets for various types of fire accidents in tunnels; Extracting annotations from the tunnel video according to the feature annotation data set to obtain an annotated video; The labeled video is cropped to obtain a firefighting video.

10. A device for generating early warning information based on fire video, characterized in that: include: A video acquisition module is used to acquire the tunnel video collected by the video detection equipment in the tunnel, annotate and extract the tunnel video, and obtain the firefighting video; An image extraction module is used to extract images from the firefighting video according to a preset frame rate to obtain a target image; An accident determination module, configured to perform accident detection on the target image and determine the type of accident when a traffic accident is detected in the target image; The accident location module is used to determine the accident location according to the accident type, the video detection equipment and the target image, and to generate warning information according to the accident location.

Images