A method and system for fire source search using a fire-fighting robot

Abstract

This invention discloses a fire source search method and system for fire-fighting robots in the field of image recognition, comprising: controlling a high-definition camera of a fire-fighting robot to scan a fire scene and obtain real-time high-definition images of the scanned area; inputting the real-time high-definition images into a trained primary deep learning model to perform primary detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, redundant segmentation is performed on the real-time high-definition images to obtain unit detection images; inputting the unit detection images into a trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection images; and controlling the fire-fighting robot to perform fire-fighting actions based on the extracted fire area coordinates; effectively balancing the real-time performance and accuracy of fire source search, and assisting the fire-fighting robot to perform timely and effective fire-fighting actions.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to a method and system for fire source search by a fire-fighting robot. Background Technology

[0002] With rapid socio-economic development and the unique characteristics of construction and enterprise production, the risks of chemical and radioactive material leaks, as well as fires, explosions, and collapses, have increased, leading to a corresponding rise in the probability of fire accidents. Firefighters face hazardous environments such as dense smoke, high temperatures, darkness, and toxic substances when fighting fires; direct entry into such environments poses significant safety risks. Intelligent firefighting robots can greatly improve firefighting efficiency, buy precious time in extinguishing fires, suppress their spread, and reduce property damage and casualties.

[0003] Some existing technologies are slow to identify major fire sources, thus losing the best time to fight the fire; others have low identification rates for small fire sources, which can easily lead to the spread of the fire, and the identification distance is short, which cannot meet the needs of intelligent fire-fighting robots for long-distance fire fighting. As a result, existing fire source search methods for fire-fighting robots cannot effectively balance the real-time performance and accuracy of fire source search. Summary of the Invention

[0004] The purpose of this invention is to provide a fire source search method and system for fire-fighting robots, which effectively balances the real-time performance and accuracy of fire source search, and assists fire-fighting robots in performing timely and effective fire-fighting operations.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The first aspect of this invention provides a method for fire source search using a fire-fighting robot, comprising:

[0007] The high-definition camera controlling the firefighting robot scans the fire scene to obtain real-time high-definition images of the scanned area;

[0008] The real-time high-definition image is input into the trained primary deep learning model to perform a preliminary detection of whether a fire has occurred in the scanned area; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the real-time high-definition image.

[0009] If no fire is detected in the scanned area, redundant segmentation is performed on the real-time high-definition image to obtain the unit detection image;

[0010] The unit detection image is input into the trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, the scanned area in the fire scene is changed to search for the fire source again; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection image.

[0011] Firefighting robots are controlled to perform firefighting operations based on the coordinates of the fire area extracted during the primary or advanced detection process.

[0012] Preferably, the method for obtaining unit detection images by performing redundant segmentation on real-time high-definition images includes:

[0013] The real-time high-definition image is first segmented to obtain the basic unit image;

[0014] A 2×2 basic unit image matrix is ​​extracted from the real-time high-definition image, and overlapping unit images are extracted from the center of the 2×2 basic unit image matrix; the basic unit images and overlapping unit images are used as unit detection images.

[0015] Preferably, the overlapping unit image and the base unit image have the same size.

[0016] Preferably, the method for obtaining basic unit images by performing a first segmentation on real-time high-definition images includes:

[0017] Real-time high-definition images are segmented into 2×2 basic unit image matrices or 3×3 basic unit image matrices.

[0018] Preferably, there is an overlapping area between two adjacent unit detection images.

[0019] Preferably, the training methods for the primary deep learning model and the advanced deep learning model include:

[0020] Collect historical high-resolution images of the fire scene to construct a primary training set; train the deep learning model using the primary training set to obtain a primary deep learning model with a detection accuracy greater than a set threshold A;

[0021] Historical high-definition images are segmented to obtain historical unit detection images, and a high-level training set is constructed. The deep learning model is trained using the high-level training set to obtain a high-level deep learning model with a detection accuracy greater than a set threshold B.

[0022] Preferably, the deep learning model is configured as a high real-time model, including but not limited to the YOLOv3 model, YOLOv4 model, or EfficientDet model.

[0023] A second aspect of the present invention provides a fire source search system for a fire-fighting robot, comprising:

[0024] The data acquisition module is used to control the high-definition camera of the fire-fighting robot to scan the fire scene and obtain real-time high-definition images of the scanned area;

[0025] The primary detection module is used to input real-time high-definition images into a pre-trained primary deep learning model to perform primary detection of whether a fire has occurred within the scanned area; if a fire is detected within the scanned area, the coordinates of the fire area are extracted from the real-time high-definition image.

[0026] The image segmentation module is used to redundantly segment the real-time high-definition image to obtain the unit detection image if no fire is detected in the scanned area.

[0027] The advanced detection module is used to input the unit detection image into the trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, the scanned area in the fire scene is changed to search for the fire source again; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection image.

[0028] The control module controls the fire-fighting robot to perform fire-fighting operations based on the coordinates of the fire area extracted during the primary or advanced detection process.

[0029] Preferably, the firefighting robot is equipped with an infrared sensor for detecting temperature; the control module controls the high-definition camera of the firefighting robot to move in the direction of rising temperature to scan the fire scene.

[0030] Preferably, the high-definition camera has a resolution of 2K or 4K.

[0031] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the fire source search method.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] This invention inputs real-time high-definition images into a trained primary deep learning model to perform primary detection of whether a fire has occurred within the scanned area. If no fire is detected within the scanned area, redundant segmentation is performed on the real-time high-definition images to obtain unit detection images. The unit detection images are then input into a trained advanced deep learning model to perform advanced detection of whether a fire has occurred within the scanned area. Primary detection ensures the real-time performance of fire source search, while advanced detection ensures the accuracy of fire source search. At the same time, advanced retrieval can improve the fire-fighting robot's detection and extinguishing distance, avoiding the impact of too close a detection and extinguishing distance on the fire-fighting robot's operation.

[0034] In this invention, a basic unit image is obtained by first segmenting a real-time high-definition image; a 2×2 basic unit image matrix is ​​extracted from the real-time high-definition image, and an overlapping unit image is extracted from the center of the 2×2 basic unit image matrix; the basic unit image and the overlapping unit image are used as unit detection images; by extracting the overlapping unit image, the detection results can be avoided after the fire source image is segmented, thus improving the accuracy of fire source search. Attached Figure Description

[0035] Figure 1 This is a flowchart of a fire source search method for a fire-fighting robot provided in Embodiment 1 of the present invention;

[0036] Figure 2 This is a segmentation diagram of a real-time high-definition image provided in Embodiment 1 of the present invention. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0038] Example 1

[0039] like Figure 1 As shown, the first aspect of the present invention provides a method for detecting potential hazards based on inspection videos, comprising:

[0040] Methods for training basic and advanced deep learning models include:

[0041] Collect historical high-resolution images of the fire scene to construct a primary training set; train the deep learning model using the primary training set to obtain a primary deep learning model with a detection accuracy greater than a set threshold A;

[0042] Historical high-definition images are segmented to obtain historical unit detection images, and a high-level training set is constructed. A deep learning model is trained using the high-level training set to obtain a high-level deep learning model with a detection accuracy greater than a set threshold B. The deep learning model is set as a high real-time model, including but not limited to the YOLOv3 model, YOLOv4 model, or EfficientDet model.

[0043] The high-definition camera controlling the firefighting robot scans the fire scene to obtain real-time high-definition images of the scanned area;

[0044] Real-time high-definition images are input into a pre-trained primary deep learning model to perform a preliminary detection of whether a fire has occurred within the scanned area. If a fire is detected, the coordinates of the fire area are extracted from the real-time high-definition images. Based on the fire area coordinates extracted during the primary detection process, the fire-fighting robot is controlled to perform fire-fighting actions. The primary detection ensures the real-time nature of fire source search and avoids losing the best fire-fighting time due to excessively long search times.

[0045] If no fire is detected within the scanned area, such as Figure 2 As shown, the method for segmenting real-time high-definition images to obtain unit detection images includes:

[0046] The real-time high-definition image is first segmented to obtain basic unit images; the real-time high-definition image is segmented into a 2×2 basic unit image matrix, and the real-time high-definition image is divided into 4 equal parts; or, the real-time high-definition image is segmented into a 3×3 basic unit image matrix, and the real-time high-definition image is divided into 9 equal parts.

[0047] A 2×2 basic unit image matrix is ​​extracted from real-time high-definition images, and overlapping unit images are extracted from the center of the 2×2 basic unit image matrix. The image segmentation method provided in this embodiment can avoid the segmentation of large fire source images. The overlapping unit images and basic unit images are the same size. The basic unit images and overlapping unit images are used as unit detection images. By extracting the overlapping unit images, the fire source image segmentation can avoid affecting the detection results and improve the accuracy of fire source search.

[0048] The unit detection image is input into the trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, the scanned area in the fire scene is changed to search for the fire source again; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection image.

[0049] The fire-fighting robot is controlled to perform fire-fighting operations based on the coordinates of the fire area extracted during the advanced detection process. Advanced detection ensures the accuracy of fire source search, while advanced retrieval can improve the fire-fighting robot's detection and extinguishing distance, avoiding the impact of the fire-fighting robot's operation if the detection and extinguishing distance is too close.

[0050] Example 2

[0051] A fire source search system for a fire-fighting robot, the system provided in this embodiment can be applied to the method described in embodiment one, the fire source search system includes:

[0052] The data acquisition module is used to control the high-definition camera of the fire-fighting robot to scan the fire scene and obtain real-time high-definition images of the scanned area;

[0053] The primary detection module is used to input real-time high-definition images into a pre-trained primary deep learning model to perform primary detection of whether a fire has occurred within the scanned area; if a fire is detected within the scanned area, the coordinates of the fire area are extracted from the real-time high-definition image.

[0054] The image segmentation module is used to redundantly segment the real-time high-definition image to obtain the unit detection image if no fire is detected in the scanned area.

[0055] The advanced detection module is used to input the unit detection image into the trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, the scanned area in the fire scene is changed to search for the fire source again; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection image.

[0056] The control module controls the fire-fighting robot to perform fire-fighting operations based on the coordinates of the fire area extracted during the primary or advanced detection process.

[0057] Preferably, the firefighting robot is equipped with an infrared sensor for detecting temperature; the control module controls the high-definition camera of the firefighting robot to move in the direction of rising temperature to scan the fire scene.

[0058] Example 3

[0059] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the fire source search method described in Embodiment 1.

[0060] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for fire source search using a firefighting robot, characterized in that, include: The high-definition camera controlling the firefighting robot scans the fire scene to obtain real-time high-definition images of the scanned area; The real-time high-definition image is input into the trained primary deep learning model to perform a preliminary detection of whether a fire has occurred in the scanned area; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the real-time high-definition image. If no fire is detected within the scanned area, redundant segmentation is performed on the real-time high-definition image to obtain the unit detection image, specifically including: The real-time high-definition image is first segmented to obtain the basic unit image; Real-time high-definition image extraction The basic unit image matrix is ​​composed of The overlapping unit image is extracted from the center of the basic unit image matrix; the basic unit image and the overlapping unit image are used as the unit detection image; The unit detection image is input into the trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, the scanned area in the fire scene is changed to search for the fire source again; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection image. Firefighting robots are controlled to perform firefighting operations based on the coordinates of the fire area extracted during the primary or advanced detection process.

2. The fire source search method for a fire-fighting robot according to claim 1, characterized in that, The overlapping unit image and the base unit image are the same size.

3. The fire source search method for a fire-fighting robot according to claim 1, characterized in that, Methods for obtaining basic unit images through the first segmentation of real-time high-definition images include: Segment the real-time high-definition image into Basic unit image matrix or Basic unit image matrix.

4. The fire source search method for a fire-fighting robot according to claim 1, characterized in that, The training methods for the primary and advanced deep learning models include: Collect historical high-resolution images of the fire scene to construct a primary training set; train the deep learning model using the primary training set to obtain a primary deep learning model with a detection accuracy greater than a set threshold A; Historical high-definition images are segmented to obtain historical unit detection images, and a high-level training set is constructed. The deep learning model is trained using the high-level training set to obtain a high-level deep learning model with a detection accuracy greater than a set threshold B.

5. A fire source search method for a fire-fighting robot according to claim 4, characterized in that, The deep learning model is set to a YOLOv3 model, a YOLOv4 model, or an EfficientDet model.

6. A fire source search system for a fire-fighting robot, characterized in that, include: The data acquisition module is used to control the high-definition camera of the fire-fighting robot to scan the fire scene and obtain real-time high-definition images of the scanned area; The primary detection module is used to input real-time high-definition images into a pre-trained primary deep learning model to perform primary detection of whether a fire has occurred within the scanned area; if a fire is detected within the scanned area, the coordinates of the fire area are extracted from the real-time high-definition image. The image segmentation module is used to redundantly segment the real-time high-definition image to obtain the unit detection image if no fire is detected in the scanned area. The advanced detection module is used to input the unit detection image into the trained advanced deep learning model to perform advanced detection of whether a fire has occurred in the scanned area; if no fire is detected in the scanned area, the scanned area in the fire scene is changed to search for the fire source again; if a fire is detected in the scanned area, the coordinates of the fire area are extracted from the unit detection image. The control module controls the firefighting robot to perform firefighting operations based on the coordinates of the fire area extracted during the primary or advanced detection process. The image segmentation module performs redundant segmentation on real-time high-definition images to obtain unit detection images, specifically including: The real-time high-definition image is first segmented to obtain the basic unit image; Real-time high-definition image extraction The basic unit image matrix is ​​composed of The overlapping unit image is extracted from the center of the basic unit image matrix; the basic unit image and the overlapping unit image are used as the unit detection image.

7. A fire source search system for a fire-fighting robot according to claim 6, characterized in that, The firefighting robot is equipped with an infrared sensor for detecting temperature; the control module controls the high-definition camera of the firefighting robot to move in the direction of rising temperature to scan the fire scene.

8. A fire source search system for a fire-fighting robot according to claim 6, characterized in that, The high-definition camera has a resolution of 2K or 4K.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the fire source search method according to any one of claims 1 to 5.

Images