Method for detecting location of fire, automatic fire extinguishing method, system and fire truck

Abstract

The application provides a method for detecting the position of a fire source, an automatic fire extinguishing method, a system and a fire truck, and belongs to the technical field of fire trucks. The method comprises the following steps: acquiring an image of a fire scene and scanned radar point cloud data, the image and the radar point cloud data being associated through a mapping model; determining a flame ROI in the image and a first depth value of the flame ROI; extending the flame ROI and determining a target space region based on the mapping model, the extended flame ROI and the first depth value; and identifying a target fire source and position data of the target fire source based on target point cloud data, the target point cloud data being radar point cloud data located in the target space region. The application can be used for long-distance automatic fire extinguishing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fire fighting vehicles, in particular to a method for detecting the position of a fire object, an automatic fire extinguishing method for fire fighting equipment, a system for detecting the position of a fire object, an electronic device, a fire fighting vehicle and a machine readable storage medium. BACKGROUND

[0002] Fire is a strong chemical reaction in the process of burning of matter, and a flame is a gaseous mixture of matter participating in the burning process, which is visible. The flame at the scene of a fire often has a high temperature, and can quickly cool and effectively extinguish the fire object when aimed at the fire object.

[0003] At present, the fire fighting vehicles that complete the fire extinguishing operation need firefighters to observe the target flame position, manually control or remotely control the adjustment of the posture of the fire gun, preliminarily aim at the flame area, and then open the jet. During the fire extinguishing operation, the posture of the fire gun needs to be finely adjusted, and the position and posture need to be constantly corrected. Usually, the adjustment time of the forearm bracket and the fire gun before extinguishing the fire is at least more than 60 seconds, and multiple people need to cooperate. The personnel operation of the conventional fire extinguishing method is difficult, the fire extinguishing process is complicated, the preparation time is long, the fire extinguishing efficiency is low, the professional accomplishment of the operator is required, and the safety of the firefighters is difficult to guarantee. SUMMARY

[0004] The purpose of the present application is to provide a method for detecting the position of a fire object, an automatic fire extinguishing method, a system and a fire fighting vehicle, so as to realize the determination of the position of a fire object, the full-automatic fire extinguishing fire fighting equipment and the safety of the personnel in the fire extinguishing operation, and improve the accuracy and efficiency of the fire extinguishing operation.

[0005] In order to achieve the above purpose, the embodiment of the present application provides a method for detecting the position of a fire object, which comprises the following steps:

[0006] obtaining an image of a fire scene and scanned radar point cloud data, wherein the image and the radar point cloud data are associated through a mapping model;

[0007] determining a flame ROI in the photographed image and a first depth value of the flame ROI;

[0008] extending the flame ROI, and determining a target space region based on the mapping model, the extended flame ROI and the first depth value;

[0009] based on the target point cloud data, identifying a target fire object and position data of the target fire object, wherein the target point cloud data is radar point cloud data located in the target space region.

[0010] Specifically, the flame ROI is extended, and a target space region is determined based on the mapping model, the extended flame ROI, and the first depth value, including:

[0011] A three-dimensional cone region is determined based on the mapping model and the extended flame ROI;

[0012] The three-dimensional cone region is intercepted using a neighborhood of the first depth value to obtain the target space region.

[0013] Specifically, the flame ROI is extended, and a target space region is determined based on the mapping model, the extended flame ROI, and the first depth value, including:

[0014] A three-dimensional ring region is determined based on a neighborhood of the first depth value;

[0015] The three-dimensional ring region is intercepted based on the mapping model and the extended flame ROI to obtain the target space region.

[0016] Specifically, the captured image includes an infrared panoramic image and a visible light panoramic image, and an image mapping relationship exists between the infrared panoramic image and the visible light panoramic image;

[0017] The determination of the flame ROI in the captured image includes:

[0018] A first flame region is obtained by detecting a flame feature of the visible light panoramic image;

[0019] A suspected flame region is determined by performing background division on the infrared panoramic image, and a second flame region is obtained by detecting a flame feature of the suspected flame region;

[0020] A flame ROI is determined according to the first flame region, the second flame region, and the image mapping relationship.

[0021] Specifically, the determination of the flame ROI according to the first flame region, the second flame region, and the image mapping relationship includes:

[0022] The first flame region is mapped into the infrared panoramic image;

[0023] An overlapping region between the second flame region and the mapped first flame region is determined;

[0024] A region with a color temperature less than a specified threshold in the overlapping region is removed to obtain the flame ROI.

[0025] Specifically, the determination of the first depth value of the flame ROI includes:

[0026] determine two center points between images of two adjacent frames containing a target region, which is a flame ROI in the infrared panoramic image, calculate a pixel deviation value between the two center points;

[0027] Based on the rotation angle of the two adjacent frames of images, the camera intrinsic parameter and the pixel deviation value, the depth value of any one center point is determined.

[0028] Specifically, the target space region is obtained by intercepting the three-dimensional cone region using the neighborhood of the first depth value, including:

[0029] Determine the neighborhood boundary value of the first depth value, and the neighborhood has a specified threshold interval size;

[0030] Determine the cross section with a left neighborhood boundary value and the cross section with a right neighborhood boundary value in the three-dimensional cone region;

[0031] The body region between the two cross sections in the three-dimensional cone region is taken as the target space region.

[0032] Specifically, the target point cloud data is based on the target point cloud data, and the position data of the target fire object is identified, including:

[0033] The target point cloud data is clustered to obtain the target fire object and the position data of the target fire object.

[0034] Specifically, the determination method of the mapping model includes:

[0035] Calibrate the radar coordinate system and the image coordinate system, and determine the mapping model of the to-be-solved parameter matrix;

[0036] Convert the radar point cloud data into a two-dimensional point cloud image, extract the feature points overlapping each other in the captured image and the two-dimensional point cloud image;

[0037] Determine the numerical value of the to-be-solved parameter matrix through the mutually overlapping feature points.

[0038] The embodiment of the application also provides an automatic fire extinguishing method of fire fighting equipment, the fire fighting equipment comprising a control unit and a fire monitor, the automatic fire extinguishing method comprising:

[0039] Determine the position data of the target fire object by executing the aforementioned method for detecting the position of the fire object through the control unit;

[0040] Control the fire monitor to rotate and aim at the specified position of the target fire object based on the position data through the control unit, and execute the fire extinguishing operation.

[0041] Specifically, the fire-fighting equipment further comprises a camera unit and a radar unit, both of which are arranged on the fire monitor, and the automatic fire extinguishing method further comprises:

[0042] rotating the fire monitor to a specified angle by the control unit, taking a visible light image and an infrared image of the fire site by the camera unit, and synchronously scanning the fire site by the radar unit;

[0043] rotating the fire monitor to a next specified angle by the control unit, taking a visible light image and an infrared image of the fire site by the camera unit, and synchronously scanning the fire site by the radar unit;

[0044] wherein the visible light image and the infrared image taken at each angle are used to form a visible light panoramic image and an infrared panoramic image respectively, and the data of scanning the fire site at each angle is used to form radar point cloud data.

[0045] The embodiment of the present application also provides a system for detecting the position of a fire object, which comprises:

[0046] an acquisition module, configured to acquire an image taken of a fire site and radar point cloud data scanned, the image and the radar point cloud data being associated by a mapping model;

[0047] an image processing module, configured to determine a flame ROI in the taken image and a first depth value of the flame ROI;

[0048] a point cloud processing module, configured to extend the flame ROI and determine a target space region based on the mapping model, the extended flame ROI and the first depth value;

[0049] a determination module, configured to identify a target fire object and position data of the target fire object based on target point cloud data, the target point cloud data being radar point cloud data located in the target space region.

[0050] In still another aspect, the embodiment of the present application provides an electronic device, which comprises:

[0051] at least one processor;

[0052] a memory connected with the at least one processor;

[0053] wherein the memory stores instructions executable by the at least one processor, and the at least one processor realizes the foregoing method by executing the instructions stored in the memory.

[0054] In another aspect, the embodiments of the present application provide a fire truck equipped with the fire-fighting equipment in the automatic fire extinguishing method of the fire-fighting equipment and / or the electronic device.

[0055] In another aspect, the embodiments of the present application provide a machine readable storage medium storing machine instructions, which, when executed on a machine, cause the machine to perform the method.

[0056] In the present application, the captured image is used to find the flame ROI, and the burning object is marked by the extended flame ROI. Since the specified tangent point or tangent plane of the flame root in the fire scene (which can be regarded as three-dimensional) can be in close contact with the burning surface of the burning object, or the shortest distance can be less than the distance threshold, the depth value (range) of the image area where the extended flame ROI is located intersects with the depth value range of the burning object in the image depth direction, and the depth value range of the target burning object can be located by using the first depth value. At the same time, by using the image feature information carried by the extended flame ROI and the mapping model between the image and the radar point cloud data, the key field of view of the radar point cloud data containing the burning object point cloud is found in the scanned radar point cloud data, and the three-dimensional body region containing the burning object point cloud is correlatedly located, which is the target space region. The target point cloud data in the target space region will contain the burning object point cloud and the non-burning object point cloud, and finally, the target burning object is identified in the target point cloud data in the target space region, and the position data of the target burning object is determined.

[0057] The embodiments of the present application can realize automatic identification of the position of the target burning object, and compared with the conventional manual fire extinguishing method, can effectively improve the identification efficiency and accuracy of the target burning object, which is helpful to support the fire extinguishing work of the target burning object, reduce the fire extinguishing preparation time, improve the fire extinguishing efficiency, and further realize remote automatic fire extinguishing and protect the personal safety of the firefighters.

[0058] The present application provides an example selection of the intercepted three-dimensional cone region as the target space region, and the three-dimensional cone region with projected planar image features can be found by using the mapping model and the extended flame ROI. The planar image features are similar or consistent with the image features of the extended flame ROI. Then, the target space region is obtained by intercepting the three-dimensional cone region in the depth direction by using the neighborhood of the first depth value.

[0059] This invention provides an example selection of a truncated three-dimensional annular region as a target spatial region. First, using the neighborhood of a first depth value, a three-dimensional annular region can be determined in the depth direction within the space containing radar point cloud data. Then, using a mapping model and an extended flame ROI, a volumetric region (truncated from the three-dimensional annular region) with projected planar image features is found. The planar image features are similar to or consistent with the image features of the extended flame ROI. The truncated volumetric region is the target spatial region. That is, the use of the extended flame ROI image feature information combined with the mapping model can precede or follow the use of the first depth value information. Besides the truncated three-dimensional annular region and the truncated three-dimensional cone region mentioned above, other different simple volumetric region divisions can also be performed on the three-dimensional space containing radar point cloud data. Simple volumetric regions include, for example, rectangular regions and cylindrical regions (for example, a sphere is constructed from the origin, and the diameter of the sphere can be the same as the diameter of the base of the cylindrical region), etc.

[0060] In this invention, the flame area in the visible light image can be specifically mapped to the flame area in the infrared image, and the low color temperature area of ​​the overlapping area can be eliminated and filtered out, which can greatly improve the impact of interference caused by the acquisition process and the on-site environment; the flame area in the infrared image can also be specifically mapped to the flame area in the visible light image, making it easy to mark the burning objects and personnel for observation in the established panoramic image of the fire scene.

[0061] In this invention, the first depth value can be determined by using two adjacent frames of images containing the target region. By combining the camera intrinsic parameters and the captured images with the similarity relationship in the camera coordinate system and the image pixel deviation value, the depth value of the center point of the target region in the two adjacent frames of images can be obtained.

[0062] In this invention, the neighborhood boundary value of the first depth value can be used as the volume region for cross-section acquisition, which reduces the difficulty of selecting point cloud data; or more cross-sections and / or multiple volume regions can be used on this basis, which reduces the scale of point cloud data involved in the calculation.

[0063] This invention can specifically employ clustering for identification, offering advantages in accuracy; alternatively, it can utilize trained machine learning models for identification, providing advantages in processing efficiency. In this invention, radar point cloud data and captured images can be stored in radar coordinate systems and image coordinate systems, respectively. A transformation relationship can be established between these coordinate systems; this transformation relationship is a mapping model. Before using the aforementioned mapping model in conjunction with the extended flame ROI, overlapping feature points can be used to determine the specific expression of the mapping model. Thus, by using the extended flame ROI as input to the mapping model, the three-dimensional cone region or the intercepted volume region can be determined as the output of the mapping model.

[0064] The specific features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0065] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0066] Figure 1 This is a schematic diagram of the main method steps in an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of the hardware unit of an exemplary fire-fighting equipment according to an embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram illustrating the positions of an exemplary camera unit and radar unit on a fire monitor according to an embodiment of the present invention.

[0069] Figure 4 This is an exemplary schematic diagram illustrating the relative relationship between the radar coordinate system and the world coordinate system according to an embodiment of the present invention;

[0070] Figure 5 This is a schematic cross-sectional view of an exemplary Cube region according to an embodiment of the present invention;

[0071] Figure 6 This is an exemplary three-dimensional schematic diagram of a Cube region according to an embodiment of the present invention;

[0072] Figure 7 This is a schematic cross-sectional view of an exemplary Cube cut-out region according to an embodiment of the present invention;

[0073] Figure 8 This is a schematic diagram of a three-dimensional annular region in an exemplary application scenario of an embodiment of the present invention;

[0074] Figure 9 This is a schematic diagram illustrating the steps of a control unit performing an operation in an exemplary application scenario according to an embodiment of the present invention;

[0075] Figure 10 This is a schematic diagram illustrating the installation location of fire-fighting equipment deployed on an exemplary fire truck according to an embodiment of the present invention. Detailed Implementation

[0076] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0077] Fire detection and monitoring technologies can be used for fire suppression, but these technologies are mainly used in large indoor spaces and fixed locations, such as forest farms, petrochemical production and storage sites, and tunnels. However, these detectors are fixed in location, and in outdoor environments, with dynamically changing spatial positions of the detection equipment, or in situations with dense smoke, accurate flame detection is difficult, easily leading to delayed fire detection and resulting in personal injury and property damage. Flame location technology can also be used, typically employing monocular or binocular vision for location identification. However, the effective location distance is very limited, often restricted to within 15 meters, making it difficult to handle long-distance fire sources and carry out fire suppression operations. Long distances typically refer to distances exceeding 50 meters, making it ineffective in scenarios involving fire trucks and autonomous robots. Furthermore, due to the high-temperature plasma and high-temperature reactants of flames, effective point cloud data of the flame will not be obtained when attempting to locate the flame using radar scanning. This invention will provide solutions for automated fire suppression equipment and fire trucks.

[0078] Example 1

[0079] This invention provides a method for detecting the location of a burning object, which can be applied to fire-fighting equipment, such as... Figure 1 The method may include:

[0080] A1) Acquire images of the fire scene and scanned radar point cloud data, wherein the images and the radar point cloud data are associated through a mapping model;

[0081] A2) Determine the flame ROI in the captured image, and the first depth value of the flame ROI;

[0082] A3) Extend the flame ROI and determine the target spatial region based on the mapping model, the extended flame ROI, and the first depth value;

[0083] A4) Based on the target point cloud data, identify the target burning object and the location data of the target burning object, wherein the target point cloud data is radar point cloud data located in the target spatial region.

[0084] In this embodiment of the invention, the fire-fighting equipment may include a control unit and a data acquisition unit. The control unit can execute a method for detecting the location of a burning object. In the aforementioned step A1), the captured images and radar point cloud data may be obtained before step A1) through communication between the control unit and the data acquisition unit, and the data obtained by the control unit may be obtained, for example, through bus communication, wireless communication, or other means.

[0085] In an exemplary example of fire-fighting equipment disclosed in this invention, the fire-fighting equipment may include a data acquisition unit, a control unit, and a fire monitor. For example... Figure 2The camera unit in the data acquisition unit can include infrared (light) cameras (i.e., infrared cameras) and visible light cameras (i.e., visible light cameras). The radar unit in the data acquisition unit can be lidar or long-range scanning radar. The control unit can include an industrial computer (or server) and a controller. The industrial computer can be installed in the vehicle body and can include high-performance image processing units and central processing units, etc., which facilitate rapid data processing. The industrial computer can receive data transmitted by the camera unit and radar unit through a wireless bridge, which will help reduce wiring and lower costs. The industrial computer can be configured with an executable application / script, which can be used to build a fire scene from radar point cloud data, match and stitch panoramic images, and perform steps A1) to A4). The controller can include a logic programmable controller, a motor controller, a pump controller, an integrated circuit with a microcontroller chip and / or a system-on-a-chip, etc. The controller can connect to the industrial computer through a USB-CAN bus converter. Based on the data output by the industrial computer, the controller can control the motor to drive the fire monitor to rotate and operate the fire extinguishing electrical equipment, etc.

[0086] like Figure 3 The aforementioned fire monitor may include a motor, a rotating mechanism, and a barrel. The barrel can be rotated by two motors (motor 1 and motor 2) via the rotating mechanism, causing the barrel's axis to point to a designated position. The two motors can be used to rotate the fire monitor horizontally and the fire monitor's barrel vertically, respectively. This firefighting equipment can be deployed on tracked, wheeled, or autonomous robotic fire trucks. These fire trucks may be equipped with support legs and a lifting arm, which, when extended, can raise the working platform where the fire monitor is mounted. In this example of firefighting equipment, infrared cameras, visible light cameras, and lidar can all be fixed to the fire monitor using brackets. In scenarios that simplify calculations and calibration, the optical axes of the infrared camera, visible light camera, and lidar can be parallel to any two of them, and can also be parallel to the fire monitor's axis. Furthermore, the orientation of the infrared camera, visible light camera, and lidar is the same as the direction of the fire monitor's jet stream, thereby simplifying calculations and structure, reducing the cost of the motion mechanism, and eliminating the need for separate pan-tilt units and transmission mechanisms for the camera unit, radar unit, and lidar. In scenarios where operators can observe the fire scene and fire trends, the aforementioned firefighting equipment may also include a touchscreen with input functionality. This touchscreen can be connected to an industrial control computer to display captured or created fire scenes and scanned radar point cloud data. Simultaneously, the industrial control computer can also communicate with mobile devices carried by firefighters to display captured or created fire scenes and scanned radar point cloud data. Mobile devices include dedicated handheld terminals, tablets, and mobile phones.

[0087] In this embodiment of the invention, the image in step A1) can be an image captured within a specified angle range, and the radar point cloud data can be point cloud data scanned synchronously within that specified angle range. In some exemplary application scenarios, for example, if the fire scene is located on one side of the line connecting the front and rear of a fire truck, firefighters can activate the firefighting equipment via electrical switches, touch screens, etc., and select the working mode of the specified acquisition side. During detection, the control unit can set the specified angle range to 0°~180° or 180°~360°. After the camera unit and radar unit scan the fire scene, the images and point cloud data of the remaining angle range (the side not at the fire scene) can be designated as the default image (specified value) and default point cloud data (specified value). The captured image can then be combined with the default image to form a combined image containing the fire scene, and the radar point cloud data can be combined with the default point cloud data to form combined point cloud data containing the fire scene. This avoids data interference unrelated to the fire scene, reduces data computation, and speeds up the determination of the location data of the burning object. For example, firefighters can activate firefighting equipment via electrical switches or touchscreens without any selection. The fire monitor can then rotate 0° to 360° vertically, capturing a panoramic image of the fire scene. This panoramic image includes both infrared and visible light panoramic images, and the scanned radar point cloud data is also a 0°–360° panoramic point cloud data. The scanning of the radar point cloud data is synchronized with the image capture. An image captured at any given viewpoint has a mapping relationship with the radar point cloud data scanned at that viewpoint (or within the field of view of that viewpoint). For example, with the aforementioned installation arrangement, rotating and scanning are performed, and the optical axes of each unit are parallel to each other. This coordinate system can be used to calibrate the mapping model, allowing the image and point cloud data to be used as input data and matched with the mapping model without data correction or adjustment, reducing computation and accelerating the determination of the location of the burning object.

[0088] In this embodiment of the invention, the panoramic image can be at least two frames captured by the camera unit before and after being rotated by at least one angle, and the captured images are matched and stitched together. In some scenarios that are more conducive to achieving full automation, as mentioned above, the camera unit and radar unit can rotate with the fire monitor, being rotated 360 degrees step by step. The fire monitor can be rotated by the same fixed angle α in each step (the interval between two adjacent steps is fixed by the same angle, which helps to simplify calculations). The angle of rotation of the optical axis of each camera is the rotation angle of the fire monitor. The image captured before rotation can be placed in a specified reference plane (which can be perpendicular to the depth direction). The feature points of two adjacent frames before and after rotation can be matched to determine the transformation relationship between the two adjacent frames (e.g., translation, rotation, affine transformation, etc.). Then, the two frames are superimposed on the reference plane according to the transformation relationship, thereby using the captured images to match and stitch together the images to form a panoramic image of the fire scene in the reference plane.

[0089] In this embodiment of the invention, the radar unit also rotates synchronously with the fire monitor. The rotation angle of the radar unit's optical axis is the same as the rotation angle of the fire monitor. A point cloud data stitching ( / registration) algorithm can be used to stitch together the point cloud data scanned before and after rotation. By using the radar point cloud data scanned step by step by the radar unit, radar point cloud data of the fire scene is formed. Examples of point cloud data stitching algorithms include RANSAC (Random Sample Consensus) and ICP (Iterated Closest Point). In some scenarios where computation is more favorable, since the optical axes of the radar unit and the camera unit are parallel to each other, the point cloud data scanned before and after the radar unit's rotation can be projected along the depth direction onto the aforementioned reference plane. Based on the edge information (corners, line segments, etc.) in the images captured before and after rotation, the projected point cloud images in the reference plane are stitched / registered, thereby reducing the computational load and accelerating the formation of images and a 3D point cloud panorama of the fire scene, as shown in Table 1.

[0090] Table 1. Examples of one method for generating panoramic images and point cloud data.

[0091]

[0092] In this embodiment of the invention, adjacent frames (with a rotational shooting sequence) can be two images taken before and after the camera unit rotates by a fixed angle, or they can be images taken by a depth-value camera. Adjacent frames can contain depth information, and feature points can also have depth features. They are not multiple images taken from the same angle (this can be used to improve image quality). In this embodiment, the depth (value) can be the horizontal distance between the radar unit and the scanned object, or between the camera unit and the photographed object.

[0093] In this fully automated scenario, since the positions of the camera unit and the radar unit are fixed, in terms of implementation, multiple calibrated acquisition positions can be taken from various viewpoints (where there is a mapping relationship between the point cloud data in the field of view and the corresponding objects in the image). The feature matching relationship between the radar point cloud data scanned by the radar unit (three-dimensional) radar coordinate system and the position points of the image captured by the infrared camera in the camera unit (two-dimensional) infrared vision coordinate system can be determined. This determined feature matching relationship is used as a mapping model. The mapping model may include a feature matching matrix (convolutional layer network) and / or coordinate transformation formula (which can be represented by a parameter matrix). This mapping model can be called the fusion mapping model of infrared vision and lidar.

[0094] In this embodiment of the invention, the method for determining the mapping model may include:

[0095] D1) Calibrate the radar coordinate system and image coordinate system, and determine the mapping model of the parameter matrix to be solved;

[0096] D2) Convert radar point cloud data into a two-dimensional point cloud image, and extract feature points from either the captured image or the two-dimensional point cloud image.

[0097] D3) Determine the feature points in the other that overlap with the feature points of either of the aforementioned ones;

[0098] D4) Determine the value of the parameter matrix to be solved by using the overlapping feature points.

[0099] Both the radar coordinate system and the image coordinate system can be represented by a world coordinate system (or another established reference system). The mapping model between the coordinate systems can be represented by a parameter matrix (and / or a system of equations). The radar point cloud data can be projected onto a specified plane along the depth direction. The projected image is a two-dimensional point cloud image, which has edge information, region information, spatial relationship information, and other features that match the captured image. The captured image can include infrared (panoramic) images and visible light (panoramic) images. An image mapping model can be determined between the infrared image and the visible light image. Whether feature points overlap can be identified using any feature matching algorithm. In some implementation-friendly scenarios, the SURF (Speeded-up robust features) feature matching algorithm has the advantage of rotational stability. Overlapping feature points can be used as the input and output of the mapping model in step D1), respectively, so that the numerical values ​​of the parameter matrix can be solved, making the mapping model usable in practice.

[0100] In some exemplary instances disclosed in this invention, the mapping model may include an image mapping model and a fusion mapping model, and the method for determining the mapping model may include:

[0101] D1-a) Calibrate the visible light image coordinate system and the infrared image coordinate system, and determine the image mapping model of the parameter matrix to be solved;

[0102] D2-a) Select two adjacent frames of images captured by each camera in the camera unit before and after rotating at a fixed angle, and extract SURF feature points in the two infrared images. The features of the two adjacent infrared images can provide a basis for feature matching for the same target object from different perspectives, avoiding incorrect matching caused by different objects having similar features in one frame.

[0103] D3-a) Using the SURF feature matching algorithm, overlapping feature points in the visible light image to be matched can be determined, and the RANSAC algorithm can be used to remove feature points that are incorrectly matched with SURF feature points.

[0104] D4-a) Then, based on the matching overlapping feature points, solve for the value of the parameter matrix to obtain the image mapping model, thereby completing the stitching of infrared and visible light images.

[0105] Regarding the use of this image mapping model, it can be used to match feature points with corresponding image features in one of the visible light images and infrared images, based on given feature points (or feature points extracted from a given local image or vector). For example, an image region in an infrared image can be identified as an image region in a visible light image with corresponding image features by using this image mapping model (extracting its feature points).

[0106] For the fusion mapping model between point cloud images and infrared images, the method for determining the mapping model may also include:

[0107] D1-b) Calibrate the radar coordinate system and, in conjunction with the calibrated infrared image coordinate system, determine the fusion mapping model of the parameter matrix to be solved;

[0108] D2-b) Select two adjacent frames of images acquired by the infrared camera in the camera unit before and after rotating at a fixed angle, and two frames of point cloud data scanned by the radar unit before and after rotating at a fixed angle. Convert the point cloud data into two-dimensional point cloud images and extract SURF feature points from the two infrared images. The features of the two adjacent infrared images can provide the basis for feature matching of the same target object from different perspectives, avoiding incorrect matching caused by similar features of different objects in one frame.

[0109] D3-b) Using a feature matching algorithm, the overlapping feature points in the two-dimensional point cloud image to be matched are determined, and the RANSAC algorithm is used to remove feature points that are incorrectly matched with SURF feature points.

[0110] D4-b) Then, based on the matched feature points, solve for the values ​​of the parameter matrix to obtain the fusion mapping model, thereby completing the stitching of the two-dimensional point cloud image and the infrared image.

[0111] Regarding the use of this fusion mapping model, it can be used to match feature points with corresponding image features in the other image based on given feature points in either a point cloud image or an infrared image (or feature points extracted from a given local image or vector). For example, an image region in an infrared image will be determined by this fusion mapping model (extracting its feature points) to determine an image region with corresponding image features in a two-dimensional point cloud image. This image region, located in the projection plane and the origin of the radar coordinate system, can determine a three-dimensional cone region in the radar coordinate system. If this image region has image features corresponding to the flame ROI, then the point cloud data in the three-dimensional cone region is the preliminary point cloud data.

[0112] It should be noted that, in this embodiment of the invention, the feature points in the point cloud image can describe the three-dimensional volume region segmented by the unit. Since the radar unit (and camera unit) is relatively far from the scanned object, the three-dimensional volume region segmented by the unit can be a three-dimensional cone region. That is, the three-dimensional point cloud image can be regarded as being composed of a large number of three-dimensional cone regions. A certain three-dimensional cone region can include the point cloud data scanned at a certain rotation angle or field of view. The feature points of this point cloud data correspond to the feature points of the image region in the infrared image. The radar point cloud data scanned by the radar unit can include local coordinate system coordinate points, identifiers, echo intensity values, world coordinate system coordinate points, etc. The mapping model can be integrated and compiled into one, having the functions corresponding to the aforementioned steps D1) to D4), and the order of using the functions can be adjusted according to actual product requirements. Considering the different hardware processing capabilities of the control unit, program design, and computation time requirements, other feature matching algorithms different from the SURF feature matching algorithm can be used, such as the scale-invariant feature transform (SIFT) algorithm. Understandably, compared to visible light images, using infrared images to establish mappings with visible light images and with point cloud images offers the advantage of less interference information. Here, "relatively far distance" can refer to, at least under acceptable cost conditions, the distance between a monocular or binocular camera and the object being photographed, a distance that makes it difficult for the monocular or binocular camera to determine the spatial coordinates of the object, while the radar unit of this embodiment can scan the object under these cost conditions.

[0113] The aforementioned mapping model can stitch together visible light images, infrared images, and point cloud images (via feature points). In some application scenarios, using this mapping model, based on any given image region (or feature points extracted from a given local image or vector), the corresponding image region or volume region among the three can be determined, thereby constructing a 360° panoramic infrared image, a 360° panoramic visible light image, and a 360° panoramic radar point cloud map of the scene environment, with a one-to-one correspondence. It can realize the conversion of any two-dimensional image region in the infrared visual coordinate system to the cross section and volume region in the radar coordinate system, as shown in Table 2.

[0114] Table 2 shows an exemplary implementation of the mapping model and the image correspondence used.

[0115]

[0116] In some application scenarios disclosed in this invention that are beneficial for simplifying calculations, such as Figure 4 A world coordinate system can be established with the boom rotation center O as the origin, and a radar coordinate system can be established with the radar unit scanning center O1(x1,y1,z1) as the origin, as follows: Figure 4 As shown, between adjacent scanned point cloud images, the horizontal adjustment angle of the fire monitor around the vertical direction is α. A radar coordinate system transformation model is established before and after the fire monitor rotates, and the three-dimensional coordinates of the radar point cloud data after rotation are transformed to the initial radar coordinate system O-XYZ:

[0117]

[0118] In the formula, P p (x p ,y p ,z p P is any three-dimensional coordinate point (not shown) in the radar point cloud data before rotation. i (x i ,y i ,z i () is the three-dimensional coordinate point (not shown) corresponding to the radar point cloud data after rotation.

[0119] In this embodiment of the invention, an image mapping relationship exists between the infrared panoramic image and the visible light panoramic image, which serve as inputs for ROI detection. This image mapping relationship can include the transformation relationship between the infrared image coordinate system and the visible light image coordinate system, and / or the synchronization relationship of the camera's optical axis rotation (whether the rotation is synchronized, and the rotation angle deviation). The image mapping relationship can be determined based on the camera selection and calibration coordinate system. For example, when using a visible light-infrared multi-functional camera, the optical axis rotation synchronization relationship can be selected to determine the image mapping relationship. When using two cameras, a visible light camera and an infrared camera, the transformation relationship between the infrared image coordinate system and the visible light image coordinate system, as well as the synchronization relationship of the camera's optical axis rotation, can be selected. Alternatively, any region in the infrared image coordinate system can be used to determine a corresponding region in the visible light image coordinate system through the image mapping relationship, and vice versa. To reduce interference, the flame region in the panoramic image can be detected based on visible light and visual fusion technology, and the flame ROI (Region of Interest) can be marked in the infrared panoramic image. In order to facilitate the creation of a bounding box for the flame ROI in the fire scene, the flame region in the infrared panoramic image can be mapped to the visible light panoramic image for marking.

[0120] In order to identify the combustion temperature T of various substances f and avoid ambient temperature T h (These effects can all be detected through image analysis, such as brightness temperature or color temperature detection), and a dynamic flame warning value T can be set to adapt to the material and environment. a :

[0121] T>T a ,T a =0.65T f +0.2T h

[0122] In the image, the ambient temperature T in the image region is greater than the warning value T. a At this time, the detection step A2 of the flame ROI can be initiated.

[0123] In step A2) above, determining the flame ROI in the captured image may include:

[0124] A201) Detect the flame features of the visible light panoramic image to obtain the first flame region;

[0125] A202) The infrared panoramic image is divided into background sections to determine suspected flame areas, and flame features of the suspected flame areas are detected to obtain a second flame area;

[0126] (A203) Determine the flame ROI based on the first flame region, the second flame region, and the image mapping relationship.

[0127] In an exemplary example of flame feature calculation disclosed in this invention, in step A201), a flame candidate region C1 (i.e., the first flame region) can be detected based on the flame's YCbCr space color features, morphological features, and motion features. The calculation rules for flame feature detection in visible light panoramic images may include:

[0128] Color rule: |(x,y)-Y means |≥ε1 and (x, y)-Cb(x, y)|≥ε2

[0129] In this formula, Y(x,y), Cr(x,y), and Cb(x,y) represent the Y, Cr, and Cb component values ​​of the (rasterized) image at pixel (x,y). means ε1 and ε2 represent the mean of the Y component of the image, and ε1 and ε2 are threshold values.

[0130] Morphological rules: and and

[0131] In this formula, Q(x,y) represents the high-frequency energy of the Haar wavelet, and S and L represent the area and perimeter of the suspected flame region, respectively. T Let ε3, ε4, and ε5 represent the area of ​​the minimum bounding rectangle, ε5 be the threshold values, I represent the suspected flame region to be screened, and (,y)∈I means that the pixel (,y) belongs to the suspected flame region I to be screened.

[0132] Motion characteristic rules:

[0133] In this formula, f(x,y) represents the pixel gray value at (x,y), N represents the number of SURF feature point pairs between two adjacent flame images, and x1 i y1 i x2 represents the coordinates of the i-th pair of feature points in image 1. i y2 i Let x represent the coordinates of the i-th pair of feature points in image 2, S represent the area of ​​the suspected flame region to be screened, and n be a coefficient. In this example of flame feature calculation, x and y are not used in subsequent embodiments.

[0134] The aforementioned step A202) may include: traversing each pixel of the infrared panoramic image; when the color temperature (or the actual temperature represented by the color temperature) corresponding to the pixel is lower than or equal to a set threshold, the pixel is classified as a background region; when the color temperature corresponding to the pixel is higher than the set threshold, the pixel is classified as a suspected flame region C2. Morphological and motion features are calculated for the suspected flame region C2. The calculation rules can still use the aforementioned morphological and motion feature rules. False flame regions that do not conform to the calculation rules are eliminated to obtain the suspected flame region C2 (second flame region) in the infrared panoramic image.

[0135] As described above regarding position calibration and coordinate system establishment, since the visible light camera and infrared camera are fixedly installed, the coordinate system of the infrared camera and the coordinate system of the visible light camera can be calibrated based on their relative positions. Furthermore, the intrinsic and extrinsic parameters of the cameras can be obtained, allowing for the establishment of a mapping model between the infrared image and the visible light image. In some application scenarios, in step A203), to reduce interference, the flame region selected from the visible light panoramic image can be mapped to the infrared panoramic image. Step A203) may include:

[0136] A241) Map the first flame region onto the infrared panoramic image;

[0137] A242) Remove flame regions in the overlapping region whose color temperature is less than a specified threshold;

[0138] A243) The flame ROI is obtained by removing the overlapping area from the infrared panoramic image.

[0139] Specifically, the flame region C1 in the visible light panoramic image can be mapped onto the infrared panoramic image to obtain region C1' (the mapped first flame region). The flame regions C2 and C1' in the infrared panoramic image are then fused (intersected) to obtain the overlapping region. Pixels with a color temperature lower than a set threshold in the overlapping region are removed. The flame region C, i.e., the flame ROI, is finally generated in the infrared panoramic image after removal.

[0140] In step A2) above, estimating the depth value of the flame ROI based on images from adjacent frames may include:

[0141] A205) Determine the center point containing the target region between two adjacent frames of the image, and calculate the pixel deviation value between the two center points. The target region is the flame ROI in the infrared panoramic image.

[0142] (A206) ​​Based on the rotation angle of the images of the two adjacent frames, the camera's intrinsic parameters, and the pixel deviation value, estimate the depth value of any center point. The rotation angle can be the rotation angle of the optical axis of the camera unit, and the arbitrary center point can be the center point of the target area or the center point of the flame of the target burning object for coarse positioning.

[0143] In this system, two adjacent frames can be infrared images, and the flame ROI detected by the infrared panoramic image can be used as the target region. The bounding box of the ROI of the same burning object between the two adjacent infrared images can have an offset. The camera's intrinsic parameters, such as the imaging point and focus point, and the center point of the bounding box, can be connected in three-dimensional space, forming a similar triangle relationship (or geometric relationship, with the rotation angle also known). Based on the similar triangle relationship and the offset, the coarse localization depth value of the flame ROI of the target burning object can be determined. Therefore, two adjacent infrared images (unstitched) containing the target region can be extracted to determine the flame ROI or target region. The offset is represented by pixels (each pixel represents a unit length), and the center point of the bounding box of the target region in the two images is determined. The horizontal pixel deviation value between the center points of the two bounding boxes (the number of pixels in the horizontal direction between the two center points) is calculated. Based on the horizontal pixel deviation value, the camera rotation angle α, and the camera intrinsic parameters, the depth value h of the target region is estimated. This depth value h can be used as the depth value of the flame ROI, which can be used as the aforementioned first depth value or coarse localization depth value.

[0144] Because the high-temperature plasma of a flame interferes with electromagnetic waves through absorption and refraction, it is difficult for lidar to scan effective flame point cloud data. However, the actual burning target is close to the fire source. Therefore, the flame ROI area can be expanded to include the burning target within the expanded ROI area. In step A3) above, extending (expanding or adding, etc.) the flame ROI can include:

[0145] A301) Expand the size of the flame ROI to the specified area size;

[0146] A302) Increase the area size of the flame ROI to a specified value.

[0147] The region size can be the area, the region dimensions can be the side length, diagonal, etc., and the region size increment Δs and region dimension increment Δl can be configured.

[0148] In some application scenarios, in embodiments of the present invention, image feature information can be used first, followed by depth information, which has the advantage of accuracy. Step A3) described above may further include:

[0149] A303a) Based on the mapping model and the extended flame ROI, determine the three-dimensional cone region;

[0150] A304a) Using the neighborhood of the first depth value, the three-dimensional cone region is truncated to obtain the target spatial region.

[0151] In some application scenarios, a three-dimensional space (containing radar point cloud data) can be divided into a finite number of rectangular boxes on a specified projection plane perpendicular to the depth direction. The spatial region enclosed by the lines connecting the vertices of each rectangle to the origin is a three-dimensional cone region. In this embodiment of the invention, the three-dimensional cone region can be regarded as the basic unit for dividing the space containing radar point cloud data. A mapping model based on the extended image features of the flame ROI can be used to uniquely determine a three-dimensional cone region with the image features among all basic units in the entire space. Figure 5 A fusion mapping model combining infrared panoramic images and radar point cloud data (point cloud images) can be used. The extended flame ROI is taken as the image region and input into the mapping model. This results in a three-dimensional cone region, which contains preliminary radar point cloud data. This three-dimensional cone region can be called a Cube region. Figure 5 The mid-section region is a triangular region. A three-dimensional cone region can include solid regions such as conical regions or polyhedral regions. For example, ... Figure 6 If the ROI is selected as a rectangle, then the three-dimensional cone region can be a tetrahedral cone region with the coordinate points of the radar unit as vertices (e.g., Figure 6 The Cube region shown, or a cone region with the radar unit as its vertex, is projected onto a specified plane (such as a horizontal plane). The maximum angle at which the optical axis of the radar unit scans around the coordinate point of the radar unit to cover this projected region can be equal to the maximum angle at which the optical axis of the infrared camera in the camera unit scans around the coordinate point of the infrared camera to cover the flame ROI (such as the location point corresponding to the frame vertex). Figure 6 Mid-angle γ.

[0152] Based on the aforementioned three-dimensional cone region, in step A4), to further expedite the determination of the location data of the target burning object, the three-dimensional cone region can be truncated. Step A304a) may include:

[0153] A341a) Determine the neighborhood boundary value of the first depth value, wherein the neighborhood has a specified threshold interval size;

[0154] A342a) Determine the cross section with a depth value of the left neighbor boundary value and the cross section with a depth value of the right neighbor boundary value in the three-dimensional cone region;

[0155] A343a) The volume region between the two cross sections in the three-dimensional cone region is taken as the target space region.

[0156] For interception, such as Figure 7 Based on the estimated depth value h of the flame ROI, the [h-Δh, h+Δh] portion of the Cube region can be extracted. Figure 7 The cross-sectional area is a quadrilateral volume region. The point cloud data within the intercepted volume region is retained and can be called the Cube intercept region. This Cube intercept region will contain the point cloud data of the target burning object. At this time, for segmentation, nearest neighbor search and Euclidean clustering can be performed on the Cube intercept region to segment out the point cloud data of the target burning object.

[0157] In embodiments of the present invention, in some exemplary application scenarios, depth information can be used first, followed by image feature information, which has the advantage of processing efficiency. The aforementioned step A3) may further include:

[0158] A303b) Determine a three-dimensional annular region based on the neighborhood of the first depth value;

[0159] A304b) Based on the mapping model and the extended flame ROI, the three-dimensional annular region is extracted to obtain the target spatial region.

[0160] In step A303b), a large amount of radar point cloud data unrelated to the flame area and the burning target can be filtered out. This significantly reduces the size of the radar point cloud data involved in the mapping model calculation within the three-dimensional annular volume region in A304b), allowing the mapping model to more quickly determine the intercepted volume region based on the extended flame ROI. For example, Figure 8 Step A304b) may include:

[0161] A341b) Initialize a basic unit of fixed size, and use the basic unit to copy radar point cloud data within a specified angle range in the three-dimensional annular region. The size and shape of the basic unit can be consistent with the size and shape of a segment of the volume region extracted from the three-dimensional annular region.

[0162] A342b) The mapping model can be used to determine whether the radar point cloud data in the basic unit has similar or consistent features in the depth direction with the extended flame ROI. If so, step A343b) is executed. If not, the basic unit is cleared, and the radar point cloud data in the next specified angle range in the three-dimensional annular region is copied using the basic unit. The radar point cloud data in the basic unit is determined to have similar or consistent image features in the depth direction with the extended flame ROI. This process is iterated and the entire three-dimensional annular region is scanned clockwise or counterclockwise from the specified angle.

[0163] A343b) Determine the volume region intercepted from the three-dimensional annular volume region corresponding to the basic unit at this time. This intercepted volume region is the target space region.

[0164] For point cloud data within the target spatial region, clustering algorithms, support vector machines, or convolutional neural networks (CNNs) can be used to identify the burning target and its point cloud data. Finally, the location data can be determined using this point cloud data. The training samples for the machine learning algorithms can collect point cloud data of burning objects, enabling the algorithms to independently classify the feature recognition capabilities of the point cloud data of burning targets.

[0165] The location data may include a coordinate point in the point cloud data of the burning target, such as the centroid coordinate point, geometric center point, etc.

[0166] The location data may also include a set of coordinate points in the point cloud data of the burning target, such as the set of all coordinate points (belonging to the point cloud data) determined on a straight line between one specified coordinate point and another specified coordinate point in the point cloud data, or the set of all coordinate points (belonging to the point cloud data) determined on a plane determined by three specified coordinate points in the point cloud data.

[0167] The location data can also include a location range defined by continuous intervals. These continuous intervals can be defined by two coordinate points in the point cloud data of the burning target along a straight line. For example, in the world coordinate system, the location range of the first coordinate point (P1, Q1, R1) and the second coordinate point (P2, Q2, R2) in the point cloud data can be specified as follows: a continuous interval in the X-axis direction, i.e., x ∈ [P1, P2]; y belongs to a continuous interval in the point cloud data consisting of the minimum and maximum coordinate values ​​in the Y-axis direction; and z belongs to a continuous interval in the point cloud data consisting of the minimum and maximum coordinate values ​​in the Z-axis direction. In other scenarios, y and z can be similarly restricted to the continuous intervals of [Q1, Q2] and [R1, R2], respectively.

[0168] The location data of the detected burning object is located within the three-dimensional scatter envelope of the point cloud data. It can be the coordinate points / sets of the point cloud data itself, or the selected coordinate points or continuous intervals between coordinate points within the three-dimensional scatter envelope. It can adapt to the characteristics of fire truck products and can adopt specific data formats of location data. In some actual fire scenes, the combustion pattern and surface characteristics of the burning object, as well as the selected fire extinguishing operation method and on-site environmental conditions, can be considered to use one or more types of location data to improve the accuracy and effectiveness of automatic fire extinguishing.

[0169] In the first exemplary application scenario disclosed in this embodiment of the invention, step A4) may include:

[0170] A401a) Cluster the target point cloud data within the intercepted volume region to obtain the point cloud data of the target burning object. The intercepted volume region may contain objects that are not the target burning object as well as point clouds of the target burning object. The clustering algorithm can distinguish the object from the target burning object (determine the classification) and segment the point cloud data within the intercepted volume region that belongs to the target burning object.

[0171] A402a) Based on the point cloud data of the target burning object, obtain the location data of the target burning object.

[0172] In the second exemplary application scenario disclosed in this embodiment of the invention, the aforementioned neighborhood can be a selected continuous range or a discrete set of points. To facilitate the rapid determination of the point cloud data of the target burning object, multiple points with similar depth values ​​are selected at fixed intervals along the depth direction in the neighborhood of the first depth value. For example, 10 to 50 points are selected in the left neighborhood and 10 to 50 points are selected in the right neighborhood. The point cloud data of the section with the first depth value and multiple sections with similar depth values ​​within the extracted volume region still retain the characteristics of the target burning object along the depth direction. Therefore, within the three-dimensional cone region, sections with the first depth value and multiple sections with similar depth values ​​can be determined. Step A4) may also include:

[0173] A401b) Cluster the target point cloud data on multiple determined cross sections to obtain the target burning object and its point cloud data. The clustering algorithm can be either the nearest neighbor or the mean clustering algorithm.

[0174] A402b) Based on the point cloud data of the target burning object, obtain the location data of the target burning object.

[0175] In the third exemplary application scenario disclosed in the embodiments of the present invention, a neighborhood of a first depth value and a plurality of neighborhoods with similar depth values ​​can be determined. Within the three-dimensional cone region, a micro-volume region corresponding to each neighborhood is extracted. Based on the first and second exemplary application scenarios, step A4) may also include:

[0176] A401c) Clustering the target point cloud data within the captured multi-segment micro-body region, segmenting it to obtain the target burning object and its point cloud data;

[0177] A402c) Based on the point cloud data of the target burning object, the location data of the target burning object is obtained.

[0178] Specifically, for points at the first depth value and multiple points at similar depth values, neighborhoods of the same threshold interval size (e.g., radius values ​​of Δh*5%) are selected. The aforementioned Cube-trimmed region is further truncated. The point cloud data within these truncated micro-regions still retain characteristics of the burning target along the depth direction. Then, clustering or machine learning algorithms are applied to the point cloud data within these truncated micro-regions to classify the burning target and its location. This method of processing discrete cross-sections and micro-regions significantly improves the efficiency of point cloud data processing using high-resolution LiDAR, thereby accelerating the determination of the burning target's location.

[0179] This invention also provides an automatic fire extinguishing method for fire-fighting equipment under the same inventive concept. The fire-fighting equipment includes a control unit and a fire monitor. The automatic fire extinguishing method may include:

[0180] B1) The aforementioned method for detecting the location of a burning object is executed by the control unit to determine the location data of the target burning object;

[0181] B2) The control unit controls the fire monitor to rotate and aim at the designated location of the target burning object based on the location data, and performs fire extinguishing operation.

[0182] In the aforementioned embodiments, the location data can be selected in a way that matches the specific method of the fire extinguishing operation. For example, the fire extinguishing operation can be a static spray aimed at a coordinate point or a dynamic sweep within a limited area.

[0183] In an exemplary static fire extinguishing operation scenario disclosed in an embodiment of the present invention, the location data can be a coordinate point, which serves as the designated location of the target burning object.

[0184] Point cloud data of the target burning object can be calculated using methods such as local averaging, weighted averaging, or geometric center calculation to obtain the second depth value of the point cloud data. This allows the determination (or the generation of coordinate points or the use of point cloud data with this depth value) of the three-dimensional coordinates P(x1,y2,z2) of the fire source. The fire source can be considered as an identifiable, effective, and operational jet strike point for fire extinguishing. The jet for long-distance fire extinguishing is not a diffused umbrella-shaped jet, but a slender jet. However, after long-distance projection, it will diverge into a diffused range. Once a fire source is found, long-distance automated fire extinguishing can be performed.

[0185] In this embodiment of the invention, step B2) may include:

[0186] B201a) The control unit calculates the deflection angle of the fire monitor's axis relative to the fire source point;

[0187] B202a) The control unit controls the motion mechanism to drive the fire monitor to rotate by the deflection angle. The control can be achieved by the aforementioned control unit sending a signal carrying the fire source point or deflection angle information to the motor controller or the controller of the motion mechanism.

[0188] In the aforementioned embodiments, the rotation of the fire monitor and the rotation in step B202a) can both be achieved by a motion mechanism. This motion mechanism can be a rotatable (e.g., 360-degree) boom or a steering motor for the fire monitor. Alternatively, the boom can rotate the fire monitor, allowing the camera unit and radar unit to simultaneously capture and scan. Then, the steering motor of the fire monitor aligns the fire source point. In some application scenarios, the fire monitor can be driven by motors 1 and 2 to achieve vertical and horizontal rotation, respectively. The fire source point is then converted to the radar coordinate system to obtain coordinate point P(x1, y1, z1), and the fixed installation position deviation P(x1, y1, z1) of the radar unit and fire monitor is determined. p1 ,y p1 ,z p1 Calculate the horizontal deflection angle α' and vertical deflection angle β of the fire monitor relative to the coordinate point P(x1,y1,z1):

[0189]

[0190] Motor 2 controls the fire monitor to rotate α', and motor 1 controls the fire monitor to rotate β, thereby enabling the fire monitor to aim at the fire source. Finally, it can perform fire extinguishing operations such as opening electric valves and starting liquid pumps, so that the fire monitor can spray fire extinguishing agent flow to accurately hit the target and realize fully automatic fire extinguishing operation.

[0191] In an exemplary dynamic fire extinguishing operation scenario disclosed in this embodiment of the invention, the location data can be a location range determined by a continuous interval, which can be an angular range, serving as the designated location of the target burning object. In some cases, the point cloud data of the target burning object can be calculated to obtain a second depth value of the point cloud data, determining the three-dimensional coordinates P(x1,y2,z2) of the fire source point, and a spherical region with the three-dimensional coordinates P(x1,y2,z2) as its center and a radius of a specified value r is taken; the location range can be a spherical region. For ease of fully automated control, step B2) may also include:

[0192] B201b) The control unit determines the tangents of the spherical region passing through the coordinates of the fire monitor and the cross section passing through the two tangents. The straight line formed by the two tangents can pass through the centroid of the point cloud data or the distance between the centroid and the straight line is less than a specified value.

[0193] B202b) By controlling the motion mechanism through the control unit, the fire monitor is moved, so that the range of movement of the fire monitor's shaft is limited to the section passing through the two tangents and is located within the angle range formed by the two tangents. This can be regarded as the identifiable, effective, and fire-extinguishing jet strike range.

[0194] Finally, it can perform fire extinguishing operations such as opening electric valves and starting liquid pumps, so that the fire monitor can spray fire extinguishing agent. At this time, the fire monitor can be in continuous motion and can accurately sweep the burning objects in a row at a distance, realizing fully automatic fire extinguishing operation.

[0195] In other cases, the point cloud data of the burning target can be calculated to obtain the coordinates of the maximum and minimum values ​​of the point cloud data on each coordinate axis. Step B2) may also include:

[0196] B201c) The control unit determines the coordinate interval from the minimum to the maximum value in the X-axis direction, selects 10% to 50% of the length of this coordinate interval, which is a distance in the X-axis direction, and determines two coordinate points in the point cloud data of the target burning object that have at least this distance in the X-axis direction based on the selected distance. At this time, the position range can be specified as: a continuous interval in the X-axis direction, that is, x belongs to the continuous interval of the coordinate values ​​in the X-axis direction of the two coordinate points, y belongs to the continuous interval in the point cloud data consisting of the minimum and maximum coordinate values ​​in the Y-axis direction, and z belongs to the continuous interval in the point cloud data consisting of the minimum and maximum coordinate values ​​in the Z-axis direction.

[0197] B202c) By controlling the motion mechanism through the control unit to move the fire monitor along a specified trajectory or freely, the range of movement of the fire monitor's shaft is limited to the position range specified in step B201c), which can be regarded as the identifiable, effective, and fire-extinguishing jet strike range.

[0198] Finally, fire extinguishing operations such as opening electric valves and starting liquid pumps can be performed, causing the fire monitor to spray extinguishing agent. At this time, the fire monitor can be in continuous motion, and can accurately locate and cover targets that are burning at a distance and with a large area of ​​fire and large amount of combustible material, so as to realize fully automatic fire extinguishing operation.

[0199] In this embodiment of the invention, the control unit can select the specific data format of the aforementioned location data based on the simple geometric features of the point cloud data of the target burning object and / or the static or dynamic working mode of the fire extinguishing operation. The simple geometric features may include the maximum distance between coordinate points in the depth direction, the maximum distance between coordinate points in any coordinate axis direction, and the distance between coordinate points in multiple directions. It is understood that the disclosed values ​​in this embodiment of the invention are exemplary and not intended as limiting implementation methods. Adjustments and different choices are possible to suit product characteristics and actual application environments.

[0200] The automatic fire extinguishing method of this invention can also be applied to a data acquisition unit, which includes a camera unit and a radar unit. The automatic fire extinguishing method may further include:

[0201] C1) Rotate the fire monitor to a specified angle, capture visible light and infrared images of the fire scene at the current angle using the camera unit, and simultaneously scan the scene at the current angle using the radar unit;

[0202] C2) Rotate the fire monitor to the next specified angle, capture visible light and infrared images of the fire scene at the current angle through the camera unit, and simultaneously scan the scene at the current angle of the fire scene through the radar unit;

[0203] The camera unit and the radar unit are both arranged on the fire monitor. The fire monitor is driven to rotate by a motor. Visible light images and infrared images captured from various angles are used to form visible light panoramic images and infrared panoramic images, respectively. Data from scanning the fire scene from various angles are used to form radar point cloud data.

[0204] The present invention overcomes the problem that radar has difficulty scanning flames (high-temperature plasma in flames will interfere with electromagnetic waves through absorption, refraction and other means). By using point clouds in the body area or taking point clouds from multiple cross sections, the point cloud of the burning object can be determined. The fire monitor is then controlled to aim at the target burning object, realizing automated fire extinguishing operation. It also realizes long-distance automated fire extinguishing and firefighting equipment.

[0205] In this embodiment of the invention, a cut-off volume region can be determined within a three-dimensional cone region using the neighborhood of depth values. Clustering or model detection is then performed within this cut-off volume region to segment the point cloud data of the target burning object. Alternatively, multiple cross-sections within the three-dimensional cone region can be obtained using depth values, and the point cloud data of the target burning object can be segmented from the point clouds on these cross-sections. Simultaneously captured visible light and infrared images, as well as scanned radar point cloud data, can be used, which will facilitate the establishment of the mapping model and simplify calculations. The size or dimensions of the flame ROI can be specifically changed so that the extended ROI includes the target burning object. To adapt to application needs and simplify calculations, a specific morphology of the three-dimensional cone region can be specified as the default generated shape, such as a conical region or a polyhedral region. The neighborhood boundary values ​​of depth values ​​can be used to obtain cross-sections, and the volume region between the cross-sections is used as the cut-off volume region. Optimized algorithms can be selected, such as nearest neighbor search, Euclidean clustering, and local averaging. The fire monitor can be configured to be driven by a motor to rotate to the calculated angle, achieving automated fire source aiming.

[0206] This invention employs infrared and visible light cameras to detect flames during the dynamic movement of a fire monitor, solving the problem of flame detection in situations such as rotating equipment and dense smoke, thus extending the applicability of fire detection technology to outdoor moving scenarios. This invention integrates infrared vision, visible light vision, and radar point cloud data, combined with flame characteristics, to achieve 360° all-around monitoring, fire source detection, and high-precision positioning of virtual flames at long distances. It also solves the key challenge of short visual positioning distance (within 15m) in the field of flame detection. This invention uses infrared cameras, visible light cameras, and lidar for scene scanning, flame detection, and positioning, thereby controlling the fire monitor to aim at and strike the burning object, achieving fully automated fire truck extinguishing without any human intervention, reducing fire extinguishing efficiency, and ensuring the safety of firefighters.

[0207] Example 2

[0208] This invention provides a system for detecting the location of a burning object, which may include:

[0209] The acquisition module is used to acquire images of the fire scene and scanned radar point cloud data, wherein the images and the radar point cloud data are associated through a mapping model.

[0210] An image processing module is used to determine the flame ROI in the captured image and a first depth value of the flame ROI;

[0211] The point cloud processing module is used to extend the flame ROI and determine the target spatial region based on the mapping model, the extended flame ROI, and the first depth value.

[0212] The determination module is used to identify the target burning object and the location data of the target burning object based on the target point cloud data, wherein the target point cloud data is radar point cloud data located in the target spatial region.

[0213] Specifically, the flame ROI is extended, and based on the mapping model, the extended flame ROI, and the first depth value, the target spatial region is determined, including:

[0214] Based on the mapping model and the extended flame ROI, the three-dimensional cone region is determined;

[0215] Using the neighborhood of the first depth value, the three-dimensional cone region is truncated to obtain the target spatial region.

[0216] Specifically, the flame ROI is extended, and based on the mapping model, the extended flame ROI, and the first depth value, the target spatial region is determined, including:

[0217] Based on the neighborhood of the first depth value, a three-dimensional annular region is determined;

[0218] Based on the mapping model and the extended flame ROI, the three-dimensional annular region is extracted to obtain the target spatial region.

[0219] Specifically, the captured images include infrared panoramic images and visible light panoramic images, and there is an image mapping relationship between the infrared panoramic images and the visible light panoramic images;

[0220] The process of determining the flame ROI in the captured image includes:

[0221] The flame features of the visible light panoramic image are detected to obtain the first flame region;

[0222] The infrared panoramic image is divided into background sections to identify suspected flame areas. Flame features in the suspected flame areas are then detected to obtain a second flame area.

[0223] The flame ROI is determined based on the first flame region, the second flame region, and the image mapping relationship.

[0224] Specifically, the flame ROI is determined based on the first flame region, the second flame region, and the image mapping relationship, including:

[0225] Map the first flame region onto the infrared panoramic image;

[0226] Determine the overlapping area between the second flame region and the mapped first flame region;

[0227] Remove regions with a color temperature lower than a specified threshold from the overlapping regions to obtain the flame ROI.

[0228] Specifically, determining the first depth value of the flame ROI includes:

[0229] Two center points are determined between two adjacent frames of an image containing the target region, and the pixel deviation value between the two center points is calculated. The target region is the flame ROI in the infrared panoramic image.

[0230] Based on the rotation angle of the images captured in the two adjacent frames, the camera intrinsic parameters, and the pixel deviation value, the depth value of any center point is determined.

[0231] Specifically, using the neighborhood of the first depth value, the three-dimensional cone region is truncated to obtain the target spatial region, including:

[0232] Determine the neighborhood boundary value of the first depth value, wherein the neighborhood has a specified threshold interval size;

[0233] Determine the cross section with a depth value of the left neighbor boundary value and the cross section with a depth value of the right neighbor boundary value in the three-dimensional cone region;

[0234] The volume region between the two cross sections in the three-dimensional cone region is taken as the target space region.

[0235] Specifically, based on target point cloud data, identifying the target burning object and its location data includes:

[0236] Clustering is performed on the target point cloud data to obtain the target burning object and its location data.

[0237] Specifically, the methods for determining the mapping model include:

[0238] The radar coordinate system and image coordinate system are calibrated, and the mapping model of the parameter matrix to be solved is determined.

[0239] The radar point cloud data is converted into a two-dimensional point cloud image, and the overlapping feature points in the captured image and the two-dimensional point cloud image are extracted.

[0240] The numerical values ​​of the parameter matrix to be solved are determined by using the overlapping feature points.

[0241] This invention also provides an automatic fire extinguishing system for fire-fighting equipment, the fire-fighting equipment including a control unit and a fire monitor, the automatic fire extinguishing system comprising:

[0242] The control module is used to execute the aforementioned method for detecting the location of a burning object through the control unit to determine the location data of the target burning object;

[0243] The control module is used to control the fire monitor to rotate and aim at the designated location of the target burning object based on the location data through the control unit, and to perform fire extinguishing operation.

[0244] Specifically, the fire-fighting equipment also includes a camera unit and a radar unit, both of which are located on the fire monitor. The automatic fire extinguishing system also includes:

[0245] The data acquisition module is used to rotate the fire monitor to a specified angle through the control unit, capture visible light and infrared images of the fire scene through the camera unit, and simultaneously scan the fire scene through the radar unit;

[0246] The data acquisition module is used to rotate the fire monitor to the next specified angle through the control unit, capture visible light and infrared images of the fire scene through the camera unit, and simultaneously scan the fire scene through the radar unit;

[0247] The visible light images and infrared images captured from various angles are used to form visible light panoramic images and infrared panoramic images, respectively, while the data from scanning the fire scene from various angles are used to form radar point cloud data.

[0248] This invention provides a method, device, and system for 360° all-around fire source detection and location in fire truck rescue scenarios using infrared cameras, visible light cameras, and lidar. It also enables automatic control of the fire monitor for precise aiming and striking of the target fire source, achieving full automation of fire truck firefighting. The invention utilizes the fire monitor to drive the lidar, infrared camera, and visible light camera in a 360° rotational scan of the work scene, then stitches the images together to form a 360° panoramic vision and point cloud map, achieving comprehensive monitoring of the on-site environment without human intervention, such as initial target information input or fire monitor adjustment. Furthermore, by integrating the radar and camera units into the fire monitor, it eliminates the need for gimbals, attitude sensors, and other equipment, resulting in a simpler system structure. This invention integrates flame temperature, color, shape, high-frequency energy, and dynamic characteristics for comprehensive detection of the flame area, calculating the changes in flame intensity before and after the fire monitor rotation. This invention achieves automatic flame detection across the entire field environment by compensating for flame center offset and camera rotation. It extracts infrared scan images from flame targets detected in panoramic images, calculates flame center offset, fire monitor rotation, and camera intrinsic parameters, enabling coarse flame localization based on monocular infrared vision. Furthermore, based on information fusion, it accurately locates virtual flames at long distances, expands the flame ROI to include the target burning object, and then maps the two-dimensional ROI to a cube region of the point cloud. The target cube range is constrained according to the coarse flame localization distance h, thus achieving precise flame localization. Finally, this invention uses flame spatial position detection to calculate the horizontal and vertical rotation of the fire monitor, automatically controlling the fire monitor to aim and strike the target burning object, improving the intelligence level of the fire truck system and ensuring the personal safety of firefighters.

[0249] Example 3

[0250] The embodiments of this invention belong to the same inventive concept as embodiments 1 and 2. The embodiments of this invention provide the fire truck in embodiment 1, which may include:

[0251] The fire monitor is driven by an electric motor and can also be driven by the boom.

[0252] Both the camera unit and the radar unit are arranged on the fire monitor and rotate synchronously with the fire monitor.

[0253] The control unit is used to execute the method in Example 1.

[0254] In one exemplary application scenario disclosed in this invention, such as Figure 9 The control unit can be specifically used for:

[0255] K1) obtains data collected by the camera unit and radar unit, and controls the fire monitor to rotate horizontally by an angle α;

[0256] K2) Determine whether the 360-degree scan has been completed. If not, return to step K1); otherwise, continue to step K3.

[0257] K3) Match and stitch adjacent infrared and visible light images to obtain infrared panoramic images and visible light panoramic images of the fire scene, respectively;

[0258] K4) Based on the pose relationship between the radar coordinate system and the camera visual coordinate system, establish a visual-radar fusion model;

[0259] K5) Matches and stitches radar point cloud data to obtain a panoramic radar point cloud image, and establishes a mapping relationship between the infrared panoramic image and the point cloud panoramic image based on the vision-radar fusion model.

[0260] K6) Detect the flame region C1 in the visible light panoramic image based on the flame color, shape, and motion characteristics;

[0261] K7) Detect the flame region C2 in the infrared panoramic image based on the flame temperature, shape, and motion characteristics;

[0262] K8) calibrates the mapping model between visible light and infrared vision, maps the flame region C1 to the infrared panoramic image to obtain the flame region C1', and filters out pseudo flame regions where the overlap between the flame region C2 and the flame region C1' is less than a set threshold.

[0263] K9) Extract the edge and center point information of the flame ROI, extract two adjacent infrared scan images containing the target area in the panoramic image, calculate the horizontal pixel deviation of the target flame center in the two images, and preliminarily calculate the depth information of the target based on the rotation angle α corresponding to the image.

[0264] K10) Extract the corresponding Cube region from the point cloud map based on the 2D-3D mapping model;

[0265] K11) Based on the preliminary calculated and coarsely located depth information, the range of the Cube region is further narrowed to obtain the Cube interception region;

[0266] K12) performs nearest neighbor search and clustering segmentation on the point cloud data within the Cube interception area to obtain the point cloud of the target flame;

[0267] K13) Calculates the three-dimensional coordinates of the target point cloud data based on the local averaging method, and uses them as the spatial location points of the flame target;

[0268] K14) Based on the fixed installation position deviation of the radar unit and the fire monitor, the relative positional relationship between the fire monitor and the target fire source is calculated by subtracting the installation position deviation from the flame coordinates in the radar coordinate system.

[0269] K15) Based on the positional deviations of the fire source point and the muzzle in the horizontal and vertical directions, as well as the distance between the fire source point and the muzzle, the horizontal and elevation angle deviations of the fire monitor are calculated.

[0270] K16) sends the calculated horizontal and vertical angle deviation values ​​to the fire truck controller, which controls the fire monitor to rotate the corresponding angle, so as to achieve precise aiming of the fire monitor at the fire source.

[0271] The K17 fire truck controller controls the water pump and electric valves to open, and the water jet accurately hits the target fire source to complete the fully automatic fire extinguishing operation.

[0272] The aforementioned selection of fire trucks can also include aerial ladder / work platform fire trucks, ladder trucks, and aerial spray fire trucks. For example... Figure 10 The fire truck may include a vehicle body, a boom-type lifting arm, and a working platform. The working platform may be equipped with the aforementioned firefighting equipment. The fire monitor is equipped with an infrared fire camera, a visible light camera, and a lidar. Alternatively, the fire monitor's casing can be modified (e.g., by incorporating a mounting cavity and a card interface, with the cavity opening facing the same direction as the muzzle) to integrate the two cameras and the lidar into a single fire monitor. The working platform may also be equipped with a wireless bridge transmitter connected to the infrared fire camera, visible light camera, and lidar respectively. The vehicle body may carry multiple hardware devices, including a wireless bridge receiver, a control unit, and a touchscreen. The control unit may include an industrial computer and a controller. The industrial computer can receive data transmitted by the wireless bridge transmitter via the wireless bridge receiver and executes the method described in Example 1. The controller can connect to the industrial computer via a USB-CAN bus converter. Based on data output by the industrial computer (e.g., the location data of the target burning object), the controller can control the motor to rotate the fire monitor and operate the firefighting electrical equipment. It is understandable that the specific implementation of the control unit may be based on considerations such as the product integration level and the computing and instruction processing capabilities of the device, and may employ hardware options different from industrial control computers and controllers, such as industrial control computers and integrated circuit boards with system-on-a-chip chips, to implement the method in the aforementioned embodiment 1.

[0273] This fire truck can perform flame detection based on visible light, infrared vision, and lidar; and based on a multi-information fusion flame positioning module, it can accurately control the horizontal and vertical angles of the fire monitor.

[0274] First, the fire truck can use the fire monitor motor to drive the camera unit and radar unit to rotate 360 ​​degrees horizontally to establish the mapping relationship between the infrared panoramic image and the point cloud panoramic image of the fire scene.

[0275] Then, the fire truck can use visible light and infrared data fusion technology to detect the flame ROI in the infrared panoramic image and then fuse radar point cloud data to locate the target fire source.

[0276] Finally, the fire truck can calculate the elevation and horizontal angle control parameters of the fire monitor based on the target's three-dimensional spatial position and the equipment's installation posture, automatically controlling the fire monitor to quickly and accurately track, aim, and strike the target fire source. In some typical fire truck examples disclosed in this invention, the fire truck can quickly complete preparations for fire extinguishing operations such as flame scanning and fire monitor aiming at the fire source within 15 seconds. In some application scenarios, the fire truck can also control the electrical equipment on the fire truck used for fire extinguishing operations based on wind speed, monitor outlet pressure, and the distance from the nozzle to the fire source, enabling the jet to more accurately hit the fire source.

[0277] This invention also provides an electronic device, which may include:

[0278] At least one processor;

[0279] A memory connected to the at least one processor;

[0280] The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the method of Embodiment 1 by executing the instructions stored in the memory.

[0281] This invention also provides a machine-readable storage medium storing machine instructions that, when executed on a machine, cause the machine to perform the method of Embodiment 1.

[0282] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0283] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0284] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may be non-transient and may include various media capable of storing program code, such as a USB flash drive, hard disk, read-only memory (ROM), random access memory (RAM), flash memory, magnetic disk, or optical disk.

[0285] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A method for detecting the location of a burning object, characterized in that, The method includes: Images of the fire scene and scanned radar point cloud data are acquired, and the images and radar point cloud data are associated through a mapping model; Determine the flame ROI in the captured image, and the first depth value of the flame ROI; Extend the flame ROI, expand the region size of the flame ROI to a specified region size, and determine the target spatial region based on the mapping model, the extended flame ROI, and the first depth value; Based on target point cloud data, identify the target burning object and the location data of the target burning object, wherein the target point cloud data is radar point cloud data located in the target spatial region; The process of extending the flame ROI and determining the target spatial region based on the mapping model, the extended flame ROI, and the first depth value includes: Based on the mapping model and the extended flame ROI, the three-dimensional cone region is determined; Determine the neighborhood boundary value of the first depth value, wherein the neighborhood has a specified threshold interval size; Determine the cross section with a depth value of the left neighbor boundary value and the cross section with a depth value of the right neighbor boundary value in the three-dimensional cone region; The volume region between the two cross sections in the three-dimensional cone region is taken as the target space region.

2. The method for detecting the location of a burning object according to claim 1, characterized in that, The captured images include infrared panoramic images and visible light panoramic images, and there is an image mapping relationship between the infrared panoramic images and the visible light panoramic images; The process of determining the flame ROI in the captured image includes: The flame features of the visible light panoramic image are detected to obtain the first flame region; The infrared panoramic image is divided into background sections to identify suspected flame areas. Flame features in the suspected flame areas are then detected to obtain a second flame area. The flame ROI is determined based on the first flame region, the second flame region, and the image mapping relationship.

3. The method for detecting the location of a burning object according to claim 2, characterized in that, The step of determining the flame ROI based on the first flame region, the second flame region, and the image mapping relationship includes: Map the first flame region onto the infrared panoramic image; Determine the overlapping area between the second flame region and the mapped first flame region; Remove regions with a color temperature lower than a specified threshold from the overlapping regions to obtain the flame ROI.

4. The method for detecting the location of a burning object according to claim 3, characterized in that, Determining the first depth value of the flame ROI includes: Two center points are determined between two adjacent frames of an image containing the target region, and the pixel deviation value between the two center points is calculated. The target region is the flame ROI in the infrared panoramic image. Based on the rotation angle of the images captured in the two adjacent frames, the camera intrinsic parameters, and the pixel deviation value, the depth value of any center point is determined.

5. The method for detecting the location of a burning object according to claim 1, characterized in that, The identification of the target burning object and its location data based on the target point cloud data includes: Clustering is performed on the target point cloud data to obtain the target burning object and its location data.

6. The method for detecting the location of a burning object according to claim 1, characterized in that, The method for determining the mapping model includes: The radar coordinate system and image coordinate system are calibrated, and the mapping model of the parameter matrix to be solved is determined. The radar point cloud data is converted into a two-dimensional point cloud image, and the overlapping feature points in the captured image and the two-dimensional point cloud image are extracted. The numerical values ​​of the parameter matrix to be solved are determined by using the overlapping feature points.

7. An automatic fire extinguishing method for firefighting equipment, characterized in that, The fire-fighting equipment includes a control unit and a fire monitor, and the automatic fire extinguishing method includes: The control unit executes the method for detecting the location of a burning object as described in any one of claims 1 to 6 to determine the location data of the target burning object. Based on the location data, the control unit controls the fire monitor to rotate and aim at the designated location of the target burning object to perform a fire extinguishing operation.

8. The automatic fire extinguishing method for fire-fighting equipment according to claim 7, characterized in that, The fire-fighting equipment also includes a camera unit and a radar unit, both of which are arranged on the fire monitor. The automatic fire-fighting method further includes: The control unit rotates the fire monitor to a specified angle, the camera unit captures visible light and infrared images of the fire scene, and the radar unit simultaneously scans the fire scene. The control unit rotates the fire monitor to the next specified angle, the camera unit captures visible light and infrared images of the fire scene, and the radar unit simultaneously scans the fire scene. The visible light images and infrared images captured from various angles are used to form visible light panoramic images and infrared panoramic images, respectively, while the data from scanning the fire scene from various angles are used to form radar point cloud data.

9. A system for detecting the location of a burning object, characterized in that, The system includes: The acquisition module is used to acquire images of the fire scene and scanned radar point cloud data, wherein the images and the radar point cloud data are associated through a mapping model. An image processing module is used to determine the flame ROI in the captured image and a first depth value of the flame ROI; The point cloud processing module is used to extend the flame ROI and determine the target spatial region based on the mapping model, the extended flame ROI, and the first depth value. The determination module is used to identify the target burning object and the location data of the target burning object based on the target point cloud data, wherein the target point cloud data is radar point cloud data located in the target spatial region; The process of extending the flame ROI, expanding its size to a specified area, and determining the target spatial region based on the mapping model, the extended flame ROI, and the first depth value includes: Based on the mapping model and the extended flame ROI, the three-dimensional cone region is determined; Determine the neighborhood boundary value of the first depth value, wherein the neighborhood has a specified threshold interval size; Determine the cross section with a depth value of the left neighbor boundary value and the cross section with a depth value of the right neighbor boundary value in the three-dimensional cone region; The volume region between the two cross sections in the three-dimensional cone region is taken as the target space region.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; A memory connected to the at least one processor; The memory stores instructions executable by the at least one processor, which implements the method described in any one of claims 1 to 8 by executing the instructions stored in the memory.

11. A fire truck, characterized in that, The fire truck uses the automatic fire extinguishing method of the fire-fighting equipment as described in claim 7 or 8.

12. A fire truck, characterized in that, The fire truck is equipped with the electronic equipment described in claim 10.

13. A machine-readable storage medium storing machine instructions that, when executed on a machine, cause the machine to perform the method of any one of claims 1 to 8.

Images