Automatic fire monitor control method and device and automatic fire monitor

Abstract

The invention is suitable for the technical field of fire fighting, and provides an automatic fire monitor control method and device, and an automatic fire monitor, the method comprises the following steps: obtaining a detection image and a depth image of a target scene, the detection image comprising a pixel coordinate value of a target detection frame, the target detection frame comprising a flame target in the target scene, and the depth image comprising a pixel coordinate value of the target detection frame; the depth image comprises pixel depth values of all objects in the target scene; pixel matching mapping is carried out on the detection image and the depth image, and the three-dimensional coordinate value of the fuel area center point of the flame target is determined; according to the three-dimensional coordinate value of the fuel area center point of the flame target, the injection angle value of a monitor head of the automatic fire monitor is determined, and the injection angle value comprises a horizontal angle value and a pitching angle value; and according to the horizontal angle value and the pitching angle value, a monitor head of the automatic fire monitor is controlled to spray a fire extinguishing agent to the center point of the fuel area of the flame target. The automatic fire monitor carried on the mobile platform can automatically identify the fire source through the method, and aims at the fire source to accurately extinguish fire.

Description

Technical Field

[0001] The present application belongs to the field of fire fighting technology, and in particular, relates to a control method and device for an automatic fire fighting cannon and an automatic fire fighting cannon. Background Art

[0002] Fire monitors are an important firefighting equipment for long-distance fire fighting. At present, most of the fire monitors on the market are remote-controlled water monitors, and there are only a few automatic fire monitors, and these few automatic fire monitors are fixed by infrared and ultraviolet sensors and suspended at high places in buildings.

[0003] In recent years, mobile firefighting platforms such as firefighting robots have gradually emerged, but the fire cannons they carry are still remote-controlled. Moreover, the current fixed suspended automatic firefighting water cannons are difficult to adapt to the working conditions of mobile firefighting platforms due to their different sensors and positioning methods. They cannot automatically identify the source of fire and cannot accurately aim at the source of fire for fire extinguishing. Summary of the invention

[0004] The embodiments of the present application provide a control method and device for an automatic fire cannon, and an automatic fire cannon, which can solve the problem in the prior art that the fire cannon mounted on a mobile platform cannot automatically identify the fire source and cannot aim to extinguish the fire.

[0005] In a first aspect, an embodiment of the present application provides a control method for an automatic fire monitor, the method comprising:

[0006] Acquire a detection image and a depth image of a target scene, wherein the detection image includes pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes pixel depth values ​​of all objects in the target scene;

[0007] Perform pixel matching mapping on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0008] Determine the spray angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel zone of the flame target, wherein the spray angle value includes a horizontal angle value and a pitch angle value;

[0009] According to the horizontal angle value and the elevation angle value, the gun head of the automatic fire-fighting cannon is controlled to spray the fire extinguishing agent toward the center point of the fuel area of ​​the flame target.

[0010] In a possible implementation manner of the first aspect, before acquiring the detection image and the depth image of the target scene, the method includes:

[0011] Scanning the target scene simultaneously by using a first camera and a second camera in a binocular camera to obtain a first initial image of the target scene captured by the first camera and a second initial image of the target scene captured by the second camera;

[0012] Identifying the flame target appearing in the first initial image, and generating the detection image having the target detection frame;

[0013] Calculate pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image to generate the depth image of the target scene.

[0014] In a possible implementation manner of the first aspect, identifying the flame target appearing in the first initial image and generating the detection image having the target detection frame includes:

[0015] Using a target detection model, the flame target in the first initial image is identified to obtain the target detection frame;

[0016] Determining whether the flame target in the target detection frame has a flame dynamic flickering feature by an inter-frame analysis algorithm;

[0017] In the case where it is determined that the flame target has the flame dynamic flickering feature, the detection image is generated.

[0018] In a possible implementation manner of the first aspect, calculating pixel depth values ​​of all objects in the target scene according to a parallax difference between the first initial image and the second initial image to generate the depth image of the target scene includes:

[0019] According to preset matching conditions, a plurality of preset feature points in the first initial image and the second initial image are respectively identified to obtain a plurality of first feature points in the first initial image and a plurality of second feature points in the second initial image;

[0020] Perform feature point matching on the plurality of the first feature points and the plurality of the second feature points to obtain a matching feature point pair corresponding to each of the preset feature points;

[0021] According to each of the matching feature point pairs, calculating a position difference between the first feature point and the second feature point in the matching feature point pair to obtain a disparity difference;

[0022] Calculating a pixel depth value corresponding to the preset feature point according to the parallax difference and the internal and external parameters of the binocular camera;

[0023] The depth image of the target scene is generated according to the pixel depth values ​​corresponding to the plurality of preset feature points.

[0024] In a possible implementation manner of the first aspect, performing pixel matching mapping on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target includes:

[0025] Determine, in the depth image, a pixel depth value corresponding to the pixel coordinate value of the bottom midpoint of the target detection frame according to a pixel coordinate mapping relationship between the detection image and the depth image and a pixel coordinate value of the bottom midpoint of the target detection frame in the detection image;

[0026] Converting the pixel coordinate value of the bottom midpoint of the target detection frame into a three-dimensional coordinate value according to the pixel depth value and the internal and external parameters of the binocular camera;

[0027] The three-dimensional coordinate value is used as the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target.

[0028] In a possible implementation of the first aspect, determining the spray angle value of the gun head of the automatic fire monitor according to the three-dimensional coordinate value of the center point of the fuel zone of the flame target includes:

[0029] Determine the horizontal angle value of the automatic fire monitor according to the horizontal coordinate value of the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0030] After determining the horizontal angle value of the automatic fire monitor, controlling the gun head of the automatic fire monitor to rotate horizontally by the horizontal angle value to obtain a first gun head position of the gun head of the automatic fire monitor, wherein the first gun head position is used to indicate that the gun head of the automatic fire monitor has been aimed at the center point of the fuel area of ​​the flame target in the horizontal direction; determining the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target according to the first gun head position;

[0031] The elevation angle value of the automatic fire cannon is determined according to the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target and the fire cannon jet trajectory function of the automatic fire cannon spraying the fire extinguishing agent.

[0032] In a possible implementation manner of the first aspect, after controlling the gun head of the automatic fire monitor to spray the fire extinguishing agent at the center point of the fuel area of ​​the flame target according to the horizontal angle value and the elevation angle value, the method further includes:

[0033] The jet landing point position of the fire extinguishing agent sprayed by the automatic fire monitor is detected by a landing point detection model to obtain the jet landing point position coordinate value;

[0034] If the jet landing point position coordinate value is different from the three-dimensional coordinate value of the center point of the fuel zone of the flame target, a position deviation value is obtained according to the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel zone of the flame target;

[0035] According to the position deviation value, the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor are adjusted to obtain an updated horizontal angle value and an updated elevation angle value.

[0036] In a second aspect, an embodiment of the present application provides a control device for an automatic fire monitor, the device comprising:

[0037] An acquisition module, used to acquire a detection image and a depth image of a target scene, wherein the detection image includes pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes pixel depth values ​​of all objects in the target scene;

[0038] A mapping module, used for performing pixel matching mapping between the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0039] A determination module, used to determine the spray angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, wherein the spray angle value includes a horizontal angle value and a pitch angle value;

[0040] The spray module is used to control the gun head of the automatic fire-fighting cannon to spray the fire extinguishing agent at the center point of the fuel area of ​​the flame target according to the horizontal angle value and the elevation angle value.

[0041] In a third aspect, an embodiment of the present application provides an automatic fire cannon, which includes: a cannon head, a binocular camera, a fire cannon pan / tilt and a control unit, wherein:

[0042] The binocular camera is parallel to and in the same direction as the cannon head, and is used to scan the target scene, obtain a first initial image and a second initial image of the target scene, and send the first initial image and the second initial image to the control unit;

[0043] The control unit is used to identify the flame target appearing in the first initial image and generate a detection image with a target detection frame; calculate the pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image, and generate the depth image of the target scene; wherein the detection image includes the pixel coordinate values ​​of the target detection frame, the target detection frame includes the flame target in the target scene, and the depth image includes the pixel depth values ​​of all objects in the target scene; and, perform pixel matching mapping between the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; and, determine the injection angle value of the gun head of the automatic fire cannon according to the position coordinate value of the center point of the fuel area of ​​the flame target, wherein the injection angle value includes a horizontal angle value and a pitch angle value; and, send the injection angle value to the fire cannon pan / tilt;

[0044] The fire gun pan / tilt platform includes a horizontal angle motor and a pitch angle motor, which are used to support the gun head, and after receiving the spray angle value sent by the control unit, the horizontal angle motor and the pitch angle motor are used to control the gun head to rotate to a corresponding angle to aim at the center point of the fuel area of ​​the flame target and spray the fire extinguishing agent.

[0045] In a possible implementation manner of the third aspect, the control unit further includes a trajectory correction module, wherein:

[0046] The trajectory correction module is used to detect the jet landing position of the fire extinguishing agent sprayed by the automatic fire monitor through a landing point detection model to obtain the jet landing point position coordinate value;

[0047] If the jet landing point position coordinate value is different from the three-dimensional coordinate value of the center point of the fuel zone of the flame target, a position deviation value is obtained according to the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel zone of the flame target;

[0048] According to the position deviation value, the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor are adjusted to obtain an updated horizontal angle value and an updated elevation angle value.

[0049] In a fourth aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the control method of the automatic fire cannon described in any one of the above items when executing the computer program.

[0050] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the control method of the automatic fire cannon described in any one of the above items is implemented.

[0051] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any of the above-described methods for controlling an automatic fire cannon.

[0052] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0053] The embodiment of the present application provides a control method for an automatic fire cannon, which first obtains a detection image and a depth image of a target scene, wherein the detection image includes the pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes the pixel depth values ​​of all objects in the target scene; then, the detection image and the depth image are pixel-matched and mapped to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; then, according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, the spray angle value of the gun head of the automatic fire cannon is determined, wherein the spray angle value includes a horizontal angle value and a pitch angle value; according to the horizontal angle value and the pitch angle value, the gun head of the automatic fire cannon is controlled to spray a fire extinguishing agent to the center point of the fuel area of ​​the flame target. The automatic fire cannon mounted on a mobile platform can automatically identify the source of fire through this method, and accurately aim at the source of fire for precise fire extinguishing. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 It is a flow chart of a control method of an automatic fire monitor provided in one embodiment of the present application;

[0056] Figure 2 is a flow chart of a control method of an automatic fire monitor provided in another embodiment of the present application;

[0057] Figure 3 It is a structural schematic diagram of an automatic fire monitor provided in one embodiment of the present application;

[0058] Figure 4 It is a structural schematic diagram of a control device for an automatic fire monitor provided in one embodiment of the present application;

[0059] Figure 5 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0060] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0061] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0062] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0063] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0064] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0065] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0066] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a control method of an automatic fire monitor provided in an embodiment of the present application. Figure 1 In the present invention, the control method of the automatic fire monitor includes:

[0067] S11, obtaining a detection image and a depth image of the target scene, wherein the detection image includes pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes pixel depth values ​​of all objects in the target scene;

[0068] S12, performing pixel matching mapping on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0069] S13, determining the spray angle value of the gun head of the automatic fire monitor according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, wherein the spray angle value includes a horizontal angle value and a pitch angle value;

[0070] S14. According to the horizontal angle value and the elevation angle value, the gun head of the automatic fire monitor is controlled to spray the fire extinguishing agent toward the center point of the fuel area of ​​the flame target.

[0071] It should be noted that in this embodiment, the execution subject can be an electronic device such as a terminal device or a server, and the control method of the automatic fire monitor is applied to the automatic fire monitor. The automatic fire monitor can automatically detect fire, locate the fire source, and extinguish the fire by spraying fire extinguishing agents (such as foam, water, etc.).

[0072] In step S11, the detection image is an image obtained after flame target detection is performed on an image reflecting the current state of the target scene captured by an image acquisition device such as a binocular camera. The flame target is an object identified as a flame in the detection image. The target detection frame is a rectangular frame surrounding the flame target in the detection image, and the target detection frame is used to identify the position and size of the flame target. In the detection image, the flame target is marked by the target detection frame, and at the same time, the pixel coordinate value of the target detection frame is also reflected.

[0073] The depth image is an image that reflects the depth information of all objects in the target scene. It is obtained by processing an image that reflects the current state of the target scene captured by an image acquisition device such as a binocular camera. The depth image includes the pixel depth values ​​of all objects in the target scene. Among them, the pixel depth value reflects the distance between the surface of the object in the target scene and the image acquisition device such as a binocular camera. It should be understood that the position of the flame target can be identified by the detection image, and the depth information of all objects (such as the flame target) in the target scene can be obtained by the depth image. These two images help determine the three-dimensional position of the flame target.

[0074] In step S12, pixel matching mapping refers to one-to-one correspondence between pixels in two images. In this embodiment, the pixels in the detection image and the depth image are matched one-to-one so as to match the position of the flame target in the detection image with the depth information in the depth image, so that the actual three-dimensional position of the center point of the fuel area of ​​the flame target, that is, the three-dimensional coordinate value, can be calculated. The center point of the fuel area of ​​the flame target is the root or the flame core of the flame, which is where the combustion starts, and is also the area where the fuel is most concentrated and the reaction between the fuel and oxygen is most intense. It should be understood that by combining the information of the detection image and the depth image, the positioning of the flame target in the target scene can be achieved.

[0075] In step S13, the spray angle value refers to the direction and angle when the gun head of the automatic fire cannon sprays the fire extinguishing agent. The spray angle value determines the direction of the jet in space. The spray angle value mainly includes a horizontal angle value and a pitch angle value. Among them, the horizontal angle value refers to the rotation angle of the gun head of the automatic fire cannon in the horizontal direction relative to the axis of the gun body, which determines the direction of the jet on the horizontal plane. By adjusting the horizontal angle value, the jet can be moved left and right on the horizontal plane, thereby covering a wider area and ensuring that the horizontal position of the flame target can be reached. The pitch angle value is the inclination angle of the gun head of the automatic fire cannon in the vertical direction relative to the axis of the gun body, which determines the direction of the jet in the vertical direction. By adjusting the pitch angle value, the jet can be moved up and down on the vertical plane, thereby adapting to flame targets of different heights and distances.

[0076] Specifically, after obtaining the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, the horizontal angle value and the pitch angle value to which the gun head of the automatic fire cannon needs to be adjusted can be calculated according to the three-dimensional coordinate value of the center point of the fuel area, so that the gun head can accurately aim at the center point of the fuel area of ​​the flame target.

[0077] In step S14, the fire extinguishing agent is a substance used to extinguish the flame target, such as water, foam, dry powder, etc. Specifically, according to the calculated horizontal angle value and elevation angle value, the gun head of the automatic fire cannon is adjusted to rotate to a corresponding angle so that it is aimed at the center point of the fuel area of ​​the flame target, and then the gun head of the automatic fire cannon is controlled to spray the fire extinguishing agent at the center point of the fuel area of ​​the flame target to extinguish the fire, thereby realizing automatic fire extinguishing.

[0078] It can be understood that the control method of the automatic fire cannon provided in this embodiment includes obtaining a detection image and a depth image of the target scene, wherein the detection image includes the pixel coordinate values ​​of the target detection frame, the target detection frame includes the flame target in the target scene, and the depth image includes the pixel depth values ​​of all objects in the target scene; then, the detection image and the depth image are pixel-matched and mapped to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; then, according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, the spray angle value of the gun head of the automatic fire cannon is determined, wherein the spray angle value includes a horizontal angle value and a pitch angle value; according to the horizontal angle value and the pitch angle value, the gun head of the automatic fire cannon is controlled to spray the fire extinguishing agent to the center point of the fuel area of ​​the flame target. The automatic fire cannon mounted on the mobile platform can automatically identify the fire source through this method, and accurately aim at the fire source for precise fire extinguishing.

[0079] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a control method of an automatic fire monitor provided by another embodiment of the present application. Before acquiring the detection image and depth image of the target scene, Figure 2 In the present invention, the control method of the automatic fire monitor includes:

[0080] S21, scanning the target scene simultaneously by using a first camera and a second camera in the binocular camera to obtain a first initial image of the target scene captured by the first camera and a second initial image of the target scene captured by the second camera;

[0081] S22, identifying the flame target appearing in the first initial image, and generating a detection image with a target detection frame;

[0082] S23. Calculate pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image, and generate a depth image of the target scene.

[0083] In step S21, the binocular camera is an image acquisition device comprising two or more cameras, which are arranged at a certain distance and angle and can capture multiple images of the same scene at the same time. In this embodiment, the binocular camera includes a first camera and a second camera, which can scan the target scene at the same time and simultaneously capture two parallel images with a certain horizontal spacing, namely, a first initial image and a second initial image. The first initial image is the original image of the target scene captured by the first camera; the second initial image is the original image of the target scene captured by the second camera. It should be understood that the two cameras in the binocular camera can capture images of the target scene at the same time, but due to differences in camera positions and viewing angles, the captured images will be different, providing data for the subsequent calculation of the depth information of all objects in the target scene.

[0084] In step S22, the flame target in the first initial image is identified to generate a detection image. The detection image includes a target detection frame, which is a rectangular frame surrounding the detected flame target in the image, and can identify the position and range of the flame target. In this embodiment, since the first initial image and the second initial image are original images of the current state of the target scene simultaneously acquired by the binocular camera, the flame target in the second initial image can also be identified to obtain a corresponding detection image.

[0085] In step S23, the parallax difference is the position difference of objects in the same target scene captured by the two cameras in the binocular camera due to the different positions and viewing angles of the two cameras. By calculating the parallax difference, the depth information of the object, that is, the pixel depth value, can be inferred. The pixel depth value is the value of each pixel in the depth image, which represents the depth of the object in the scene. Each pixel in the depth image corresponds to a point in the target scene, and the pixel depth value reflects the distance between the point in the target scene and the binocular camera. Specifically, by comparing the difference between the corresponding points in the first initial image and the second initial image, the parallax difference is calculated, so that the pixel depth values ​​of all objects in the target scene can be inferred, and then the depth image of the target scene is obtained.

[0086] It should be understood that in this embodiment, detailed visual information of the target scene can be obtained by detecting images and depth images, including the position of the flame target and the depth information of objects in the target scene, which provides data support for the subsequent three-dimensional position positioning of the center point of the fuel area of ​​the flame target.

[0087] In a possible implementation, identifying a flame target appearing in the first initial image and generating a detection image with a target detection frame includes:

[0088] Using the target detection model, the flame target in the first initial image is identified to obtain a target detection frame;

[0089] Through the inter-frame analysis algorithm, it is determined whether the flame target in the target detection frame has the dynamic flickering characteristics of the flame;

[0090] When it is determined that the flame target has the dynamic flickering characteristics of the flame, a detection image is generated.

[0091] It should be noted that, in the present embodiment, the target detection model is a flame target detection model based on deep learning, which uses the computer vision algorithm YOLOv5 as the base model of the embodiment, and is based on its excellent balance between speed and accuracy, and is trained and optimized for flame detection tasks, and has a stable flame target detection capability. In addition, since the target detection model is to identify a single frame image, there is a possibility of false alarm (i.e., inter-frame analysis algorithm can be used to make full use of multi-frame images with time information, and the dynamic flickering characteristics of flames are detected in continuous frame images, eliminating the influence of interfering objects, and further improving the reliability of detection. Among them, the inter-frame analysis algorithm is an algorithm for processing image sequences for analyzing the differences between continuous frames. Since the dynamic flickering of flames is a dynamic behavior unique to flames, the dynamic flickering characteristics of flames can be used as an important feature for identifying flames.

[0092] Specifically, when a flame appears in the first initial image, the target detection model will identify the flame target and generate a corresponding target detection frame, which will surround the flame target and prompt the system to find the fire source. After that, the inter-frame analysis algorithm is used to further detect whether the target detection frame has the dynamic flickering characteristics of the flame. When the inter-frame analysis algorithm determines that the flame target in the target detection frame has the dynamic flickering characteristics of the flame, it can be determined that it is a real flame target, and then the final detection image is generated.

[0093] It should be understood that in this embodiment, by combining the target detection model and the inter-frame analysis algorithm, the flame target can be more accurately identified and located, and a detection image can be generated so that the automatic fire cannon can subsequently locate the fire source more accurately and quickly to perform effective fire extinguishing operations.

[0094] In a possible implementation, calculating pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image to generate a depth image of the target scene includes:

[0095] According to the preset matching conditions, a plurality of preset feature points in the first initial image and the second initial image are respectively identified to obtain a plurality of first feature points in the first initial image and a plurality of second feature points in the second initial image;

[0096] Perform feature point matching on the plurality of first feature points and the plurality of second feature points to obtain a matching feature point pair corresponding to each preset feature point;

[0097] According to each matching feature point pair, a position difference between a first feature point and a second feature point in the matching feature point pair is calculated to obtain a disparity difference;

[0098] According to the parallax difference and the internal and external parameters of the binocular camera, the pixel depth value corresponding to the preset feature point is calculated;

[0099] A depth image of the target scene is generated according to the pixel depth values ​​corresponding to the plurality of preset feature points.

[0100] It should be noted that the preset matching conditions are a series of pre-set rules for matching feature points, which are mainly used to determine which preset feature points need to be matched. Among them, the preset matching conditions may include the consistency of the shape, size, color and other features of the feature points. The preset feature points are points with significant features pre-set in the image, such as corner points, edge points, etc. The first feature point is a preset feature point identified in the first initial image, and correspondingly, the second feature point is a feature point identified in the second initial image for the same preset feature point. Due to the difference in the position and viewing angle of the first camera and the second camera in the binocular camera, the position of the feature points in the same target scene in the two initial images will be different. The matching feature point pair is a pair of feature points that are matched after pairing the feature points in the first initial image and the second initial image. Each matching feature point pair represents the features of a feature point in the target scene under two different viewing angles.

[0101] The parallax difference is the position difference between the first feature point in the first initial image and the corresponding second feature point in the second initial image in the matching feature point pair. The larger the parallax difference, the closer the object is to the binocular camera, and the smaller the parallax difference, the farther the object is from the binocular camera. The internal and external parameters of the binocular camera are parameters that describe the geometric characteristics of the binocular camera. Among them, the internal parameters mainly include the focal length and principal point coordinates of the camera, and the external parameters mainly describe the relative position and posture between the two cameras. In the binocular camera, the pixel depth value of each feature point in the target scene, that is, the distance from the object to the binocular camera, can be calculated through the parallax difference and the internal and external parameters of the camera.

[0102] Specifically, in the process of generating a depth image of a target scene according to a first initial image and a second initial image, preset feature points with significant features in the first initial image and the second initial image are first identified according to preset matching conditions, and a plurality of first feature points and a plurality of second feature points are obtained respectively; then, the plurality of first feature points and the plurality of second feature points are feature point matched to obtain a pair of matched feature points after successful matching. For a pair of matched feature points that successfully matches, the position difference between the first feature point in the first initial image and the second feature point in the second initial image in the matching feature point pair is calculated, that is, the parallax difference of the corresponding preset feature points. After obtaining the parallax difference, the pixel depth value corresponding to each preset feature point is calculated in combination with the internal and external parameters of the binocular camera to obtain the three-dimensional depth of each preset feature point. Finally, the depth image of the target scene can be generated according to the pixel depth values ​​of the plurality of preset feature points. In the depth image of the target scene, each pixel depth value represents the depth information of the corresponding point of each object in the target scene.

[0103] In a possible implementation, pixel matching mapping is performed on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, including:

[0104] Determine, in the depth image, a pixel depth value corresponding to the pixel coordinate value of the bottom midpoint of the target detection frame according to a pixel coordinate mapping relationship between the detection image and the depth image and a pixel coordinate value of the bottom midpoint of the target detection frame in the detection image;

[0105] According to the pixel depth value and the internal and external parameters of the binocular camera, the pixel coordinate value of the bottom midpoint of the target detection frame is converted into a three-dimensional coordinate value;

[0106] The three-dimensional coordinate value is used as the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target.

[0107] It should be noted that the pixel coordinate mapping relationship is used to describe the correspondence between the pixel coordinates in the detection image and the pixel coordinates in the depth image. The pixel in the detection image can be uniquely mapped to a pixel in the depth image through the pixel coordinate mapping relationship. The bottom midpoint of the target detection frame refers to the midpoint of the bottom edge of the rectangular frame surrounding the flame target. In this embodiment, the two-dimensional pixel coordinates in the image can be converted into three-dimensional coordinates through the pixel depth value and the internal and external parameters of the binocular camera.

[0108] Specifically, by using the pixel coordinate mapping relationship between the detection image and the depth image, the pixel position corresponding to the pixel coordinate value of the bottom midpoint of the target detection frame can be found in the depth image, and the value of the pixel position in the depth image, that is, the pixel depth value, can be read. The pixel depth value reflects the physical depth information of the flame starting point of the flame target in the target scene. Afterwards, a geometric transformation is performed in combination with the pixel depth value and the internal and external parameters of the binocular camera to convert the pixel coordinate value of the bottom midpoint of the target detection frame into a three-dimensional coordinate value. The three-dimensional coordinate value is the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, which represents the position of the center point of the fuel area of ​​the flame target in three-dimensional space.

[0109] It should be understood that this embodiment can accurately obtain the position information of the center point of the fuel area of ​​the flame target in three-dimensional space, providing data support for subsequent targeted fire extinguishing.

[0110] In a possible implementation, the spray angle value of the gun head of the automatic fire monitor is determined according to the three-dimensional coordinate value of the center point of the fuel zone of the flame target, including:

[0111] Determine the horizontal angle value of the automatic fire monitor according to the horizontal coordinate value of the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0112] After determining the horizontal angle value of the automatic fire monitor, the gun head of the automatic fire monitor is controlled to rotate horizontally to the horizontal angle value to obtain the first gun head position of the gun head of the automatic fire monitor. The first gun head position is used to indicate that the gun head of the automatic fire monitor has been aimed at the center point of the fuel area of ​​the flame target in the horizontal direction;

[0113] Determine updated three-dimensional coordinate values ​​of the center point of the fuel zone of the flame target according to the first gun head position;

[0114] The elevation angle value of the automatic fire cannon is determined according to the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target and the fire cannon jet trajectory function of the automatic fire cannon spraying the fire extinguishing agent.

[0115] It should be noted that the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target reflects the actual position of the flame target in three-dimensional space. According to the horizontal coordinate value of the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, that is, the X-axis coordinate value and the Y-axis coordinate value, the relative position of the center point of the fuel area of ​​the flame target and the gun head of the automatic fire cannon in the horizontal direction can be determined, thereby obtaining the horizontal angle value of the automatic fire cannon.

[0116] The fire monitor jet trajectory function is the relationship between the pitch angle value of the automatic fire monitor's gun head and the jet trajectory of the fire extinguishing agent sprayed, which is obtained in advance based on a calibration experiment.

[0117] After the gun head of the automatic fire cannon is adjusted in the horizontal direction, the focus of the binocular camera may not be on the Z-axis rotation axis, which will cause the relative position of the flame target and the binocular camera (gun head of the automatic fire cannon) to change. Therefore, it is necessary to re-detect the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target.

[0118] Specifically, according to the horizontal coordinate value in the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, the horizontal angle value of the automatic fire cannon is first determined; then, the gun head of the automatic fire cannon is horizontally rotated by the horizontal angle value, so that the gun head of the automatic fire cannon is aimed at the center point of the fuel area of ​​the flame target in the horizontal direction. At this time, the gun head position of the automatic fire cannon is the first gun head position; then, according to the first gun head position, the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target is recalculated to obtain the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target. Finally, according to the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target combined with the fire cannon jet trajectory function of the automatic fire cannon, the pitch angle value that the gun head of the automatic fire cannon needs to be adjusted can be calculated so that the gun head of the automatic fire cannon is aimed at the center point of the fuel area of ​​the flame target in the vertical direction.

[0119] It should be noted that in this embodiment, the horizontal angle value of the automatic fire cannon can also be calculated based on the deviation value between the central axis of the target detection frame and the preset vertical reference line and the viewing angle parameter of the binocular camera. Among them, the central axis of the target detection frame is the center line of the target detection frame in the first initial image; the preset vertical reference line represents the horizontal orientation of the gun head of the automatic fire cannon. The horizontal angle values ​​of the automatic fire cannon calculated by these two methods can be averaged to improve the accuracy of the calculation.

[0120] It should be noted that in this embodiment, PID control can also be used directly, that is, the PID controller can be used to directly control the automatic fire cannon's gun head to rotate until the central axis of the target detection frame coincides with the preset vertical reference line. At this time, the second gun head position of the automatic fire cannon's gun head is obtained; then, according to the second gun head position, the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target is determined. In this way, in this embodiment, there is no need to obtain the horizontal angle value of the automatic fire cannon's gun head, and only the elevation angle value of the automatic fire cannon's gun head needs to be determined.

[0121] It should be understood that by adjusting the gun head of the automatic fire cannon to the corresponding angle according to the determined horizontal angle value and pitch angle value, and spraying the fire extinguishing agent, accurate fire extinguishing of the flame target can be achieved.

[0122] In a possible implementation, after controlling the gun head of the automatic fire cannon to spray the fire extinguishing agent at the center point of the fuel zone of the flame target according to the horizontal angle value and the elevation angle value, the control method of the automatic fire cannon further includes:

[0123] Through the landing point detection model, the jet landing point position of the automatic fire monitor spraying fire extinguishing agent is detected to obtain the jet landing point position coordinate value;

[0124] If the jet landing point position coordinate value is different from the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, then the position deviation value is obtained according to the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0125] According to the position deviation value, the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor are adjusted to obtain an updated horizontal angle value and an updated elevation angle value.

[0126] It should be noted that due to detection errors, environmental influences and other factors. After the first aiming, the jet of the fire extinguishing agent sprayed by the automatic fire cannon may deviate from the center of the fuel area of ​​the flame target. At this time, the landing point detection model will be used to monitor the landing point of the jet. If the landing point of the jet deviates from the flame target, the deviation value will be used as feedback, and the horizontal angle value and pitch angle value of the cannon head will be calculated again, and the cannon head will be adjusted to control the jet landing point to coincide with the center of the fuel area of ​​the flame target, so as to achieve precise fire extinguishing.

[0127] The landing point detection model is a model for detecting and determining the actual landing point position of the jet of the fire extinguishing agent sprayed by the automatic fire cannon. In this embodiment, the camera with a large horizontal offset from the cannon head in the binocular camera is mainly used to monitor the jet landing point position. The jet landing point position coordinate value is a three-dimensional coordinate value representing the actual landing of the fire extinguishing agent jet in the target scene. The jet landing point position coordinate value is obtained through the landing point detection model. The position deviation value is the difference between the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel zone of the flame target, which reflects the deviation between the jet of the fire extinguishing agent and the flame target at the current angle of the automatic fire cannon spraying the fire extinguishing agent.

[0128] Specifically, after the automatic fire monitor sprays the fire extinguishing agent, the drop point detection model is used to detect the jet drop point position of the fire extinguishing agent in real time to obtain the three-dimensional coordinate value of the jet drop point, that is, the jet drop point position coordinate value; then, by comparing the jet drop point position coordinate value with the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, if the two are different, it means that the jet of the fire extinguishing agent does not accurately hit the flame target. At this time, the difference between the two coordinate values ​​is calculated to obtain the position deviation value; according to the position deviation value, the horizontal angle value and the pitch angle value that need to be adjusted are calculated to obtain the updated horizontal angle value and the updated pitch angle value. Finally, according to the updated horizontal angle value and the updated pitch angle value, the gun head of the automatic fire monitor is adjusted to the updated angle value to spray the fire extinguishing agent. It should be noted that this step may need to be performed multiple times. By performing this step multiple times, the automatic fire monitor can realize real-time detection and adjustment of the drop point position of the fire extinguishing agent jet, thereby ensuring that the fire extinguishing agent can accurately hit the flame target and improve the fire extinguishing efficiency.

[0129] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0130] Corresponding to the control method of an automatic fire monitor in the above embodiment, Figure 3 A schematic structural diagram of an automatic fire monitor provided in one embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0131] Reference Figure 3 The automatic fire cannon 3 of this embodiment includes: a cannon head 31, a binocular camera 32, a fire cannon pan / tilt 33 and a control unit (not shown in the figure), wherein:

[0132] The binocular camera 32 is parallel to and in the same direction as the cannon head 31, and is used to scan the target scene, obtain a first initial image and a second initial image of the target scene, and send the first initial image and the second initial image to the control unit;

[0133] A control unit, for identifying the flame target appearing in the first initial image, generating a detection image with a target detection frame; calculating the pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image, and generating a depth image of the target scene; wherein the detection image includes the pixel coordinate values ​​of the target detection frame, the target detection frame includes the flame target in the target scene, and the depth image includes the pixel depth values ​​of all objects in the target scene; and, performing pixel matching mapping between the detection image and the depth image, determining the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; and, determining the injection angle value of the gun head of the automatic fire cannon according to the position coordinate value of the center point of the fuel area of ​​the flame target, wherein the injection angle value includes the horizontal angle value and the pitch angle value; and, sending the injection angle value to the fire cannon pan / tilt 33;

[0134] The fire gun platform 33 includes a horizontal angle motor and a pitch angle motor, which are used to support the gun head 31, and, after receiving the spray angle value sent by the control unit, control the gun head 31 to rotate to the corresponding angle through the horizontal angle motor and the pitch angle motor to aim at the center point of the fuel area of ​​the flame target and spray the fire extinguishing agent.

[0135] It should be noted that the binocular camera 32 is an image acquisition device comprising two or more cameras, which are arranged at a certain distance and angle and can capture multiple images of the same scene at the same time. In the present embodiment, the binocular camera 32 includes a first camera and a second camera, which can scan the target scene at the same time and simultaneously acquire two parallel images with a certain horizontal spacing, namely, a first initial image and a second initial image. After the binocular camera 32 acquires the first initial image and the second initial image, the first initial image and the second initial image are sent to the control unit for processing. In the present embodiment, when the binocular camera 32 is installed, the binocular camera 32 should be parallel to the direction of the gun head 31 and in the same direction. Among them, the relative position of the focus of the binocular camera 32 and the center point of the end face of the gun head 31 is pre-set.

[0136] The control unit is the core processing unit of the automatic fire cannon 3, which mainly processes the data of the first initial image and the second initial image sent by the binocular camera 42, that is, the target detection model in the control unit identifies the flame target of the target scene in the first initial image and generates a detection image; according to the parallax difference between the first initial image and the second initial image, the pixel depth values ​​of all objects in the target scene are calculated to generate a depth image of the target scene; then, the pixel coordinate values ​​of the target detection frame in the detection image are combined with the pixel depth values ​​in the depth image to obtain the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target. Then, the automatic aiming model in the control unit determines the injection angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel area, and sends the injection angle value to the fire cannon pan-tilt 33, so that the fire cannon pan-tilt 33 can adjust the direction of the gun head 31.

[0137] The fire cannon platform 33 is the core component of the automatic fire cannon 3, which is responsible for supporting the cannon head 31 and can adjust the spraying direction of the cannon head 31. In this embodiment, the fire cannon platform 33 includes two motors, namely, a horizontal angle motor that controls the cannon head 31 to adjust the angle in the horizontal direction and a pitch angle motor that controls the cannon head 31 to adjust the angle in the pitch direction. During the automatic aiming process of the automatic fire cannon, after receiving the spraying angle value sent by the control unit, the fire cannon platform 33 will adjust the cannon head 31 through the motor to rotate, rotate the cannon head 31 to the angle corresponding to the spraying angle value to aim at the center point of the fuel area of ​​the flame target, and then, after aiming, spray the fire extinguishing agent (water, foam, etc.).

[0138] It should be noted that the fire cannon platform 33 can quickly aim at the flame in actual use, and can withstand the aftereffect force from the automatic fire cannon 3, ensuring that the direction will not be passively adjusted during the fire extinguishing work.

[0139] It should be noted that after the flame target is aimed, the solenoid valve will be controlled to open to achieve automatic spraying of the fire extinguishing agent.

[0140] It can be understood that the automatic fire cannon 3 provided in this embodiment can realize rapid identification and precise positioning of flame targets through the combination of the binocular camera 32 and the target detection model and the automatic aiming model in the control unit, thereby realizing accurate fire extinguishing.

[0141] In a possible implementation, the control unit further includes a trajectory correction module, wherein:

[0142] The trajectory correction module is used to detect the jet landing position of the fire extinguishing agent sprayed by the automatic fire monitor through the landing point detection model to obtain the jet landing point position coordinate value;

[0143] If the jet landing point position coordinate value is different from the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, then the position deviation value is obtained according to the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0144] According to the position deviation value, the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor are adjusted to obtain an updated horizontal angle value and an updated elevation angle value.

[0145] It should be noted that after the automatic fire cannon 3 sprays the fire extinguishing agent, the control unit corrects the spray angle of the gun head of the automatic fire cannon through the trajectory correction module. Specifically, the jet landing point position of the fire extinguishing agent is detected in real time using the landing point detection model to obtain the three-dimensional coordinate value of the jet landing point, that is, the jet landing point position coordinate value; then, by comparing the jet landing point position coordinate value with the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, if the two are different, it means that the jet of the fire extinguishing agent did not accurately hit the flame target. At this time, the difference between the two coordinate values ​​is calculated to obtain the position deviation value; according to the position deviation value, the horizontal angle value and the pitch angle value that need to be adjusted are calculated, so as to obtain the updated horizontal angle value and the updated pitch angle value. Finally, according to the updated horizontal angle value and the updated pitch angle value, the gun head of the automatic fire cannon is adjusted to the updated angle value to spray the fire extinguishing agent.

[0146] It should be noted that the control unit may need to perform the step of correcting the spray angle of the automatic fire cannon through the trajectory correction module multiple times. By performing this step multiple times, the automatic fire cannon 3 can achieve real-time detection and adjustment of the jet landing point position of the fire extinguishing agent, thereby ensuring that the fire extinguishing agent can accurately hit the flame target and improve the fire extinguishing efficiency.

[0147] It should be understood that when the automatic fire cannon 3 starts to spray fire extinguishing agent, the control unit uses the position deviation value between the jet landing point of the fire extinguishing agent collected by the landing point detection model and the flame target as feedback to dynamically calibrate the automatic aiming model in the control unit.

[0148] It should be noted that the automatic fire monitor 3 is not only suitable for fixed installation scenarios, but also for suspended mobile platforms, ground mobile platforms, etc.

[0149] It should be noted that the information interaction, execution process and other contents between the modules in the above-mentioned automatic fire cannon 3 are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0150] Corresponding to the control method of an automatic fire monitor in the above embodiment, Figure 4A schematic structural diagram of a control device for an automatic fire cannon provided in one embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0151] Reference Figure 4 The control device 4 of the automatic fire cannon of this embodiment includes:

[0152] An acquisition module 41 is used to acquire a detection image and a depth image of a target scene, wherein the detection image includes pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes pixel depth values ​​of all objects in the target scene;

[0153] A mapping module 42 is used to perform pixel matching mapping between the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0154] A determination module 43 is used to determine the spray angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, wherein the spray angle value includes a horizontal angle value and a pitch angle value;

[0155] The spray module 44 is used to control the gun head of the automatic fire cannon to spray the fire extinguishing agent at the center point of the fuel area of ​​the flame target according to the horizontal angle value and the elevation angle value.

[0156] It can be understood that the control device of an automatic fire cannon provided in this embodiment, the control device 4 of the automatic fire cannon, acquires the detection image and depth image of the target scene through the acquisition module 41, wherein the detection image includes the pixel coordinate value of the target detection frame, the target detection frame includes the flame target in the target scene, and the depth image includes the pixel depth value of all objects in the target scene; then, the detection image and the depth image are pixel-matched and mapped by the mapping module 42 to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; then, the determination module 43 determines the spray angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, wherein the spray angle value includes the horizontal angle value and the pitch angle value; the spray module 44 controls the gun head of the automatic fire cannon to spray the fire extinguishing agent to the center point of the fuel area of ​​the flame target according to the horizontal angle value and the pitch angle value. The fire cannon mounted on the mobile platform with the control device 4 of the automatic fire cannon can automatically identify the fire source, and accurately aim at the fire source for precise fire extinguishing.

[0157] Furthermore, the control device 4 of the automatic fire monitor includes:

[0158] An initial image acquisition submodule is used to simultaneously scan the target scene through the first camera and the second camera in the binocular camera to obtain a first initial image of the target scene captured by the first camera and a second initial image of the target scene captured by the second camera;

[0159] A detection image generation submodule is used to identify the flame target appearing in the first initial image and generate a detection image with a target detection frame;

[0160] The depth image generation submodule is used to calculate the pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image, and generate a depth image of the target scene.

[0161] Furthermore, the detection image generation submodule includes:

[0162] A target recognition unit, used to recognize the flame target in the first initial image through a target detection model to obtain a target detection frame;

[0163] An inter-frame analysis unit, used to determine whether the flame target in the target detection frame has a flame dynamic flickering feature through an inter-frame analysis algorithm;

[0164] The detection image generating unit is used to generate a detection image when it is determined that the flame target has the dynamic flickering feature of the flame.

[0165] Furthermore, the depth image generation submodule includes:

[0166] A feature point recognition unit, used to recognize a plurality of preset feature points in the first initial image and the second initial image respectively according to a preset matching condition, to obtain a plurality of first feature points in the first initial image and a plurality of second feature points in the second initial image;

[0167] A feature point matching unit, used to perform feature point matching on the plurality of first feature points and the plurality of second feature points to obtain a matching feature point pair corresponding to each preset feature point;

[0168] a disparity calculation unit, configured to calculate, according to each matching feature point pair, a position difference between a first feature point and a second feature point in the matching feature point pair to obtain a disparity difference;

[0169] A depth determination unit, used to calculate the pixel depth value corresponding to the preset feature point according to the parallax difference and the internal and external parameters of the binocular camera;

[0170] The depth image generating unit is used to generate a depth image of the target scene according to the pixel depth values ​​corresponding to the plurality of preset feature points.

[0171] Further, the mapping module 42 includes:

[0172] A pixel mapping unit, configured to determine a pixel depth value corresponding to the pixel coordinate value of the bottom midpoint of the target detection frame in the depth image according to a pixel coordinate mapping relationship between the detection image and the depth image and a pixel coordinate value of the bottom midpoint of the target detection frame in the detection image;

[0173] A coordinate conversion unit, used for converting a pixel coordinate value of a bottom midpoint of a target detection frame into a three-dimensional coordinate value according to a pixel depth value and internal and external parameters of a binocular camera;

[0174] The coordinate determination unit is used to use the three-dimensional coordinate value as the three-dimensional coordinate value of the center point of the fuel zone of the flame target.

[0175] Further, the determination module 43 includes:

[0176] A horizontal angle determination unit, used to determine the horizontal angle value of the automatic fire monitor according to the horizontal direction coordinate value of the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target;

[0177] A gun head rotation unit is used to control the gun head of the automatic fire cannon to horizontally rotate the gun head of the automatic fire cannon by the horizontal angle value after determining the horizontal angle value of the automatic fire cannon, so as to obtain a first gun head position of the gun head of the automatic fire cannon, wherein the first gun head position is used to indicate that the gun head of the automatic fire cannon has been aimed at the center point of the fuel area of ​​the flame target in the horizontal direction;

[0178] A coordinate value updating unit, which determines an updated three-dimensional coordinate value of a center point of a fuel zone of the flame target according to the first gun head position;

[0179] The pitch angle determination unit is used to determine the pitch angle value of the automatic fire cannon according to the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target and the fire cannon jet trajectory function of the automatic fire cannon spraying the fire extinguishing agent.

[0180] Furthermore, the control device 4 of the automatic fire monitor includes:

[0181] The jet landing point determination module is used to detect the jet landing point position of the automatic fire monitor spraying the fire extinguishing agent through the landing point detection model to obtain the jet landing point position coordinate value;

[0182] A position deviation determination module is used to obtain a position deviation value according to the jet landing point position coordinate value and the three-dimensional coordinate value of the fuel zone center point of the flame target if the jet landing point position coordinate value is different from the three-dimensional coordinate value of the fuel zone center point of the flame target;

[0183] The spray angle update module is used to adjust the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor according to the position deviation value to obtain the updated horizontal angle value and the updated elevation angle value.

[0184] It should be noted that the information interaction, execution process and other contents between the modules in the control device 4 of the above-mentioned automatic fire cannon are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0185] The present application also provides a terminal device, such as Figure 5 As shown, Figure 5 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. Figure 5 The terminal device 5 of this embodiment includes: a memory 51, a processor 52, and a computer program stored in the memory 51 and executable on the processor 52. When the processor 52 executes the computer program, the steps in any one of the above-mentioned automatic fire cannon control method embodiments are implemented.

[0186] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0187] The embodiment of the present application also provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0188] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0189] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0190] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0191] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A control method for an automatic fire monitor, characterized in that: include: Acquire a detection image and a depth image of a target scene, wherein the detection image includes pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes pixel depth values ​​of all objects in the target scene; Perform pixel matching mapping on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; Determine the spray angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel zone of the flame target, wherein the spray angle value includes a horizontal angle value and a pitch angle value; According to the horizontal angle value and the elevation angle value, the gun head of the automatic fire-fighting cannon is controlled to spray the fire extinguishing agent toward the center point of the fuel area of ​​the flame target.

2. The control method of the automatic fire monitor according to claim 1, characterized in that: Before acquiring the detection image and the depth image of the target scene, the method includes: Scanning the target scene simultaneously by using a first camera and a second camera in a binocular camera to obtain a first initial image of the target scene captured by the first camera and a second initial image of the target scene captured by the second camera; Identifying the flame target appearing in the first initial image, and generating the detection image having the target detection frame; Calculate pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image to generate the depth image of the target scene.

3. The control method of the automatic fire monitor according to claim 2, characterized in that: The step of identifying the flame target appearing in the first initial image and generating the detection image having the target detection frame includes: Using a target detection model, the flame target in the first initial image is identified to obtain the target detection frame; Determining whether the flame target in the target detection frame has a flame dynamic flickering feature by an inter-frame analysis algorithm; In the case where it is determined that the flame target has the flame dynamic flickering feature, the detection image is generated.

4. The control method of the automatic fire monitor according to claim 2, characterized in that: The step of calculating pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image to generate the depth image of the target scene includes: According to preset matching conditions, a plurality of preset feature points in the first initial image and the second initial image are respectively identified to obtain a plurality of first feature points in the first initial image and a plurality of second feature points in the second initial image; Perform feature point matching on the plurality of the first feature points and the plurality of the second feature points to obtain a matching feature point pair corresponding to each of the preset feature points; According to each of the matching feature point pairs, calculating a position difference between the first feature point and the second feature point in the matching feature point pair to obtain a disparity difference; Calculating a pixel depth value corresponding to the preset feature point according to the parallax difference and the internal and external parameters of the binocular camera; The depth image of the target scene is generated according to the pixel depth values ​​corresponding to the plurality of preset feature points.

5. The control method of the automatic fire monitor according to claim 2, characterized in that: The step of performing pixel matching mapping on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target includes: Determine, in the depth image, a pixel depth value corresponding to the pixel coordinate value of the bottom midpoint of the target detection frame according to a pixel coordinate mapping relationship between the detection image and the depth image and a pixel coordinate value of the bottom midpoint of the target detection frame in the detection image; Converting the pixel coordinate value of the bottom midpoint of the target detection frame into a three-dimensional coordinate value according to the pixel depth value and the internal and external parameters of the binocular camera; The three-dimensional coordinate value is used as the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target.

6. The control method of the automatic fire monitor according to claim 2, characterized in that: The step of determining the spray angle value of the gun head of the automatic fire monitor according to the three-dimensional coordinate value of the center point of the fuel zone of the flame target comprises: Determine the horizontal angle value of the automatic fire monitor according to the horizontal coordinate value of the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; After determining the horizontal angle value of the automatic fire monitor, controlling the gun head of the automatic fire monitor to rotate horizontally by the horizontal angle value to obtain a first gun head position of the gun head of the automatic fire monitor, wherein the first gun head position is used to indicate that the gun head of the automatic fire monitor has been aimed at the center point of the fuel area of ​​the flame target in the horizontal direction; determining the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target according to the first gun head position; The elevation angle value of the automatic fire cannon is determined according to the updated three-dimensional coordinate value of the center point of the fuel area of ​​the flame target and the fire cannon jet trajectory function of the automatic fire cannon spraying the fire extinguishing agent.

7. The control method of the automatic fire monitor according to claim 1, characterized in that: After controlling the gun head of the automatic fire monitor to spray the fire extinguishing agent to the center point of the fuel zone of the flame target according to the horizontal angle value and the elevation angle value, the method further includes: The jet landing point position of the fire extinguishing agent sprayed by the automatic fire monitor is detected by a landing point detection model to obtain the jet landing point position coordinate value; If the jet landing point position coordinate value is different from the three-dimensional coordinate value of the center point of the fuel zone of the flame target, a position deviation value is obtained according to the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel zone of the flame target; According to the position deviation value, the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor are adjusted to obtain an updated horizontal angle value and an updated elevation angle value.

8. A control device for an automatic fire monitor, characterized in that: include: An acquisition module, used to acquire a detection image and a depth image of a target scene, wherein the detection image includes pixel coordinate values ​​of a target detection frame, the target detection frame includes a flame target in the target scene, and the depth image includes pixel depth values ​​of all objects in the target scene; A mapping module, used for performing pixel matching mapping between the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; A determination module, used to determine the spray angle value of the gun head of the automatic fire cannon according to the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target, wherein the spray angle value includes a horizontal angle value and a pitch angle value; The spray module is used to control the gun head of the automatic fire-fighting cannon to spray the fire extinguishing agent at the center point of the fuel area of ​​the flame target according to the horizontal angle value and the elevation angle value.

9. An automatic fire monitor, characterized in that: The automatic fire cannon includes: a cannon head, a binocular camera, a fire cannon pan / tilt and a control unit, wherein: The binocular camera is parallel to and in the same direction as the cannon head, and is used to scan the target scene, obtain a first initial image and a second initial image of the target scene, and send the first initial image and the second initial image to the control unit; The control unit is used to identify the flame target appearing in the first initial image and generate a detection image with a target detection frame; calculate the pixel depth values ​​of all objects in the target scene according to the parallax difference between the first initial image and the second initial image, and generate the depth image of the target scene; wherein the detection image includes the pixel coordinate values ​​of the target detection frame, the target detection frame includes the flame target in the target scene, and the depth image includes the pixel depth values ​​of all objects in the target scene; and, perform pixel matching mapping on the detection image and the depth image to determine the three-dimensional coordinate value of the center point of the fuel area of ​​the flame target; and, determine the injection angle value of the gun head of the automatic fire cannon according to the position coordinate value of the center point of the fuel area of ​​the flame target, wherein the injection angle value includes a horizontal angle value and a pitch angle value; and, send the injection angle value to the fire cannon pan / tilt; The fire gun pan / tilt platform includes a horizontal angle motor and a pitch angle motor, which are used to support the gun head, and after receiving the spray angle value sent by the control unit, the horizontal angle motor and the pitch angle motor are used to control the gun head to rotate to a corresponding angle to aim at the center point of the fuel area of ​​the flame target and spray the fire extinguishing agent.

10. The automatic fire monitor according to claim 9, characterized in that: The control unit also includes a trajectory correction module, wherein: The trajectory correction module is used to detect the jet landing position of the fire extinguishing agent sprayed by the automatic fire monitor through a landing point detection model to obtain the jet landing point position coordinate value; If the jet landing point position coordinate value is different from the three-dimensional coordinate value of the center point of the fuel zone of the flame target, a position deviation value is obtained according to the jet landing point position coordinate value and the three-dimensional coordinate value of the center point of the fuel zone of the flame target; According to the position deviation value, the horizontal angle value and the elevation angle value of the gun head of the automatic fire monitor are adjusted to obtain an updated horizontal angle value and an updated elevation angle value.

Images