A fire water cannon system and control method based on machine vision

Abstract

The present application relates to a fire water cannon control method, system, computer equipment, storage medium and computer program product based on machine vision. The method includes: obtaining real-time image information when spraying water; analyzing the real-time image information to obtain target area information representing the fire occurrence area in the image information; analyzing the real-time image information to obtain landing point information representing the landing point of the sprayed water in the image information; judging whether the landing point information matches the target area information; if not, calculating the trajectory information and the target area according to a preset angle adjustment algorithm to obtain angle adjustment information; generating and outputting an angle adjustment instruction according to the angle adjustment information. The use of this method can reduce the influence of wind factors in the on-site environment on the accuracy of the water jet position.

Description

Technical Field

[0001] The present application relates to the technical field of fire-fighting equipment, and in particular to a fire-fighting water cannon control method, system, computer equipment, storage medium and computer program product based on machine vision. Background Art

[0002] With the development of technology, intelligent fire-fighting technology has emerged, which has reduced the workload of firefighters.

[0003] The Chinese patent with the announcement number CN115414621A discloses an intelligent control method and related device for a fire monitor, which relates to the field of intelligent control technology. In this application, the fire source location information is obtained from the thermal imaging image of the target scene at the set time, and the original location information of the fire monitor in the target scene at the set time is obtained; then, from the historical location information set corresponding to the fire monitor, the historical location information of the fire monitor at the previous sampling time adjacent to the set time is selected; further, based on the historical location information and the original location information, the target location information of the fire monitor is obtained; finally, based on the obtained target location information and the fire source location information, the working state of the fire monitor is adjusted to improve the accuracy of the fire source attack.

[0004] However, the water jet of this type of fire-fighting water cannon will be affected by the wind force on site, resulting in a certain deviation between the actual water jet and the predetermined water jet, thereby affecting the fire-fighting efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide a fire water cannon control method, system, computer equipment, computer readable storage medium and computer program product based on machine vision, which can reduce the impact of wind factors in the on-site environment on the accuracy of water jet position.

[0006] In a first aspect, the present application provides a fire water cannon control method based on machine vision, the method comprising:

[0007] Get real-time image information when spraying water;

[0008] Analyzing the real-time image information to obtain target area information indicating a fire occurrence area in the image information;

[0009] Analyzing the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information;

[0010] Determining whether the landing point information matches the target area information;

[0011] If not, the trajectory information and the target area are calculated according to a preset angle adjustment algorithm to obtain angle adjustment information;

[0012] Generate and output an angle adjustment instruction based on the angle adjustment information.

[0013] In one embodiment, the specific steps of obtaining target area information representing the fire occurrence area in the image information include:

[0014] Randomly extracting a number of frames of images from the image information as sample images;

[0015] Traversing all sample images and determining whether there are feature pixels in the sample images, wherein the feature pixels are pixels whose pixel values ​​are within a preset sample image pixel range;

[0016] If so, an intersection operation is performed on the position information of the characteristic pixels in all the sample images to obtain the position information of the characteristic pixels in all the sample images and use it as the target area information.

[0017] In one embodiment, the specific steps of obtaining the landing point information representing the landing point of the sprayed water flow in the image information include:

[0018] Extracting the water jet trajectory from the image information by using a preset U-net neural network, wherein the U-net neural network is a network obtained by adding an attention module between the downsampling layer and the upsampling layer on the basis of an existing U-net network;

[0019] Binarizing the jet trajectory to generate a binary image;

[0020] The lowest point of the jet trajectory in the binary image is taken as the jet starting point, the point farthest from the jet starting point is extracted as the jet landing point, and the position information of the jet landing point is taken as the landing point information.

[0021] In one embodiment, the method further comprises:

[0022] Acquire a current infrared image, and determine whether there is a target area indicating that a fire is occurring in the current infrared image according to pixel value data in the current infrared image;

[0023] If yes, obtaining the area data of the target area in the current infrared image;

[0024] According to the area data combined with a preset viewing angle adjustment algorithm, a target viewing angle adjustment angle is calculated, and a viewing angle adjustment instruction is output.

[0025] In one embodiment, the specific method of determining whether there is a target area indicating that a fire is occurring in the current infrared image includes:

[0026] Traversing the current infrared image to determine whether there is a pixel point whose pixel value is within a preset infrared image pixel value range;

[0027] If so, it means that there is a target area indicating that a fire is occurring in the current infrared image;

[0028] If not, it means that there is no target area indicating that a fire is occurring in the current infrared image.

[0029] In one embodiment, the specific steps of presetting the viewing angle adjustment algorithm include:

[0030] According to the area data, sampling is performed within a preset distance range near the target area to obtain sampling data, the sampling data including distance sampling data, elevation sampling data and rotation angle sampling data;

[0031] Determine whether the target area is within the preset range according to the distance sampling data;

[0032] If not, the shift instruction is output, and re-sampling is performed after the shift instruction is executed;

[0033] If yes, then reacquire at least three sets of sampling data and calculate the target surface data;

[0034] Performing coordinate transformation on the target surface data according to a preset coordinate transformation equation to obtain transformed target surface data;

[0035] Calculate the transformed target surface data according to the preset center of gravity solution equation to obtain target point data;

[0036] The target point data is calculated according to a preset viewing angle calculation formula to obtain a target viewing angle adjustment angle.

[0037] In a second aspect, the present application also provides a fire water cannon system based on machine vision, the system comprising:

[0038] An image acquisition module, used for acquiring real-time image information;

[0039] An analysis and calculation module is used to analyze real-time image information to obtain target area information and landing point information, determine whether the trajectory information matches the target information, and calculate angle adjustment information according to a preset angle adjustment algorithm;

[0040] The control module is used to generate and output angle adjustment instructions according to the angle adjustment information.

[0041] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0042] Obtain real-time image information when spraying water;

[0043] Analyzing the real-time image information to obtain target area information indicating a fire occurrence area in the image information;

[0044] Analyzing the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information;

[0045] Determining whether the landing point information matches the target area information;

[0046] If not, the trajectory information and the target area are calculated according to a preset angle adjustment algorithm to obtain angle adjustment information;

[0047] Generate and output an angle adjustment instruction based on the angle adjustment information.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Obtain real-time image information when spraying water;

[0050] Analyzing the real-time image information to obtain target area information indicating a fire occurrence area in the image information;

[0051] Analyzing the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information;

[0052] Determining whether the landing point information matches the target area information;

[0053] If not, the trajectory information and the target area are calculated according to a preset angle adjustment algorithm to obtain angle adjustment information;

[0054] Generate and output an angle adjustment instruction based on the angle adjustment information.

[0055] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0056] Get real-time image information when spraying water;

[0057] Analyzing the real-time image information to obtain target area information indicating a fire occurrence area in the image information;

[0058] Analyzing the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information;

[0059] Determining whether the landing point information matches the target area information;

[0060] If not, the trajectory information and the target area are calculated according to a preset angle adjustment algorithm to obtain angle adjustment information;

[0061] Generate and output an angle adjustment instruction based on the angle adjustment information.

[0062] The above-mentioned machine vision-based fire water cannon control method, system, computer equipment, storage medium and computer program product, through real-time image acquisition of the actual water jet and the fire area at the fire scene, to determine whether the water flow is injected into the fire area. If the water jet fails to enter the fire area due to the wind force or other factors at the scene, the water jet is automatically adjusted according to the real-time image collected to achieve more accurate fire extinguishing. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of a flow chart of a fire water monitor control method based on machine vision in one embodiment;

[0064] Figure 2 is a schematic diagram of a coordinate transformation equation used in step S330 in one embodiment;

[0065] Figure 3 is a schematic diagram of a formula used in step S340 in one embodiment;

[0066] Figure 4 is a schematic diagram of a viewing angle transformation equation used in step S350 in one embodiment;

[0067] Figure 5 is the inverse coordinate transformation equation used in step S350 in one embodiment;

[0068] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] In one embodiment, Figure 1 As shown, a fire water cannon control method based on machine vision is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0071] The method comprises the following steps:

[0072] Step S100: Acquire a current infrared image, and determine whether there is a target area indicating that a fire is occurring in the current infrared image based on pixel value data in the current infrared image.

[0073] The current infrared image is infrared picture information acquired by the infrared imaging device, and the pixel value of the pixel point in the current infrared image and the temperature of the object in the actual scene corresponding to the pixel point satisfy the preset temperature pixel value relationship, and the temperature pixel value is determined by the hardware and / or software of the infrared imaging device. The specific steps of step S100 include:

[0074] Step S110: traverse the current infrared image to determine whether there is a pixel point whose pixel value is within a preset infrared image pixel value range.

[0075] Since the pixel value of the pixel point in the current infrared image and the temperature of the object in the actual scene corresponding to the pixel point satisfy the preset temperature pixel value relationship, a higher temperature range (for example, a temperature higher than 200 degrees Celsius) can be set as the preset infrared image pixel value range to determine whether there is an object in the actual scene corresponding to the current infrared image that is in an abnormally high temperature state. If so, it can be preliminarily determined that there is an object on fire in the scene.

[0076] Step S111: If yes, it means that there is a target area indicating that a fire is occurring in the current infrared image.

[0077] Step S112: If not, it means that there is no target area indicating that a fire is occurring in the current infrared image.

[0078] The above are the specific steps of step S100. Through the above steps, it can be determined whether there is a target area in the current infrared image, thereby preparing for the subsequent acquisition of regional data of the target area.

[0079] Step S200: If yes, then obtain the area data of the target area in the current infrared image.

[0080] Among them, the regional data is the position data of the target area in the current infrared image when the target area appears completely in the current infrared image, as well as the current elevation data and rotation angle data of the infrared imaging device when acquiring the current image. The elevation data and rotation angle data of the infrared imaging device can be angular displacement data that can be obtained by the infrared imaging device itself. The current elevation data is the value of the angle between the main optical axis of the infrared imaging device and the horizontal plane, and the current rotation angle data is the angle between the projection of the main optical axis of the infrared imaging device on the horizontal plane and its preset initial position.

[0081] Step S300: Calculate the target viewing angle according to the area data combined with a preset viewing angle adjustment algorithm, and output a viewing angle adjustment instruction. The specific steps of the preset viewing angle adjustment algorithm include:

[0082] Step S310: based on the area data, sampling is performed within a preset distance range near the target area to obtain sampling data, where the sampling data includes distance sampling data, elevation sampling data and rotation angle sampling data.

[0083] Among them, the sampling data is data obtained by the sampling module. In this embodiment, the sampling module is a laser ranging device, and the optical path of the laser ranging device is parallel to the optical axis of the infrared imaging device. The distance sampling data is the ranging result obtained by the sampling module. The elevation sampling data and the rotation angle sampling data are the elevation data and the rotation angle data of the infrared imaging device when obtaining the distance sampling data. In particular, in order to reduce the system error caused by the distance between the infrared imaging device and the laser ranging device, the infrared imaging device and the laser ranging device can be integrated, and the adjustment of the rotation angle and the elevation angle can be completed through a unified mechanical device.

[0084] Step S320: determining whether the target area is within a preset range according to the distance sampling data.

[0085] Through step S310, objects near the fire, such as walls around the fire in a building, can be measured by laser ranging, so as to complete the distance between the fire and the laser ranging module. Since the laser ranging module is a part of the fire monitor, the relative position between the laser ranging module and the fire monitor nozzle is a fixed value, thereby estimating the distance between the fire monitor outlet and the fire. The effective range of the fire monitor is a known parameter of the fire monitor. By comparing the estimated value of the distance between the fire monitor outlet and the fire with the effective range of the fire monitor, it can be determined whether the target area is within the preset range.

[0086] Step S321: If not, output a shift instruction, and resample after the shift instruction is executed.

[0087] In this step, the shift instruction may be a control instruction for controlling the automatic movement of the fire water cannon. After the fire water cannon executes the movement instruction, it may output a feedback signal to indicate that the control instruction has been executed. At this time, it may return to step S100 to reacquire the current infrared image and sampling data of the target area behind the fire water cannon. In another embodiment, the shift instruction may also be instruction information for instructing the user of the fire water cannon to move the fire water cannon, such as displaying "Move XX meters in XXX direction" or other similar prompt information on a display device. After executing the corresponding instruction, the operator may manually input a feedback instruction indicating the completion of the movement through an input device to indicate the completion of the movement. After receiving the feedback instruction, it may return to step S100 to reacquire the current infrared image and sampling data of the target area behind the fire water cannon.

[0088] Step S322: reacquire at least three sets of sampling data and calculate the target surface data.

[0089] When executing this step, the viewing angle adjustment and position determination have been completed. By executing this step, at least three samples are taken within a preset distance range near the target area to obtain at least three groups of sampling data. During the sampling process, the rotation angle and elevation angle of the infrared imaging device and the laser ranging device need to be fine-tuned within a preset range to obtain sampling data of at least three sampling points around the target area. Since the shape of the exterior wall of a building is usually relatively regular, the local area is a vertical plane or can be approximated as a vertical plane. In this embodiment, three samples are taken within a preset distance range near the target area to determine the vertical plane. However, since the distance sampling data, elevation sampling data, and rotation angle sampling data in the sampling data are coordinate data with the location of the infrared imaging device as the coordinate origin, coordinate transformation is required.

[0090] Step S330: performing coordinate transformation on the target surface data according to a preset coordinate transformation equation to obtain transformed target surface data.

[0091] Among them, the coordinate transformation equation is to transform the sampling data in spherical coordinates into the coordinate data of three sampling points in spatial rectangular coordinates with the position of the infrared imaging device as the origin. The coordinate transformation equation is as follows: Figure 2 As shown, r is the distance sampling data, θ is the rotation angle sampling data, φ is the elevation angle sampling data, x, y, z are the horizontal, vertical and vertical coordinates of the transformed sampling points. After substituting into the equation, the coordinates of the three sampling points in the plane rectangular coordinate system (x1, y1, z1), (x2, y2, z2) and (x3, y3, z3) can be obtained; since three points in space can determine a plane, the coordinate data of the three sampling points in the space rectangular coordinate system with the position of the infrared imaging device as the origin is the target surface data.

[0092] Step S340: Calculate the transformed target surface data according to the preset center of gravity solution equation to obtain target point data.

[0093] Specifically, since the target surface data has been obtained in step S330, the three sampling points corresponding to the target surface data can form a triangular area that includes at least part of the target area. The coordinates of the three points in the target surface data are input as follows: Figure 3 The formula, where x m ,y m 、z m are respectively the horizontal coordinate, vertical coordinate and ordinate of the centroid of the triangular area. After substituting the parameters into the data, the centroid coordinate data of the triangular area can be obtained. In this embodiment, the centroid coordinate data is the target point data.

[0094] Step S350: Calculate the target point data according to a preset viewing angle transformation equation to obtain a target viewing angle adjustment angle.

[0095] In one embodiment, the fire monitor and the laser distance measuring device are arranged at different positions. Therefore, it is necessary to transform the coordinates of the target point in the spatial rectangular coordinate system with the laser distance measuring device as the coordinate origin into the coordinates in the spherical coordinate system with the position of the fire monitor as the coordinate origin. To complete the transformation, it is necessary to first transform the coordinates of the target point in the spatial rectangular coordinate system with the laser distance measuring device as the coordinate origin (x m ,y m , z m ) is transformed into the coordinates (x t ,y t , z t ), and then the coordinates (x t ,y t , z t ) is transformed into the coordinates of the spherical coordinate system with the location of the fire monitor as the origin (r t ,θ t ,φ t ), the perspective transformation equation is as follows Figure 4 and 5 As shown, α is the angle between the horizontal axis of the spatial rectangular coordinate system with the laser ranging device as the coordinate origin and the horizontal axis of the spatial rectangular coordinate system with the position of the fire monitor as the coordinate origin. TraX, TraY, and TraZ are the offsets of the horizontal, vertical, and vertical coordinates between the spatial rectangular coordinate systems with two different origins. The four parameters α, TraX, TraY, and TraZ are all determined by the relative position between the fire monitor and the laser ranging device. Once the relative position between the fire monitor and the laser ranging device is determined, the four parameters can be determined. Substitute the relevant parameters into Figure 4 and Figure 5 After the equation shown, the obtained r t ,θ t ,φ t That is, the reference point coordinates (r t ,θ t ,φ t ), especially θ t is the rotation angle of the fire monitor, φ t is the elevation angle of the fire monitor.

[0096] Step S400: acquiring real-time image information when the water jet is being sprayed.

[0097] Through steps S100-S300, the target area has been confirmed and the parameters of the water monitor have been adjusted, that is, the preparation work before the water monitor sprays water has been completed, and the water spraying and fire extinguishing operation can be carried out. Therefore, it can be understood that before executing step S400, the fire water monitor has started the water spraying and fire extinguishing operation after program control or manual operation, and in this step, the real-time image of the water monitor spraying water is obtained, aiming to analyze the actual state of the actual water flow, so as to adjust the elevation angle, rotation angle and other states of the fire water monitor, so that the water flow can be sprayed into the target area more accurately.

[0098] Step S500: Analyze the real-time image information to obtain target area information indicating the fire occurrence area in the image information. The real-time image information is video stream information obtained by a color camera. Since flames usually have flow characteristics in images, real-time image information in the form of video stream information is more likely to reflect this feature and is also easier to distinguish from other static objects such as buildings in the image. Therefore, the method of identifying flames from video stream information has a higher recognition accuracy. In one embodiment, the specific steps of step S500 may include:

[0099] Step S510: randomly extracting a number of frames of images from the real-time image information as sample images.

[0100] Step S520: traverse all sample images and determine whether there are feature pixels in the sample images.

[0101] The characteristic pixel is a pixel whose pixel value is within a preset sample image pixel range.

[0102] Step S521: If yes, then perform an intersection operation on the position information of the characteristic pixels in all the sample images to obtain the position information of the characteristic pixels in all the sample images as the target area information.

[0103] Through steps S510-S530, the real-time image information is first framed, and the frame extraction method can be random frame extraction within a preset time range, or uniform frame extraction within the preset time range; after the frame extraction is completed and the sample image is defined, feature recognition is performed on each sample image, and the object of feature recognition is a color object. The recognition method is to traverse each pixel in the sample image and determine the position of each pixel point that meets the preset sample image pixel range in the sample image. The preset sample image pixel range is the RGB value range that usually represents the flame in the color image. After color screening is performed on each sample image and its corresponding position information is extracted, the intersection of the extracted position information of all sample images is calculated, that is, the position information that is confirmed in each sample image, the position information is the area where the flame exists with a high degree of credibility, that is, the target area that needs to be extinguished, and the position information of the pixel points in the target area is the target area information.

[0104] Step S600: Analyze the real-time image information to obtain landing point information representing the landing point of the jet water flow in the current image information.

[0105] The landing point information represents the position information of the location of the water jet landing point in the current image, and the specific steps include:

[0106] Step S610: extracting the water jet trajectory from the image information through a preset U-net neural network, wherein the U-net neural network is a network obtained by adding an attention module between the downsampling layer and the upsampling layer on the basis of the existing U-net neural network.

[0107] Step S620: binarizing the jet trajectory to generate a binary image.

[0108] Step S630: taking the lowest point of the jet trajectory in the binary image as the jet starting point, extracting the point farthest from the jet starting point as the jet landing point, and taking the position information of the jet landing point as the landing point information.

[0109] Through the above steps, the U-net neural network can be used to obtain the jet trajectory of the water beam, and the pixel area represented as the water beam in the current image is identified and extracted, and finally fitted into a curve representing the water jet trajectory. The point at the end of the curve representing the landing point of the water flow is the landing point of the jet, and the position information of the pixel point corresponding to the landing point in the real-time image information is the landing point information.

[0110] Step S700: Determine whether the landing point information matches the target area information.

[0111] Among them, the specific method of judging the trajectory information and the target area information can be to match the landing point information with the target area information. Since the landing point information and the target area information are both position information representing the position of the pixel point in the current image information, it is only necessary to judge whether the position coordinates of the landing point information are located within the position included in the target area information or within the area surrounded by the target area to complete the conditional judgment. Specifically, if the position coordinates of the landing point information are located within the position coordinates included in the target area information or within the area surrounded by the target area, it indicates that the landing point information matches the target area information; if the position coordinates of the landing point information are not located within the position coordinates included in the target area information or within the area surrounded by the target area, it indicates that the landing point information does not match the target area information.

[0112] Step S800: If not, calculating the trajectory information and the target area according to a preset angle adjustment algorithm to obtain angle adjustment information.

[0113] Among them, the preset angle adjustment algorithm is: calculate the target area information corresponding to the landing point information and its closest target area, and obtain the spatial position relationship between the landing point information and the landing point information based on the target area information corresponding to the landing point information and its closest target area; then adjust the elevation angle and rotation angle of the fire water monitor according to the spatial position relationship.

[0114] Step S900: Generate and output an angle adjustment instruction according to the angle adjustment information.

[0115] Through step S800, the positional relationship between the pixel point represented by the landing point information and the corresponding area of ​​the target area in the figure has been obtained, and the angle adjustment instruction can be generated according to the positional relationship. For example, if the point closest to the landing point in the target area is at the lower right of the landing point, the output indicates the control instruction of reducing the elevation angle of the water cannon and turning to the right; for example, if the point closest to the landing point in the target area is at the upper left of the landing point, the output indicates the control instruction of expanding the elevation angle of the water cannon and turning to the left. In one embodiment, after executing step S900, it can return to step S700 to adjust the landing point of the water flow in real time to improve the influence of factors such as wind and vibration during the operation of the fire water cannon on the landing point of the fire water cannon in actual scenes, so as to achieve a better fire extinguishing effect.

[0116] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0117] Based on the same inventive concept, the embodiment of the present application also provides a machine vision-based fire water cannon system for implementing the machine vision-based fire water cannon control method mentioned above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more machine vision-based fire water cannon system embodiments provided below can refer to the limitations of the machine vision-based fire water cannon control method above, and will not be repeated here.

[0118] In one embodiment, a fire water monitor system based on machine vision is provided, comprising: an image acquisition module, an analysis and calculation module, and a control module, wherein:

[0119] Image acquisition module, specifically including:

[0120] A real-time image acquisition submodule, used to acquire real-time image information. In this embodiment, the real-time image is color video information, and the real-time image acquisition module can be a camera device such as a color camera that can acquire a color video stream;

[0121] The infrared image acquisition submodule is used to acquire the current infrared image. In this embodiment, the current infrared image is picture information, and the infrared image acquisition module is a device such as an infrared camera that can acquire infrared thermal imaging pictures;

[0122] In a more preferred embodiment, the image acquisition module is a photographic device that can capture both color images and infrared images, and its functions can be switched and time-division multiplexed according to the image types required by other modules in the system.

[0123] An analysis and calculation module, used to analyze the real-time image information to obtain target area information and landing point information, determine whether the trajectory information matches the target information, and calculate angle adjustment information according to a preset angle adjustment algorithm;

[0124] The control module is used to generate and output angle adjustment instructions according to the angle adjustment information.

[0125] Each module in the fire water cannon system based on machine vision can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0126] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a fire water cannon control method based on machine vision is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0127] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0129] Step S100: Acquire a current infrared image, and determine whether there is a target area indicating that a fire is occurring in the current infrared image based on pixel value data in the current infrared image.

[0130] Step S200: If yes, then obtain the area data of the target area in the current infrared image.

[0131] Step S300: Calculate a target viewing angle adjustment angle according to the area data in combination with a preset viewing angle adjustment algorithm, and output a viewing angle adjustment instruction.

[0132] Step S400: acquiring real-time image information when the water jet is being sprayed.

[0133] Step S500: Analyze the real-time image information to obtain target area information representing the fire occurrence area in the image information.

[0134] Step S600: Analyze the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information.

[0135] Step S700: Determine whether the landing point information matches the target area information.

[0136] Step S800: If not, calculating the trajectory information and the target area according to a preset angle adjustment algorithm to obtain angle adjustment information.

[0137] Step S900: Generate and output an angle adjustment instruction according to the angle adjustment information.

[0138] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0139] Step S100: Acquire a current infrared image, and determine whether there is a target area indicating that a fire is occurring in the current infrared image based on pixel value data in the current infrared image.

[0140] Step S200: If yes, then obtain the area data of the target area in the current infrared image.

[0141] Step S300: Calculate a target viewing angle adjustment angle according to the area data in combination with a preset viewing angle adjustment algorithm, and output a viewing angle adjustment instruction.

[0142] Step S400: acquiring real-time image information when the water jet is being sprayed.

[0143] Step S500: Analyze the real-time image information to obtain target area information representing the fire occurrence area in the image information.

[0144] Step S600: Analyze the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information.

[0145] Step S700: Determine whether the landing point information matches the target area information.

[0146] Step S800: If not, calculating the trajectory information and the target area according to a preset angle adjustment algorithm to obtain angle adjustment information.

[0147] Step S900: Generate and output an angle adjustment instruction according to the angle adjustment information.

[0148] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0149] Step S100: Acquire a current infrared image, and determine whether there is a target area indicating that a fire is occurring in the current infrared image based on pixel value data in the current infrared image.

[0150] Step S200: If yes, then obtain the area data of the target area in the current infrared image.

[0151] Step S300: Calculate a target viewing angle adjustment angle according to the area data in combination with a preset viewing angle adjustment algorithm, and output a viewing angle adjustment instruction.

[0152] Step S400: acquiring real-time image information when the water jet is being sprayed.

[0153] Step S500: Analyze the real-time image information to obtain target area information representing the fire occurrence area in the image information.

[0154] Step S600: Analyze the real-time image information to obtain landing point information representing the landing point of the jet water flow in the image information.

[0155] Step S700: Determine whether the landing point information matches the target area information.

[0156] Step S800: If not, calculating the trajectory information and the target area according to a preset angle adjustment algorithm to obtain angle adjustment information.

[0157] Step S900: Generate and output an angle adjustment instruction according to the angle adjustment information.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0159] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0160] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fire water monitor control method based on machine vision, characterized in that: The method comprises: Step S100: obtaining a current infrared image, and judging whether there is a target area indicating that a fire is occurring in the current infrared image according to pixel value data in the current infrared image; wherein the current infrared image is infrared picture information obtained by an infrared imaging device; Step S200: If yes, obtaining the area data of the target area in the current infrared image; Step S300: Calculate the target viewing angle according to the area data combined with a preset viewing angle adjustment algorithm, and output a viewing angle adjustment instruction; wherein the specific steps of the preset viewing angle adjustment algorithm include: Step S310: According to the area data, sampling is performed within a preset distance range near the target area to obtain sampling data, the sampling data including distance sampling data, elevation sampling data and rotation angle sampling data; The sampling data is the data obtained through the sampling module. The sampling module is a laser distance measuring device, which integrates the infrared imaging device and the laser distance measuring device, and completes the adjustment of the rotation angle and elevation angle through a unified mechanical device; Step S320: judging whether the target area is within a preset range according to the distance sampling data; Step S321: if not, output a shift instruction, and resample after the shift instruction is executed; Step S330: performing coordinate transformation on the target surface data according to a preset coordinate transformation equation to obtain transformed target surface data; The coordinate transformation equation is to transform the sampling data in spherical coordinates into the coordinate data of three sampling points in spatial rectangular coordinates with the position of the infrared imaging device as the origin. The coordinate transformation equation is as follows: Where r is the distance sampling data, θ is the rotation angle sampling data, φ is the elevation angle sampling data, x, y, z are the horizontal coordinate, vertical coordinate and vertical coordinate of the transformed sampling point, and the coordinates (x1, y1, z1), (x2, y2, z2) and (x3, y3, z3) of the three sampling points in the plane rectangular coordinate system are obtained after substituting them into the equation; since three points in space determine a plane, the coordinate data of the three sampling points in the space rectangular coordinate system with the location of the infrared imaging device as the origin is the target surface data; Step S340: Calculate the transformed target surface data according to the preset center of gravity solution equation to obtain target point data; Specifically, since the target surface data has been acquired in step S330, the three sampling points corresponding to the target surface data form a triangular area that includes at least part of the target area, and the coordinates of the three points in the target surface data are input as follows: Among them, x m ,y m 、z m are the horizontal coordinate, vertical coordinate and vertical coordinate of the centroid of the triangular area respectively. The centroid coordinate data of the triangular area is obtained by substituting the parameters into the data. In this embodiment, the centroid coordinate data is the target point data. Step S350: Calculating the target point data according to a preset viewing angle transformation equation to obtain a target viewing angle adjustment angle; The fire monitor and the laser distance measuring device are set at different locations. Therefore, it is necessary to transform the target point coordinates in the spatial rectangular coordinate system with the laser distance measuring device as the coordinate origin into the coordinates in the spherical coordinate system with the fire monitor as the coordinate origin. To complete the transformation, it is necessary to first transform the target point coordinates (x m ,y m , z m ) is transformed into the coordinates (x t ,y t , z t ), and then the coordinates (x t ,y t , z t ) is transformed into the coordinates of the spherical coordinate system with the location of the fire monitor as the origin (r t ,θ t ,φ t ), the perspective transformation equation is as follows Among them, α is the angle between the horizontal axis of the spatial rectangular coordinate system with the laser ranging device as the coordinate origin and the horizontal axis of the spatial rectangular coordinate system with the position of the fire water monitor as the coordinate origin. TraX, TraY, and TraZ are the offsets of the horizontal coordinate, vertical coordinate, and vertical coordinate between the spatial rectangular coordinate systems with two different origins. The four parameters α, TraX, TraY, and TraZ are all determined by the relative position between the fire water monitor and the laser ranging device. When the relative position between the fire water monitor and the laser ranging device is determined, the four parameters can be determined. After substituting the relevant parameters into the equation, the obtained r t ,θ t ,φ t That is, the reference point coordinates (r t ,θ t ,φ t ), especially θ t is the rotation angle of the fire monitor, φ t is the elevation angle of the fire monitor; Step S400: acquiring real-time image information when water is sprayed; Step S500: analyzing the real-time image information to obtain target area information indicating a fire occurrence area in the image information; Step S600: analyzing the real-time image information to obtain the landing point information representing the landing point of the jet water flow in the current image information; Step S700: determining whether the landing point information matches the target area information; Step S800: If not, calculating the trajectory information and the target area according to a preset angle adjustment algorithm to obtain angle adjustment information; Step S900: Generate and output an angle adjustment instruction according to the angle adjustment information.

2. The fire water monitor control method based on machine vision according to claim 1 is characterized in that: The specific steps of obtaining the target area information representing the fire occurrence area in the image information include: Randomly extracting a number of frames of images from the image information as sample images; Traversing all sample images and determining whether there are feature pixels in the sample images, wherein the feature pixels are pixels whose pixel values ​​are within a preset sample image pixel range; If so, an intersection operation is performed on the position information of the characteristic pixels in all the sample images to obtain the position information of the characteristic pixels in all the sample images and use it as the target area information.

3. The fire water monitor control method based on machine vision according to claim 2 is characterized in that: The specific steps of obtaining the landing point information representing the landing point of the water jet in the current image information include: Extracting the water jet trajectory from the image information by using a preset U-net neural network, wherein the U-net neural network is a network obtained by adding an attention module between the downsampling layer and the upsampling layer on the basis of an existing U-net network; Binarizing the injection trajectory to generate a binary image; The lowest point of the injection trajectory in the binary image is taken as the injection starting point, the point farthest from the injection starting point is extracted as the injection landing point, and the position information of the injection landing point is taken as the landing point information.

4. The fire water monitor control method based on machine vision according to claim 1 is characterized in that: The specific method of determining whether there is a target area indicating that a fire is occurring in the current infrared image includes: Traversing the current infrared image to determine whether there is a pixel point whose pixel value is within a preset infrared image pixel value range; If so, it means that there is a target area indicating that a fire is occurring in the current infrared image; If not, it means that there is no target area indicating that a fire is occurring in the current infrared image.

5. A fire water monitor system based on machine vision, the system implements the method according to claim 1, characterized in that: The system comprises: An image acquisition module, used for acquiring real-time image information; An analysis and calculation module is used to analyze real-time image information to obtain target area information and landing point information, determine whether the trajectory information matches the target information, and calculate angle adjustment information according to a preset angle adjustment algorithm; The control module is used to generate and output angle adjustment instructions according to the angle adjustment information.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images