Fire extinguishing system, method and device based on image data processing

Abstract

The present disclosure relates to a fire extinguishing system, method and device based on image data processing, belonging to the technical field of device control, the system comprising a smoke detection sensor for detecting smoke concentration in a target scene; a camera for collecting image data about the target scene; a fire extinguishing device for extinguishing the indicated area in the target scene; a controller in communication connection with the smoke detection sensor, camera and fire extinguishing device respectively.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of device control, and more particularly, to an image data processing-based fire extinguishing system, method and device. BACKGROUND

[0002] With the rapid development of technology, some residences, office buildings, libraries and large warehouses are equipped with fire extinguishing systems. The fire extinguishing system can timely give an early warning or take fire extinguishing measures in the event of a fire to reduce personnel accidents or property losses.

[0003] At present, the existing fire extinguishing system can be configured with a smoke detection sensor to give a smoke alarm when the smoke concentration in some scenes is high, or configured with a camera capable of recognizing smoke to recognize smoke in some scenes. However, in some old library or large warehouse scenes, some traditional cameras are configured, which are difficult to recognize smoke, and the cost of replacing these cameras will be greatly increased. SUMMARY

[0004] An object of embodiments of the present disclosure is to provide a new technical solution for fire extinguishing based on image data processing.

[0005] According to a first aspect of the present disclosure, an image data processing-based fire extinguishing system is provided, the system comprising:

[0006] a smoke detection sensor configured to detect a smoke concentration in a target scene;

[0007] a camera configured to collect image data about the target scene;

[0008] a fire extinguishing device configured to extinguish a region indicated in the target scene;

[0009] a controller communicatively connected with the smoke detection sensor, the camera and the fire extinguishing device, the controller being configured to: receive smoke concentration information sent by the smoke detection sensor; when the smoke concentration information reflects that a current smoke concentration of the target scene is higher than a set threshold, acquire current image data of the target scene through the camera; extract image features of the current image data and obtain an image feature map; perform sparse processing on the image feature map to obtain a sparse feature map; fuse the image feature map and the sparse feature map to obtain a fused feature map; identify a target region with smoke in the fused feature map through a pre-set smoke model; and feed back the target region and the fused feature map to a user.

[0010] Optionally, the controller is further configured to: perform color sparsity processing on the image feature map by a color sparsity network to obtain a color sparsity feature map; and perform spatial sparsity processing on the color sparsity feature map by a spatial sparsity network to obtain a sparsity feature map.

[0011] Optionally, after the image feature map is processed to obtain the sparsity feature map, the controller is further configured to: determine key feature information of the sparsity feature map by a preset gated feedforward network; and mark the key feature information in the sparsity feature map.

[0012] Optionally, the gated feedforward network comprises an initial convolution layer, a gated convolution layer, and an extraction convolution layer.

[0013] The controller is further configured to: determine a first matrix of features in the sparsity feature map by the initial convolution layer and the gated convolution layer; input the first matrix into a preset activation function to obtain a second matrix; perform element multiplication on the first matrix and the second matrix to obtain a third matrix; and determine key feature information in the third matrix by the extraction convolution layer.

[0014] According to a second aspect of the present disclosure, a fire extinguishing method based on image data processing is also provided, which is based on the fire extinguishing system based on image data processing as described in the first aspect, and the execution subject of the fire extinguishing method based on image data processing is a controller. The method comprises the following steps:

[0015] Receiving smoke concentration information sent by the smoke detection sensor;

[0016] When the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, acquiring current image data of the target scene by the camera;

[0017] Extracting image features of the current image data to obtain an image feature map; and performing sparsity processing on the image feature map to obtain a sparsity feature map;

[0018] Fusing the image feature map and the sparsity feature map to obtain a fusion feature map; and identifying a target region with smoke in the fusion feature map by a preset smoke model;

[0019] Feeding back the target region and the fusion feature map to a user.

[0020] Optionally, the sparsity processing on the image feature map to obtain the sparsity feature map comprises:

[0021] performing color sparsity processing on the image feature map by a color sparsity network to obtain a color sparsity feature map; and

[0022] The color sparse feature map is subjected to spatial sparse processing through a spatial sparse network to obtain a sparse feature map.

[0023] Optionally, after the sparse feature map is obtained, the method further comprises:

[0024] The key feature information of the sparse feature map is determined through a preset gated feedforward network, and the key feature information is marked in the sparse feature map.

[0025] Optionally, the determination of the key feature information of the sparse feature map through the preset gated feedforward network comprises:

[0026] The first matrix of the features in the sparse feature map is determined through the initial convolutional layer and the gated convolutional layer.

[0027] The first matrix is input into a preset activation function to obtain a second matrix, and the first matrix and the second matrix are subjected to element multiplication to obtain a third matrix.

[0028] The key feature information in the third matrix is determined through the extraction convolutional layer.

[0029] According to a third aspect of the present disclosure, there is also provided a fire extinguishing device based on image data processing, comprising:

[0030] A receiving module is configured to receive smoke concentration information sent by the smoke detection sensor.

[0031] An obtaining module is configured to, when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, obtain current image data of the target scene through the camera.

[0032] An obtaining module is configured to extract image features of the current image data and obtain an image feature map, and to perform sparse processing on the image feature map to obtain a sparse feature map.

[0033] An identifying module is configured to fuse the image feature map and the sparse feature map to obtain a fused feature map, and to identify a target region with smoke in the fused feature map through a preset smoke model.

[0034] A control module is configured to control the fire extinguishing device to perform a fire extinguishing operation on the target region.

[0035] According to a third aspect of the present disclosure, there is also provided a fire extinguishing device based on image data processing, comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to execute the computer program to implement the method according to the second aspect of the present disclosure.

[0036] According to a fourth aspect of the present disclosure, a computer readable storage medium is also provided, on which a computer program is stored, the computer program, when executed by a processor, implements the method according to the second aspect of the present disclosure.

[0037] An advantage of the present embodiment is that the controller can receive the smoke concentration information sent by the smoke detection sensor, and when the current smoke concentration is higher than the set threshold, the current image data is obtained through the camera, and the image data is processed to obtain the texture-enhanced fusion feature map. The controller can identify the smoke model in the fusion feature map, and then accurately determine the target area with smoke, and then control the fire extinguishing device to perform fire extinguishing operation on the target area, so as to realize the identification of smoke by using the traditional camera, and without replacing the traditional camera in the scene that needs to be detected, thereby reducing the cost.

[0038] Other features and advantages of the present embodiments will become apparent from the following detailed description of illustrative embodiments thereof, which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present embodiments.

[0040] Figure 1 is a schematic diagram of the composition structure of the fire extinguishing system based on image data processing according to an embodiment;

[0041] Figure 2 is a schematic diagram of the composition structure of the fire extinguishing system based on image data processing according to an embodiment;

[0042] Figure 3 is a schematic diagram of the composition structure of the color sparse network and the spatial sparse network according to an embodiment;

[0043] Figure 4 is a schematic diagram of the composition structure of the gated feed-forward network according to an embodiment;

[0044] Figure 5 is a schematic diagram of the composition structure of the fire extinguishing device based on image data processing according to an embodiment;

[0045] Figure 6 is a schematic diagram of the hardware structure of the fire extinguishing device based on image data processing according to an embodiment. DETAILED DESCRIPTION

[0046] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present disclosure unless otherwise specifically stated.

[0047] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the scope of the application, its application, or uses.

[0048] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0049] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0050] Note that like reference numerals and letters indicate like items in the following drawings and that once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0051] <SYSTEM EMBODIMENT>

[0052] Figure 1 is a schematic diagram of a composition structure of a fire extinguishing system capable of applying image data processing based on an embodiment. As Figure 1 shown, the system includes a smoke detection sensor 100, a camera 200, a fire extinguishing device 300, and a controller 400, which is applied to a large warehouse, a large logistics station, or a port scene.

[0053] As Figure 1 shown, a smoke detection sensor 100, a camera 200, and a fire extinguishing device 300 can be provided in each scene. That is, in scenes A-D, smoke detection sensors 100A-D, cameras 200A-D, and fire extinguishing devices 300A-D are respectively provided.

[0054] In this application, the fire extinguishing device 300 can have one or more fire extinguishing agents such as carbon dioxide, foam, or clean water halon replacement, and the fire extinguishing device 300 can perform point targeting fire extinguishing on each region in the scene. For example, regions 1-9 are provided in the scene, and the fire extinguishing device 300 can open the valve switch of the corresponding region, so that the fire extinguishing agent in the fire extinguishing device 300 can be sprayed to the corresponding region.

[0055] In this application, the fire extinguishing device 300 can be a projection device that uses a configured conical cover nozzle to project the fire extinguishing agent of the corresponding region.

[0056] The smoke detection sensor 100 is configured to detect the smoke concentration in the target scene; the camera 200 is configured to collect image data of the target scene; and the controller 400 is in communication connection with the smoke detection sensor 100, the camera 200 and the fire extinguishing device 300.

[0057] In the present application, the smoke detection sensor 100 can be arranged on the roof or the side wall of a house in a scene, and the camera 200 can also be arranged on the roof or the side wall of a house in a scene.

[0058] The controller 400 is configured to receive the smoke concentration information sent by the smoke detection sensor 100; when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, obtain the current image data of the target scene through the camera 200; extract the image features of the current image data and obtain the image feature map; perform sparse processing on the image feature map to obtain the sparse feature map; fuse the image feature map and the sparse feature map to obtain the fusion feature map; identify the target region with smoke in the fusion feature map through the preset smoke model; and control the fire extinguishing device 300 to perform fire extinguishing operation on the target region.

[0059] In the present application, the set threshold is, for example, 100 ppm, 150 ppm or 200 ppm, etc., which is not limited herein.

[0060] In the present application, the image data obtained by the camera 200 can be an image with low resolution, such as an image with a resolution of 352x288, 704x576, 1280x720, etc.

[0061] In the present application, the controller 400 can extract the image features of the current image data through a convolution layer (3x3 convolution layer) and obtain the image feature map.

[0062] In the present application, the detail features in the image features can reflect the detail features of each tile in the image data, and the overall features in the image features can reflect the overall features of the overall effect of the image data.

[0063] In the present application, as shown in Figure 3 The sparse processing can be sparse processing on the detail features and the overall features in the image features to enhance the texture of the image reflected by the current image data, and thus obtain the sparse feature map. The features of the image feature map extracted through a convolution layer (3x3 convolution layer) and the features extracted from the sparse feature map are fused to obtain the fusion feature map, and a higher resolution image can be obtained through an up-sampling convolution layer.

[0064] In the present application, the smoke model can be obtained by training a large number of smoke samples. Through the smoke model, smoke of different colors and shapes can be identified. Moreover, the controller 400 can determine that the identified smoke appears in a certain area of a certain scene, and control the fire extinguishing device 300 of the scene to perform fire extinguishing operation on the area of the scene.

[0065] In other words, the controller 400 can receive the smoke concentration information sent by the smoke detection sensor 100. When the current smoke concentration is higher than the set threshold, the controller 400 can obtain the current image data through the camera 200, and perform image processing on the current image data to obtain a texture-enhanced fusion feature map. The controller 400 can identify the smoke model in the fusion feature map, and accurately determine the target area with smoke, and then control the fire extinguishing device 300 to perform fire extinguishing operation on the target area, so as to realize the identification of smoke by the traditional camera 200, and reduce the cost without replacing the traditional camera 200 in the need of detecting more scenes.

[0066] In some embodiments, the controller 400 is further configured to: perform color sparsity processing on the image feature map through a color sparsity network to obtain a color sparsity feature map; and perform spatial sparsity processing on the color sparsity feature map through a spatial sparsity network to obtain a sparse feature map.

[0067] In the present application, the color sparsity network is a Colorization Transformer network, and the spatial sparsity network is a Spatial Transformer network.

[0068] In the present application, the color sparsity processing on the image feature map can be slicing the image feature map at a set interval distance, extracting local features in the image feature map through a convolution layer group (1x1 convolution layer and 3x3 convolution layer) in the color sparsity network, calculating a matrix M1 representing the local features, determining values of Query, Key and Value elements of the matrix M1, normalizing the values of the three elements and performing matrix multiplication in the spatial dimension to obtain an attention map P1, inputting the attention map P1 into a ReLu activation function, and performing matrix multiplication with Value in the spatial dimension to obtain a matrix M2, and then performing matrix reshaping on the matrix M2 and extracting the color sparsity feature map through a 1x1 convolution layer.

[0069] In the present application, the spatial sparse processing of the color sparse feature map can be directly calculating a matrix M3 representing overall features, determining the values of Query, Key and Value of the matrix M3, normalizing the values of the three elements and performing matrix multiplication in the color dimension to obtain an attention map P2, screening the top K features based on the Top-K mechanism, inputting the top K features into a ReLu activation function, and performing matrix multiplication in the color dimension with Value to obtain a matrix M4, and then performing matrix reshaping on the matrix M4 and then extracting the sparse feature map through a 1x1 convolution layer.

[0070] In other words, by performing sparse processing on the image feature map, image texture enhancement can be achieved to identify smoke in the image and improve the accuracy of the fire extinguishing system based on image data processing.

[0071] In some embodiments, after the sparse processing of the image feature map to obtain the sparse feature map, the controller 400 is further configured to: determine the key feature information of the sparse feature map through a preset gated feedforward network; and mark the key feature information in the sparse feature map.

[0072] In some embodiments, the gated feedforward network includes an initial convolution layer, a gated convolution layer and an extraction convolution layer.

[0073] The controller 400 is further configured to: determine a first matrix of the features in the sparse feature map through the initial convolution layer and the gated convolution layer; input the first matrix into a preset activation function to obtain a second matrix; perform element multiplication on the first matrix and the second matrix to obtain a third matrix; and determine the key feature information in the third matrix through the extraction convolution layer.

[0074] In the present application, as shown in Figure 4 The gated feedforward network can include an initial convolution layer (1x1 convolution layer), a gated convolution layer (3x3 convolution layer) and an extraction convolution layer (1x1 convolution layer). The initial convolution layer and the gated convolution layer extract a first matrix of the features in the sparse feature map, the first matrix is input into a ReLu activation function to obtain a second matrix, the second matrix and the first matrix are multiplied element by element to obtain a third matrix, and finally the third matrix is extracted through the extraction convolution layer, so that the gated feedforward network can extract the key feature information in the sparse feature map, and the key feature information is the texture information of the sparse feature map.

[0075] In other words, by setting the gated feedforward network, the key texture information in the sparse feature map can be captured, and the interference of noise and redundant data in the sparse feature map can be effectively reduced.

[0076] In the embodiments of the present disclosure, the memory of the controller 400 is configured to store a computer program for controlling the processor of the controller 400 to operate to implement the control method for seawater desalination according to any embodiment. The computer program can be designed according to the scheme of the embodiments of the present disclosure. How the computer program controls the processor to operate is known in the art, and thus will not be described in detail herein.

[0077] <Method Embodiment>

[0078] Figure 2 is a flowchart of a fire extinguishing method based on image data processing according to an embodiment. The implementation subject is, for example, Figure 1 a controller in the embodiment.

[0079] As shown in Figure 2 , the fire extinguishing method based on image data processing in the embodiment can include the following steps S210 to S250:

[0080] Step S210, receiving smoke concentration information sent by a smoke detection sensor.

[0081] Step S220, when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than the set threshold, obtaining current image data of the target scene through a camera.

[0082] Step S230, extracting image features of the current image data and obtaining an image feature map; performing sparse processing on the image feature map to obtain a sparse feature map.

[0083] In some embodiments, step S230 can include the following steps S2301 and S2302:

[0084] Step S2301, performing color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map.

[0085] Step S2302, performing spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain the sparse feature map.

[0086] In some embodiments, after step S230, the method further includes the following step S310:

[0087] Step S310, determining key feature information of the sparse feature map through a pre-set gating feedforward network; marking the key feature information in the sparse feature map.

[0088] In some embodiments, step S310 can include the following steps S3101 to S3013:

[0089] In step S3101, the first matrix of the features in the sparse feature map is determined through the initial convolution layer and the gated convolution layer.

[0090] In step S3102, the first matrix is input into the preset activation function to obtain a second matrix; and the first matrix and the second matrix are multiplied element by element to obtain a third matrix.

[0091] In step S3103, the key feature information in the third matrix is determined through the extraction convolution layer.

[0092] In step S240, the image feature map and the sparse feature map are fused to obtain a fusion feature map; and the target region with smoke in the fusion feature map is identified through the preset smoke model.

[0093] In step S250, the target region is subjected to the fire extinguishing operation by the fire extinguishing device.

[0094] <Device Embodiment One>

[0095] Figure 5 is a principle block diagram of the fire extinguishing device based on image data processing according to an embodiment. As shown in Figure 5 , the fire extinguishing device based on image data processing 30 comprises:

[0096] The receiving module 31 is configured to receive the smoke concentration information sent by the smoke detection sensor.

[0097] The obtaining module 32 is configured to, when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than the set threshold, obtain the current image data of the target scene through the camera.

[0098] The obtaining module 33 is configured to extract the image features of the current image data and obtain an image feature map; and perform sparse processing on the image feature map to obtain a sparse feature map.

[0099] The identification module 34 is configured to fuse the image feature map and the sparse feature map to obtain a fusion feature map; and identify the target region with smoke in the fusion feature map through the preset smoke model.

[0100] The control module 35 is configured to control the fire extinguishing device to perform the fire extinguishing operation on the target region.

[0101] Optionally, the obtaining module 33 is further configured to perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; and perform spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain the sparse feature map.

[0102] Optionally, the image data processing-based fire extinguishing device 30 further comprises a determination module configured to determine key feature information of the sparse feature map by a preset gated feedforward network; and mark the key feature information in the sparse feature map.

[0103] Optionally, the determination module is further configured to determine a first matrix of the features in the sparse feature map by an initial convolution layer and a gated convolution layer; input the first matrix into a preset activation function to obtain a second matrix; perform element multiplication on the first matrix and the second matrix to obtain a third matrix; and determine the key feature information in the third matrix by an extraction convolution layer.

[0104] The image data processing-based fire extinguishing device 30 can be a controller 400.

[0105] <Device Embodiment Two>

[0106] Figure 6 is a hardware structure schematic diagram of an image data processing-based fire extinguishing device according to another embodiment.

[0107] As shown in Figure 6 The image data processing-based fire extinguishing device 40 comprises a processor 41 and a memory 42, the memory 42 is configured to store an executable computer program, and the processor 41 is configured to execute a method according to any method embodiment above under the control of the computer program.

[0108] The modules of the above image data processing-based fire extinguishing device 30 can be implemented by the processor 41 in this embodiment executing the computer program stored in the memory 42, or can be implemented by other structures, which are not limited here.

[0109] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith, which instructions are executable by a processor to implement aspects of the present application.

[0110] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0111] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0112] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0113] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0114] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0115] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0116] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0117] Embodiments of the present application have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the described embodiments are possible, and all such modifications and variations are intended to be within the scope of the described embodiments. The description used herein is intended to best explain the principles of the embodiments, the practical application, and the best mode of the present application, to make this disclosure understood in the art. The scope of the present application is defined by the appended claims.

Claims

1. An image data processing based fire extinguishing system, characterized by, The system comprises: a smoke detection sensor for detecting a smoke concentration in a target scene; a camera for collecting image data about the target scene; a fire extinguishing device for extinguishing a region indicated in the target scene; a controller in communication connection with the smoke detection sensor, the camera and the fire extinguishing device, respectively, and configured to: receive smoke concentration information sent by the smoke detection sensor; when the smoke concentration information reflects that a current smoke concentration of the target scene is higher than a set threshold, acquire current image data of the target scene through the camera; extract image features of the current image data and obtain an image feature map; perform sparse processing on the image feature map to obtain a sparse feature map; fuse the image feature map and the sparse feature map to obtain a fused feature map; identify a target region with smoke in the fused feature map through a preset smoke model; and control the fire extinguishing device to perform a fire extinguishing operation on the target region. The controller is further configured to: perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; perform spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain the sparse feature map; the color sparse processing on the image feature map is slicing the image feature map at a set interval distance, extracting local features in the image feature map through a convolution layer group in the color sparse network, calculating a matrix M1 representing the local features, determining values of three elements Query, Key and Value of the matrix M1, normalizing the values of the three elements and performing matrix multiplication in a spatial dimension to obtain an attention map P1, inputting the attention map P1 into a ReLu activation function and performing matrix multiplication with Value in the spatial dimension to obtain a matrix M2, and then performing matrix reshaping on the matrix M2 and extracting the color sparse feature map through a 1×1 convolution layer; the spatial sparse processing on the color sparse feature map is directly calculating a matrix M3 representing overall features, determining values of three elements Query, Key and Value of the matrix M3, normalizing the values of the three elements and performing matrix multiplication in a color dimension to obtain an attention map P2, filtering out the first K features based on a Top-K mechanism, inputting the first K features into a ReLu activation function and performing matrix multiplication with Value in the color dimension to obtain a matrix M4, and then performing matrix reshaping on the matrix M4 and extracting the sparse feature map through a 1×1 convolution layer.

2. The system of claim 1, wherein, After the sparse processing on the image feature map to obtain the sparse feature map, the controller is further configured to: determine key feature information of the sparse feature map through a preset gated feedforward network; and mark the key feature information in the sparse feature map.

3. The system of claim 2, wherein, The gated feedforward network comprises an initial convolution layer, a gated convolution layer and an extraction convolution layer. The controller is further configured to: determine a first matrix of features in the sparse feature map through the initial convolutional layer and the gated convolutional layer; input the first matrix to a preset activation function to obtain a second matrix; and perform element multiplication on the first matrix and the second matrix to obtain a third matrix; Determine key feature information in the third matrix through the extraction convolutional layer.

4. An image data processing-based fire extinguishing method based on the image data processing-based fire extinguishing system according to any one of claims 1 to 3, the execution subject of the image data processing-based fire extinguishing method being a controller, characterized by, The method comprises: Receiving smoke concentration information sent by the smoke detection sensor; When the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, acquiring current image data of the target scene through the camera; Extracting image features of the current image data and obtaining an image feature map; performing sparse processing on the image feature map to obtain a sparse feature map; Fusing the image feature map and the sparse feature map to obtain a fused feature map; identifying a target region with smoke in the fused feature map through a preset smoke model; Feeding back the target region and the fused feature map to a user; The sparse processing of the image feature map to obtain a sparse feature map comprises: Performing color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; Performing spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain a sparse feature map; the color sparse processing of the image feature map is slicing the image feature map at a set interval distance, extracting local features in the image feature map through a convolutional layer group in the color sparse network, calculating a matrix M1 representing the local features, determining values of Query, Key and Value elements of the matrix M1, normalizing the values of the three elements and performing matrix multiplication in the spatial dimension to obtain an attention map P1, inputting the attention map P1 into a ReLu activation function and performing matrix multiplication with Value in the spatial dimension to obtain a matrix M2, and then performing matrix reshaping on the matrix M2, and extracting the color sparse feature map through a 1×1 convolutional layer; the spatial sparse processing of the color sparse feature map is directly calculating a matrix M3 representing overall features, determining values of Query, Key and Value elements of the matrix M3, normalizing the values of the three elements and performing matrix multiplication in the color dimension to obtain an attention map P2, screening the first K features based on a Top-K mechanism, inputting the first K features into a ReLu activation function, and performing matrix multiplication with Value in the color dimension to obtain a matrix M4, and then performing matrix reshaping on the matrix M4 and extracting the sparse feature map through a 1×1 convolutional layer.

5. The method of claim 4, wherein, After obtaining the sparse feature map, the method further comprises: Determining key feature information of the sparse feature map through a preset gated feedforward network; and marking the key feature information in the sparse feature map.

6. The method of claim 5, wherein, The gated feedforward network comprises an initial convolutional layer, a gated convolutional layer and an extraction convolutional layer; and the determination of the key feature information of the sparse feature map through the preset gated feedforward network comprises: Determine a first matrix of features in the sparse feature map through the initial convolutional layer and the gating convolutional layer; Input the first matrix into a preset activation function to obtain a second matrix; and perform element multiplication on the first matrix and the second matrix to obtain a third matrix; Determine key feature information in the third matrix through the extraction convolutional layer.

7. An image data processing-based fire extinguishing apparatus, characterized by comprising: The device comprises: A receiving module configured to receive smoke concentration information sent by a smoke detection sensor; An obtaining module configured to, when the smoke concentration information reflects that a current smoke concentration of a target scene is higher than a set threshold, obtain current image data of the target scene through a camera; A obtaining module configured to extract image features of the current image data and obtain an image feature map; and perform sparse processing on the image feature map to obtain a sparse feature map; An identifying module configured to fuse the image feature map and the sparse feature map to obtain a fused feature map; and identify a target region with smoke in the fused feature map through a preset smoke model; A control module configured to control the fire extinguishing device to perform a fire extinguishing operation on the target region; The obtaining module is further configured to perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; perform spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain the sparse feature map; the color sparse processing on the image feature map is slicing the image feature map at a set interval distance, extracting local features in the image feature map through a convolutional layer group in the color sparse network, calculating a matrix M1 representing the local features, determining values of Query, Key, and Value of the matrix M1, normalizing the values of the three elements, and performing matrix multiplication in a spatial dimension to obtain an attention map P1, inputting the attention map P1 into a ReLu activation function, and performing matrix multiplication with Value in the spatial dimension to obtain a matrix M2, and then performing matrix reshaping on the matrix M2 and extracting the color sparse feature map through a 1×1 convolutional layer; the spatial sparse processing on the color sparse feature map is directly calculating a matrix M3 representing overall features, determining values of Query, Key, and Value of the matrix M3, normalizing the values of the three elements, and performing matrix multiplication in a color dimension to obtain an attention map P2, filtering out the first K features based on a Top-K mechanism, inputting the first K features into a ReLu activation function, and performing matrix multiplication with Value in the color dimension to obtain a matrix M4, and then performing matrix reshaping on the matrix M4 and extracting the sparse feature map through a 1×1 convolutional layer.

8. An image data processing-based fire extinguishing device, characterized by The device comprises a processor connected with a memory, the memory being configured to store a computer program; and the processor being configured to execute the computer program to implement the method according to any one of claims 4 to 6.

Images