Fire extinguishing system, method and device based on image data processing

Abstract

The invention relates to a fire extinguishing system, method and device based on image data processing, and belongs to the technical field of equipment control, the system comprises a smoke detection sensor, the smoke detection sensor is used for detecting the smoke concentration in a target scene; the camera is used for collecting image data about the target scene; the fire extinguishing device is used for extinguishing fire in an indicated area in the target scene; and the controller is in communication connection with the smoke detection sensor, the camera and the fire extinguishing device.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of equipment control, and more specifically, to a fire extinguishing system, method and device based on image data processing. Background Art

[0002] With the rapid development of technology, some residences, office buildings, libraries and large warehouses are equipped with fire extinguishing systems. Fire extinguishing systems can provide timely warnings or take fire extinguishing measures in the event of a possible fire, thereby reducing personal accidents or property losses.

[0003] At present, the existing fire extinguishing system can be equipped with smoke detection sensors to issue smoke alarms when the smoke concentration in some scenes is high, or equipped with cameras that can identify smoke to identify smoke in some scenes. However, some old libraries or large warehouses are equipped with some traditional cameras that are difficult to identify smoke, and the cost of replacing these cameras will also increase significantly. Summary of the invention

[0004] One purpose of the embodiments of the present disclosure is to provide a new technical solution for fire extinguishing based on image data processing.

[0005] According to a first aspect of the present disclosure, a fire extinguishing system based on image data processing is provided, the system comprising: A smoke detection sensor, wherein the smoke detection sensor is used to detect the smoke concentration in the target scene; A camera, the camera being used to collect image data about the target scene; A fire extinguishing device, the fire extinguishing device is used to extinguish a fire in an area indicated in the target scene; A controller, wherein the controller is communicatively connected with the smoke detection sensor, the camera and the fire extinguishing device respectively, and the controller is configured to: receive smoke concentration information sent by the smoke detection sensor; when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, obtain current image data of the target scene through the camera; extract image features of the current image data and obtain an image feature map; perform sparse processing on the image feature map to obtain a sparse feature map; fuse the image feature map and the sparse feature map to obtain a fused feature map; identify a target area with smoke in the fused feature map through a preset smoke model; and feed back the target area and the fused feature map to a user.

[0006] Optionally, the controller is further configured to: perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; and perform spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain a sparse feature map.

[0007] Optionally, after performing sparse processing on the image feature map to obtain a sparse feature map, the controller is further configured to: determine key feature information of the sparse feature map through a preset gated feedforward network; and mark the key feature information in the sparse feature map.

[0008] Optionally, the gated feedforward network includes an initial convolutional layer, a gated convolutional layer and an extraction convolutional layer; The controller is also configured to: determine a first matrix of features in the sparse feature map through the initial convolution layer and the gated convolution layer; input the first matrix into a preset activation function to obtain a second matrix; perform element-wise multiplication of the first matrix and the second matrix to obtain a third matrix; and determine key feature information in the third matrix through the extraction convolution layer.

[0009] According to a second aspect of the present disclosure, a fire extinguishing method based on image data processing is further provided. The fire extinguishing method based on image data processing is based on the fire extinguishing system based on image data processing as described in the first aspect. The execution subject of the fire extinguishing method based on image data processing is a controller. The method includes: Receiving smoke concentration information sent by the smoke detection sensor; When the smoke density information reflects that the current smoke density of the target scene is higher than a set threshold, obtaining current image data of the target scene through the camera; Extracting image features of current image data and obtaining an image feature map; performing sparse processing on the image feature map to obtain a sparse feature map; The image feature map and the sparse feature map are fused to obtain a fused feature map; a target area with smoke in the fused feature map is identified by a preset smoke model; The target area and the fused feature map are fed back to the user.

[0010] Optionally, performing sparse processing on the image feature map to obtain a sparse feature map includes: Performing color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; The color sparse feature map is spatially sparsely processed through a spatial sparse network to obtain a sparse feature map.

[0011] Optionally, after obtaining the sparse feature map, the method further includes: The key feature information of the sparse feature map is determined through a preset gated feedforward network; and the key feature information is marked in the sparse feature map.

[0012] Optionally, determining key feature information of the sparse feature map by a preset gated feedforward network includes: Determining a first matrix of features in the sparse feature map through the initial convolution layer and the gated convolution layer; Inputting the first matrix into a preset activation function to obtain a second matrix; performing element-wise multiplication of the first matrix and the second matrix to obtain a third matrix; By extracting the convolutional layer, key feature information in the third matrix is ​​determined.

[0013] According to a third aspect of the present disclosure, there is also provided a fire extinguishing device based on image data processing, the device comprising: A receiving module, used for receiving smoke concentration information sent by the smoke detection sensor; an acquisition module, configured to acquire current image data of the target scene through the camera when the smoke density information reflects that the current smoke density of the target scene is higher than a set threshold; The obtaining module is used to extract the image features of the current image data and obtain an image feature map; the image feature map is subjected to sparse processing to obtain a sparse feature map; A recognition module is used to fuse the image feature map and the sparse feature map to obtain a fused feature map; and to identify a target area with smoke in the fused feature map through a preset smoke model; A control module is used to control the fire extinguishing device to perform fire extinguishing operations on the target area.

[0014] According to a third aspect of the present disclosure, there is also provided a fire extinguishing device based on image data processing, comprising a memory and a processor, wherein the memory is used to store a computer program; the processor is used to execute the computer program to implement the method according to the second aspect of the present disclosure.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method according to the second aspect of the present disclosure is implemented.

[0016] One beneficial effect of the disclosed embodiment is that the controller is able to receive smoke concentration information sent by the smoke detection sensor, and when the current smoke concentration is higher than a set threshold, the current image data is obtained through the camera, and image processing is performed on the current image data to obtain a texture-enhanced fusion feature map. The controller can identify the smoke model in the fusion feature map, and then accurately determine the target area with smoke, and then control the fire extinguishing device to perform fire extinguishing operations on the target area, so as to realize the use of traditional cameras to identify smoke, so that there is no need to replace traditional cameras in scenes that need to detect more, thereby reducing costs.

[0017] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0019] Figure 1 is a schematic diagram of the composition structure of a fire extinguishing system based on image data processing according to an embodiment; Figure 2 is a flow chart of a fire extinguishing method based on image data processing according to an embodiment; Figure 3 is a schematic diagram of the composition structure of a color sparse network and a spatial sparse network according to an embodiment; Figure 4 is a schematic diagram of the composition structure of a gated feedforward network according to an embodiment; Figure 5 is a block diagram of a fire extinguishing device based on image data processing according to an embodiment; Figure 6 It is a schematic diagram of the hardware structure of a fire extinguishing device based on image data processing according to one embodiment. DETAILED DESCRIPTION

[0020] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0021] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0022] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0023] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0024] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0025] <System Embodiment> Figure 1 FIG. 1 is a schematic diagram of the structure of a fire extinguishing system based on image data processing according to an embodiment. Figure 1 As shown, the system includes a smoke detection sensor 100, a camera 200, a fire extinguishing device 300 and a controller 400. The system is applied to scenes such as large warehouses, large logistics stations or ports.

[0026] like Figure 1 As shown, a smoke detection sensor 100, a camera 200 and a fire extinguishing device 300 can be set in each scene. That is, in scenes A to D, smoke detection sensors 100A-D, cameras 200A-D and fire extinguishing devices 300A-D are respectively set.

[0027] In the present application, the fire extinguishing device 300 may be equipped with one or more fire extinguishing agents such as carbon dioxide, foam or clean water halon substitute. The fire extinguishing device 300 may perform targeted fire extinguishing on various areas in the scene. For example, areas 1-9 are set in the scene, and the fire extinguishing device 300 may open the valve switch of the corresponding area so that the fire extinguishing agent in the fire extinguishing device 300 may be sprayed to the corresponding area.

[0028] In the present application, the fire extinguishing device 300 may be a projectile device that uses a configured conical hood nozzle to project the fire extinguishing agent in the corresponding area.

[0029] The smoke detection sensor 100 is used to detect the smoke concentration in the target scene; the camera 200 is used to collect image data about the target scene; and the controller 400 is respectively connected to the smoke detection sensor 100, the camera 200 and the fire extinguishing device 300 for communication.

[0030] In the present application, the smoke detection sensor 100 can be set on the roof or the side wall of a house in a scene, and the camera 200 can also be set on the roof or the side wall of a house in a scene.

[0031] The controller 400 is configured to: receive smoke concentration information sent by the smoke detection sensor 100; when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, obtain current image data of the target scene through the camera 200; extract image features of the current image data and obtain an image feature map; perform sparse processing on the image feature map to obtain a sparse feature map; fuse the image feature map and the sparse feature map to obtain a fused feature map; identify a target area with smoke in the fused feature map through a preset smoke model; and control the fire extinguishing device 300 to perform a fire extinguishing operation on the target area.

[0032] In the present application, the threshold value is set to, for example, 100 ppm, 150 ppm or 200 ppm, etc., which is not limited here.

[0033] In the present application, the image data acquired by the camera 200 may be an image with a lower resolution, for example, an image with a resolution of 352×288, 704×576, 1280×720, etc.

[0034] In the present application, the controller 400 can extract image features of current image data through a convolution layer (3×3 convolution layer) and obtain an image feature map.

[0035] In the present application, the detail features in the image features can reflect the detail features of each block in the image data, and the overall features in the image features can reflect the overall features of the overall effect of the image data.

[0036] In this application, if Figure 3 As shown in the figure, sparse processing can be performed on the detail features and overall features in the image features to enhance the texture of the image reflected by the current image data, thereby obtaining a sparse feature map. The features of the image feature map extracted by a convolution layer (3×3 convolution layer) are then fused with the features extracted from the sparse feature map to obtain a fused feature map, and then a higher resolution image can be obtained by upsampling the convolution layer.

[0037] In the present application, the smoke model can be obtained by training a large number of smoke samples, and smoke of different colors and shapes can be identified through the smoke model. In addition, the controller 400 can determine that the identified smoke appears in a certain area of ​​a certain scene, and control the fire extinguishing device 300 of the scene to perform a fire extinguishing operation on the area of ​​the scene.

[0038] In other words, the controller 400 is able to receive the smoke concentration information sent by the smoke detection sensor 100. When the current smoke concentration is higher than the set threshold, the current image data is obtained through the camera 200, and the current image data is processed to obtain a texture-enhanced fusion feature map. The controller 400 can identify the smoke model in the fusion feature map, and then accurately determine the target area with smoke, and then control the fire extinguishing device 300 to perform a fire extinguishing operation on the target area, so as to realize the use of the traditional camera 200 to identify the smoke, so that in scenes that need to detect more, there is no need to replace the traditional camera 200, thereby reducing costs.

[0039] In some embodiments, the controller 400 is further configured to: perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; perform spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain a sparse feature map.

[0040] In this application, the color sparse network is the Colorization Transformer network, and the spatial sparse network is the Spatial Transformer network.

[0041] In the present application, color sparse processing of an image feature map can be performed by slicing the image feature map according to a set interval distance, extracting local features in the image feature map through a convolutional layer group (1×1 convolutional layer and 3×3 convolutional layer) in a color sparse network, and calculating a matrix M1 representing these local features, and determining the values ​​of the three elements Query, Key and Value of the matrix M1, normalizing the values ​​of these three elements and performing matrix multiplication in the spatial dimension to obtain an attention map P1, inputting the attention map P1 into a ReLu activation function, and performing matrix multiplication with Value in the spatial dimension to obtain a matrix M2, and then reshaping the matrix M2 and extracting it into a color sparse feature map through a 1×1 convolutional layer.

[0042] In the present application, spatial sparse processing of the color sparse feature map can be to directly calculate the matrix M3 representing the overall features, and determine the values ​​of the three elements Query, Key and Value of the matrix M3, normalize the values ​​of these three elements and perform matrix multiplication on the color dimension to obtain the attention map P2, and then based on the Top-K mechanism, screen out the first K features, input the first K features into the ReLu activation function, and perform matrix multiplication with Value on the color dimension to obtain the matrix M4, and then reshape the matrix M4 and extract it into the sparse feature map through a 1×1 convolution layer.

[0043] In other words, by performing sparse processing on the image feature map, image texture enhancement can be achieved to identify smoke in the image and improve the accuracy of fire extinguishing by the fire extinguishing system based on image data processing.

[0044] In some embodiments, after performing sparse processing on the image feature map to obtain the sparse feature map, the controller 400 is further configured to: determine key feature information of the sparse feature map through a preset gated feedforward network; and mark the key feature information in the sparse feature map.

[0045] In some embodiments, the gated feed-forward network includes an initial convolutional layer, a gated convolutional layer, and an extractive convolutional layer; The controller 400 is also configured to: determine a first matrix of features in a sparse feature map through an initial convolution layer and a gated convolution layer; input the first matrix into a preset activation function to obtain a second matrix; perform element-wise multiplication of the first matrix and the second matrix to obtain a third matrix; and determine key feature information in the third matrix by extracting the convolution layer.

[0046] In this application, if Figure 4 As shown, the gated feedforward network may include an initial convolution layer (1×1 convolution layer), a gated convolution layer (3×3 convolution layer) and an extraction convolution layer (1×1 convolution layer). The first matrix of features in the sparse feature map is extracted through the initial convolution layer and the gated convolution layer, and the first matrix is ​​input into the ReLu activation function to obtain the second matrix, and then the second matrix and the first matrix are element-wise multiplied to obtain the third matrix, and finally the third matrix is ​​subjected to feature extraction through the extraction convolution layer, so that the gated feedforward network can extract the key feature information in the sparse feature map, and the key feature information is the texture information of the sparse feature map.

[0047] In other words, by setting up a gated feedforward network, the key texture information in the sparse feature map can be captured, effectively reducing the interference of noise and redundant data in the sparse feature map.

[0048] As used in the embodiments of the present disclosure, the memory of the controller 400 is used to store a computer program, which is used to control the processor of the controller 400 to operate to implement the control method of seawater desalination according to any embodiment. A technician can design a computer program according to the scheme of the embodiments of the present disclosure. How the computer program controls the processor to operate is well known in the art, so it will not be described in detail here.

[0049] <Method Example> Figure 2 is a flow chart of a fire extinguishing method based on image data processing according to an embodiment. Figure 1 Middle controller.

[0050] like Figure 2 As shown, the fire extinguishing method based on image data processing in this embodiment may include the following steps S210 to S250: Step S210, receiving smoke concentration information sent by the smoke detection sensor.

[0051] Step S220, when the smoke density information reflects that the current smoke density of the target scene is higher than the set threshold, the current image data of the target scene is acquired through the camera.

[0052] Step S230, extracting image features of the current image data and obtaining an image feature map; performing sparse processing on the image feature map to obtain a sparse feature map.

[0053] In some embodiments, step S230 may include the following steps S2301 and S2302: Step S2301, performing color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map.

[0054] Step S2302, performing spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain a sparse feature map.

[0055] In some embodiments, after step S230, the method further includes the following step S310: Step S310, determining key feature information of the sparse feature map through a preset gated feedforward network; marking the key feature information in the sparse feature map.

[0056] In some embodiments, step S310 may include the following steps S3101 to S3013: Step S3101, determining a first matrix of features in a sparse feature map through an initial convolution layer and a gated convolution layer.

[0057] Step S3102, input the first matrix into a preset activation function to obtain a second matrix; perform element-wise multiplication of the first matrix and the second matrix to obtain a third matrix.

[0058] Step S3103, determining key feature information in the third matrix by extracting the convolution layer.

[0059] Step S240, fusing the image feature map and the sparse feature map to obtain a fused feature map; using a preset smoke model, identifying a target area with smoke in the fused feature map.

[0060] Step S250, controlling the fire extinguishing device to perform fire extinguishing operations on the target area.

[0061] <Equipment Example 1> Figure 5 FIG. 1 is a block diagram of a fire extinguishing device based on image data processing according to an embodiment. Figure 5 As shown, the fire extinguishing device 30 based on image data processing includes: The receiving module 31 is used to receive the smoke concentration information sent by the smoke detection sensor; The acquisition module 32 is used to acquire current image data of the target scene through a camera when the smoke density information reflects that the current smoke density of the target scene is higher than a set threshold; The obtaining module 33 is used to extract the image features of the current image data and obtain an image feature map; perform sparse processing on the image feature map to obtain a sparse feature map; The recognition module 34 is used to fuse the image feature map and the sparse feature map to obtain a fused feature map; and to identify the target area with smoke in the fused feature map through a preset smoke model; The control module 35 is used to control the fire extinguishing device to perform fire extinguishing operations on the target area.

[0062] Optionally, module 33 is obtained, and is also used to perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; and perform spatial sparse processing on the color sparse feature map through a spatial sparse network to obtain a sparse feature map.

[0063] Optionally, the fire extinguishing device 30 based on image data processing further includes a determination module, which is used to determine key feature information of the sparse feature map through a preset gated feedforward network; and mark the key feature information in the sparse feature map.

[0064] Optionally, the determination module is also used to determine a first matrix of features in the sparse feature map through an initial convolution layer and a gated convolution layer; input the first matrix to a preset activation function to obtain a second matrix; perform element-wise multiplication of the first matrix and the second matrix to obtain a third matrix; and determine key feature information in the third matrix by extracting the convolution layer.

[0065] The fire extinguishing device 30 based on image data processing may be a controller 400 .

[0066] <Equipment Example 2> Figure 6 It is a schematic diagram of the hardware structure of a fire extinguishing device based on image data processing according to another embodiment.

[0067] like Figure 6 As shown, the fire extinguishing device 40 based on image data processing includes a processor 41 and a memory 42, wherein the memory 42 is used to store an executable computer program, and the processor 41 is used to execute a method as described in any of the above method embodiments under the control of the computer program.

[0068] Each module of the fire extinguishing device 30 based on image data processing can be implemented by the processor 41 in this embodiment executing a computer program stored in the memory 42, or can be implemented by other structures, which is not limited here.

[0069] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0070] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the above. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0071] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0072] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present invention.

[0073] Various aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart and / or block diagram and the combination of the boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0074] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0075] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0076] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0077] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A fire extinguishing system based on image data processing, characterized in that: The system comprises: A smoke detection sensor, wherein the smoke detection sensor is used to detect the smoke concentration in the target scene; A camera, the camera being used to collect image data about the target scene; A fire extinguishing device, the fire extinguishing device is used to extinguish a fire in an area indicated in the target scene; A controller, wherein the controller is communicatively connected with the smoke detection sensor, the camera and the fire extinguishing device respectively, and the controller is configured to: receive smoke concentration information sent by the smoke detection sensor; when the smoke concentration information reflects that the current smoke concentration of the target scene is higher than a set threshold, obtain current image data of the target scene through the camera; extract image features of the current image data and obtain an image feature map; perform sparse processing on the image feature map to obtain a sparse feature map; fuse the image feature map and the sparse feature map to obtain a fused feature map; identify a target area with smoke in the fused feature map through a preset smoke model; and control the fire extinguishing device to perform a fire extinguishing operation on the target area.

2. The system according to claim 1, characterized in that The controller is further configured to: perform color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; The color sparse feature map is spatially sparsely processed through a spatial sparse network to obtain a sparse feature map.

3. The system according to claim 2, characterized in that After the image feature map is subjected to sparse processing to obtain a sparse feature map, the controller is further configured to: determine key feature information of the sparse feature map through a preset gated feedforward network; and mark the key feature information in the sparse feature map.

4. The system according to claim 2, characterized in that The gated feedforward network includes an initial convolution layer, a gated convolution layer and an extraction convolution layer; The controller is further configured to: determine a first matrix of features in the sparse feature map through the initial convolution layer and the gated convolution layer; input the first matrix into a preset activation function to obtain a second matrix; perform element-wise multiplication of the first matrix and the second matrix to obtain a third matrix; By extracting the convolutional layer, key feature information in the third matrix is ​​determined.

5. A fire extinguishing method based on image data processing, the fire extinguishing method based on image data processing is based on the fire extinguishing system based on image data processing according to any one of claims 1 to 4, the execution subject of the fire extinguishing method based on image data processing is a controller, characterized in that: The method comprises: Receiving smoke concentration information sent by the smoke detection sensor; When the smoke density information reflects that the current smoke density of the target scene is higher than a set threshold, obtaining current image data of the target scene through the camera; Extracting image features of current image data and obtaining an image feature map; performing sparse processing on the image feature map to obtain a sparse feature map; The image feature map and the sparse feature map are fused to obtain a fused feature map; a target area with smoke in the fused feature map is identified by a preset smoke model; The target area and the fused feature map are fed back to the user.

6. The method according to claim 5, characterized in that The step of performing sparse processing on the image feature map to obtain a sparse feature map includes: Performing color sparse processing on the image feature map through a color sparse network to obtain a color sparse feature map; The color sparse feature map is spatially sparsely processed through a spatial sparse network to obtain a sparse feature map.

7. The method according to claim 6, characterized in that After obtaining the sparse feature map, the method further includes: The key feature information of the sparse feature map is determined through a preset gated feedforward network; and the key feature information is marked in the sparse feature map.

8. The method according to claim 6, characterized in that The gated feedforward network includes an initial convolution layer, a gated convolution layer and an extraction convolution layer; the key feature information of the sparse feature map is determined by the preset gated feedforward network, including: Determining a first matrix of features in the sparse feature map through the initial convolution layer and the gated convolution layer; Inputting the first matrix into a preset activation function to obtain a second matrix; performing element-wise multiplication of the first matrix and the second matrix to obtain a third matrix; By extracting the convolutional layer, key feature information in the third matrix is ​​determined.

9. A fire extinguishing device based on image data processing, characterized in that: The device comprises: A receiving module, used for receiving smoke concentration information sent by the smoke detection sensor; an acquisition module, configured to acquire current image data of the target scene through the camera when the smoke density information reflects that the current smoke density of the target scene is higher than a set threshold; The obtaining module is used to extract the image features of the current image data and obtain an image feature map; the image feature map is subjected to sparse processing to obtain a sparse feature map; A recognition module is used to fuse the image feature map and the sparse feature map to obtain a fused feature map; and to identify a target area with smoke in the fused feature map through a preset smoke model; A control module is used to control the fire extinguishing device to perform fire extinguishing operations on the target area.

10. A fire extinguishing device based on image data processing, characterized in that: The method comprises a processor connected to a memory, wherein the memory is used to store a computer program; the processor is used to execute the computer program to implement the method according to any one of claims 4 to 8.

Images