Intelligent fire extinguisher management method based on RFID and machine vision integration

Abstract

The invention discloses an intelligent fire extinguisher management method based on RFID and machine vision integration, and the method comprises the steps: S1, scanning an RFID tag of a fire extinguisher through an RFID handheld reader-writer integrated with a camera, and reading the basic information of the fire extinguisher; and S2, clicking a related area of the reader-writer to be linked with a camera integrated on the RFID handheld reader-writer for photographing so as to obtain an area image of the pressure gauge of the fire extinguisher to be detected, and automatically calling a pre-trained pressure gauge state identification model to detect the state image of the pressure gauge of the fire extinguisher, automatically writing a fire extinguisher pressure state detection result and detection time into a corresponding column of the basic information page; s3, the basic information of the fire extinguisher is sent to a remote server, and automatic entry of management data of the fire extinguisher is completed; according to the invention, comprehensive and accurate monitoring and management of the fire extinguisher are realized, and the intelligent level of fire extinguisher management is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fire extinguisher management, and in particular to an intelligent fire extinguisher management method based on RFID and machine vision integration. Background Art

[0002] As one of the important firefighting equipment, fire extinguishers play a vital role in fire prevention and initial firefighting. In particular, as a key firefighting unit, power plants need to regularly inspect fire extinguishers to ensure that they can work properly in emergency situations, including appearance inspection, pressure gauge inspection, and missing parts, and on-site signature confirmation.

[0003] At present, there are as many as 3,000 fire extinguishers deployed on site in giant power plants, with many locations and wide distribution points. The management of fire extinguishers completely relies on the traditional manual inspection method, which has problems such as long inspection cycle, low efficiency, easy omission, and unsightly appearance. With the development of technology, RFID technology has been introduced into fire extinguisher management, which can track the location and status of fire extinguishers in real time. However, RFID technology alone cannot fully monitor the pressure and missing status of fire extinguishers. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide an intelligent fire extinguisher management method based on the integration of RFID and machine vision to solve the problems raised in the background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: an intelligent fire extinguisher management method based on RFID and machine vision integration, which includes the following steps: S1. Use an RFID handheld reader with an integrated camera to scan the RFID tag of the fire extinguisher to read the basic information of the fire extinguisher; the RFID tag is a passive RFID ultra-high frequency tag installed at a designated position of the fire extinguisher; the basic information of the fire extinguisher is displayed on the display screen of the reader, and the page has a special function of obtaining a pressure gauge image; S2. Click on the relevant area of ​​the reader to link the camera integrated in the RFID handheld reader to take a photo, so as to obtain the image of the pressure gauge area of ​​the fire extinguisher to be detected, and automatically call the pre-trained pressure gauge state recognition model to detect the fire extinguisher pressure gauge state image, and automatically write the fire extinguisher pressure state detection result and detection time into the corresponding column of the basic information page; S3. Through the data upload function, the basic information of the fire extinguisher is sent to the remote server based on wireless transmission to complete the automatic entry of fire extinguisher management data.

[0006] Furthermore, the pre-trained pressure gauge state recognition model described in step S2 is obtained by training a YOLOv7s network based on an attention mechanism using the collected historical pressure gauge state image set of fire extinguishers, wherein the model evaluation indicators set are accuracy, precision, and recall for classification and recognition tasks.

[0007] Furthermore, the fire extinguisher historical pressure gauge status image set is a large number of fire extinguisher historical pressure gauge status images collected in the power plant area, and a labeling tool is used to label the bounding box of each pressure area, pressure pointer and pressure dial in the image, and the pressure status is labeled with text.

[0008] Furthermore, the YOLOv7s network based on the attention mechanism includes four modules, namely, an image initialization feature extraction module, a backbone network feature extraction module, a neck structure feature fusion module, and a head structure target detection module.

[0009] Furthermore, the image initialization feature extraction module is used to pre-process the image and scale the image to a certain size; The backbone network feature extraction module includes CBS-2 module, CBS-3 module, ELAN module and MP-1 module, which are used to extract shallow graphic features and deep semantic features of images and obtain feature maps of different scales. The structure of the neck structure feature fusion module includes CBAM module, SPPCSPC module, ELAN-W module, MP-2 module, Upsample operation and Concat operation. It has an upsampling branch and a downsampling branch to achieve the fusion of shallow graphic features and deep semantic features, and obtain feature maps of different scales; The head structure consists of three branches, corresponding to three detection layers of different scales. Each detection layer is responsible for predicting the category and location information of the object; each branch consists of a REP module and a CBM module.

[0010] Furthermore, the specific process of automatically calling the pre-trained pressure gauge state recognition model in step S2 to detect the fire extinguisher pressure gauge state image is as follows: Image of the pressure gauge to be tested After the image initialization feature extraction module is used, the initialization feature map is obtained. ; Initialize feature map Through the backbone network feature extraction module, image feature extraction from shallow to deep is performed to obtain a multi-scale feature map , , ; Multi-scale feature map , , Through the neck structure feature fusion module, the shallow graphic features and deep semantic features are better integrated to obtain a multi-scale fusion feature map. , , ; Multi-scale fusion feature map , , The pressure state recognition result is obtained through the head structure target detection module.

[0011] Furthermore, the specific structure of the backbone network feature extraction module is: feature map Get the feature map through the first CBS-2 module of the backbone ; Feature map Get the feature map through the first CBS-3 module of the backbone ; Feature map Get the feature map through the backbone second CBS-2 module ; Feature map Get the feature map through the backbone second CBS-3 module ; Feature map Get the feature map through the first ELAN module of the backbone ; Feature map Get the feature map through the first MP-1 module of the backbone ; Feature map Get the feature map through the backbone second ELAN module ; Feature map Get the feature map through the backbone second MP-1 module ; Feature map Get the feature map through the backbone third ELAN module ; Feature map The feature map is obtained through the third MP-1 module of the backbone ; Feature map The feature map is obtained through the backbone fourth ELAN module ; In the feature extraction process, , , After being connected to a CBAM module in the unit, it is input into the neck structure, i.e., the feature map Through the first CBAM module in the connection unit, the feature map , feature map , the second CBAM module in the connection unit, feature map , feature map , the third CBAM module in the connection unit obtains the feature map .

[0012] Furthermore, the specific structure of the neck structure feature fusion module is as follows: feature map The feature map is obtained by the SPPCSPC module in the neck upsampling branch ; Feature map The feature map is obtained by the first CBAM module in the neck upsampling branch ; Feature map The feature map is obtained by the first Upsample operation in the neck upsampling branch. ; Feature map and feature map The fused feature map is obtained through the first Concat operation in the neck upsampling branch. ; Fusion feature map The feature map is obtained by the first ELAN-W module in the neck upsampling branch. ; Feature map The feature map is obtained by the second CBAM module in the neck upsampling branch ; Feature map The feature map is obtained by the second Upsample operation in the neck upsampling branch. ; Feature map and feature map The fused feature map is obtained through the second Concat operation in the neck upsampling branch. ; Fusion feature map The feature map is obtained by the second ELAN-W module in the neck upsampling branch ; Feature map The feature map is obtained by the first MP-2 module in the neck downsampling branch ; Feature map and feature map The fused feature map is obtained through the first Concat operation in the neck downsampling branch. ; Fusion feature map The feature map is obtained by the first ELAN-W module in the neck downsampling branch. ; Feature map The feature map is obtained by the second MP-2 module in the neck downsampling branch ; Feature map and feature map The fused feature map is obtained through the second Concat operation in the neck downsampling branch. ; Fusion feature map The feature map is obtained by the second ELAN-W module in the neck downsampling branch .

[0013] Furthermore, the specific structure of the head structure target detection module is as follows: feature map The large-scale feature map detection results are obtained through the REP module and CBM module of the first branch of the head ; Feature map The mid-scale feature map detection results are obtained through the REP module and CBM module of the second branch of the head ; Feature map The small-scale feature map detection results are obtained through the REP module and CBM module of the third branch of the head .

[0014] Beneficial effects of the present invention: 1. The present invention combines the location and status information obtained by RFID technology with the pressure and other information of the fire extinguisher obtained by machine vision technology, automatically generates maintenance reminders based on the analysis results, and guides management personnel to inspect and maintain fire extinguishers in a timely manner, thereby achieving comprehensive and accurate monitoring and management of fire extinguishers, and further improving the level of intelligent management of fire extinguishers.

[0015] 2. The present invention adopts RFID technology combined with image monitoring, that is, RFID equipment with integrated camera to realize intelligent management of fire extinguishers; it develops and constructs an improved YOLOv7s network based on attention mechanism to realize the reading and writing of basic fire extinguishing data and intelligent identification of the pressure status of fire extinguishers. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a YOLOv7s network framework diagram based on the attention mechanism of the present invention. DETAILED DESCRIPTION

[0017] The following is a detailed description of the present invention with reference to specific embodiments.

[0018] Embodiment 1: An intelligent fire extinguisher management method based on RFID and machine vision integration, comprising: S1. Use an RFID handheld reader with an integrated camera to scan the RFID tag of the fire extinguisher to read the basic information of the fire extinguisher; the RFID tag is a passive RFID ultra-high frequency tag installed at a designated position of the fire extinguisher; the basic information of the fire extinguisher is displayed on the display screen of the reader, and the page has a special function of obtaining a pressure gauge image; S2. Clicking the relevant area of ​​the reader can link the camera integrated in the RFID handheld reader to take a photo, so as to obtain the image of the pressure gauge area of ​​the fire extinguisher to be detected, and automatically call the pre-trained pressure gauge state recognition model to detect the fire extinguisher pressure gauge state image, and automatically write the fire extinguisher pressure state detection result and detection time into the corresponding column of the basic information page; The pre-trained pressure gauge state recognition model described in step S2 is obtained by training the YOLOv7s network based on the attention mechanism using the collected historical pressure gauge state image set of the fire extinguisher, wherein the model evaluation indicators set are accuracy, precision, and recall that are widely used in classification and recognition tasks.

[0019] The fire extinguisher historical pressure gauge status image set is a large number of fire extinguisher historical pressure gauge status images collected in the power plant area. There are 5,000 pressure gauge status images with the pointer in the red display area, 7,500 pressure gauge status images with the pointer in the green display area, and 3,000 pressure gauge status images with the pointer in the yellow display area. The image annotation tool LabelImg is used to annotate the boundary boxes of each pressure area, pressure pointer, and pressure gauge in the image, and the text annotates the pressure status (the pointer in the red display area is marked as refilling, the pointer in the green display area is marked as normal pressure, and the pointer in the yellow display area is marked as overfilling).

[0020] The YOLOv7s network based on the attention mechanism includes four modules, namely, an image initialization feature extraction module, a backbone network feature extraction module, a neck structure feature fusion module, and a head structure target detection module.

[0021] The image initialization feature extraction module is used to preprocess the image and scale the image to a certain size.

[0022] The main structures of the backbone network feature extraction module include CBS-2 module, CBS-3 module, ELAN module (a residual structure, the basic modules are CBAM and CBS-2), MP-1 module (double-branch downsampling structure, the basic modules are maximum pooling operation and CBAM and CBS-3), which are used to extract shallow graphic features and deep semantic features of images and obtain feature maps of different scales. Among them, the CBAM module is a mixed attention convolution module, which consists of a channel attention module, a spatial attention module and a Conv2d layer with a stride of 1 and a convolution kernel size of 1×1, focusing on what is and where it is in the channel and space respectively; the CBS-2 module consists of a Conv2d layer with a stride of 1 and a convolution kernel size of 3×3, a BatchNorm2d layer and a SiLu activation function; the CBS-3 module consists of a Conv2d layer with a stride of 2 and a convolution kernel size of 3×3, a BatchNorm2d layer and a SiLu activation function.

[0023] The neck structure feature fusion module consists of a CBAM module, a SPPCSPC module (a residual module with a spatial pyramid pooling operation, the basic modules are the maximum pooling operation, CBAM and CBS-2), an ELAN-W module (a residual structure, the basic modules are CBAM and CBS-2), an MP-2 module (a dual-branch downsampling structure, the basic modules are the maximum pooling operation and CBAM and CBS-3), an Upsample operation and a Concat operation. It has an upsampling branch and a downsampling branch, which can achieve better fusion of shallow graphic features and deep semantic features, and obtain feature maps of different scales.

[0024] The head structure consists of three branches, corresponding to three detection layers of different scales. Each detection layer is responsible for predicting the category and location information of the object. Through multi-scale detection, it can better adapt to objects of different sizes. Each branch consists of a REP module, a CBM module (consisting of a Conv2d layer with a stride of 1 and a convolution kernel size of 1×1, a BatchNorm2d layer, and a Sigmod activation function).

[0025] The process of automatically calling the pre-trained pressure gauge state recognition model to detect the fire extinguisher pressure gauge state image in step S2 includes: Image of the pressure gauge to be tested After the image initialization feature extraction module is used, the initialization feature map is obtained. ; Initialize feature map Through the backbone network feature extraction module, image feature extraction from shallow to deep is performed to obtain a multi-scale feature map , , ; Multi-scale feature map , , Through the neck structure feature fusion module, the shallow graphic features and deep semantic features are better integrated to obtain a multi-scale fusion feature map. , , ; Multi-scale fusion feature map , , The pressure state recognition result is obtained through the head structure target detection module.

[0026] The specific structure of the backbone network feature extraction module is: → The first CBS-2 module of backbone obtains the characteristic map ; Feature map → The first CBS-3 module of backbone obtains the characteristic map ; Feature map →Backbone the second CBS-2 module to get the feature map ; Feature map → backbone second CBS-3 module, get the feature map ; Feature map → The first ELAN module of backbone gets the characteristic map ; Feature map → The first MP-1 module of backbone obtains the feature map ; Feature map → The second ELAN module of backbone obtains the feature map ; Feature map → The second MP-1 module of backbone obtains the feature map ; Feature map → The third ELAN module of backbone obtains the characteristic map ; Feature map → The third MP-1 module of backbone obtains the feature map ; Feature map → The fourth ELAN module of backbone obtains the characteristic map In the feature extraction process, , , After being connected to a CBAM module in the unit, it is input into the neck structure, i.e., the feature map → First CBAM module in the connection unit → Feature map , feature map → Second CBAM module in the connection unit → Feature map , feature map → Third CBAM in the connection unit → Feature map .

[0027] The specific structure of the neck structure feature fusion module is as follows: feature map → SPPCSPC module in the neck upsampling branch obtains the feature map ; Feature map → The first CBAM module in the neck upsampling branch obtains the feature map ; Feature map → The first Upsample operation in the neck upsampling branch obtains the feature map ; Feature map and feature map → The first Concat operation in the neck upsampling branch obtains the fused feature map ; Fusion feature map → The first ELAN-W module in the neck upsampling branch obtains the feature map ; Feature map → The second CBAM module in the neck upsampling branch obtains the feature map ; Feature map → The second Upsample operation in the neck upsampling branch obtains the feature map ; Feature map and feature map → The second Concat operation in the neck upsampling branch obtains the fused feature map ; Fusion feature map → The second ELAN-W module in the neck upsampling branch obtains the feature map ; Feature map → The first MP-2 module in the neck downsampling branch obtains the feature map ; Feature map and feature map → The first Concat operation in the neck downsampling branch obtains the fused feature map ; Fusion feature map →The first ELAN-W module in the neck downsampling branch obtains the feature map ; Feature map → The second MP-2 module in the neck downsampling branch obtains the feature map ; Feature map and feature map → The second Concat operation in the neck downsampling branch obtains the fused feature map ; Fusion feature map → The second ELAN-W module in the neck downsampling branch obtains the feature map .

[0028] The specific structure of the head structure target detection module is: feature map → The REP module and CBM module of the first branch of the head obtain the large-scale feature map detection results ; Feature map →The REP module and CBM module of the second branch of the head obtain the mid-scale feature map detection results ; Feature map → The REP module and CBM module of the third branch of the head obtain the small-scale feature map detection results .

[0029] S3, through the data upload function, the basic information of the fire extinguisher is sent to the remote server based on wireless transmission, and the automatic entry of the fire extinguisher management data is completed; Based on a large number of historical fire extinguisher pressure gauge state image datasets collected and annotated in the power plant area, the three pressure gauge state samples in the dataset are divided into a ratio of 7:2:1, and then the training set, validation set and test set are obtained. The performance comparison of the pressure gauge state recognition model constructed by YOLOv7s based on the attention mechanism and the pressure gauge state recognition model constructed by conventional YOLOv7s is shown in the following table.

[0030]

[0031] It can be seen from the table that the performance of the pressure gauge state recognition model constructed by YOLOv7s based on the attention mechanism in the present invention is significantly better than the pressure gauge state recognition model constructed based on conventional YOLOv7s, which improves the reliability of intelligent recognition of the pressure gauge state.

[0032] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. An intelligent fire extinguisher management method based on RFID and machine vision integration, characterized in that: It includes the following steps: S1. Use an RFID handheld reader with an integrated camera to scan the RFID tag of the fire extinguisher to read the basic information of the fire extinguisher; the RFID tag is a passive RFID ultra-high frequency tag installed at a designated position of the fire extinguisher; the basic information of the fire extinguisher is displayed on the display screen of the reader, and the page has a special function of obtaining a pressure gauge image; S2. Click on the relevant area of ​​the reader to link the camera integrated in the RFID handheld reader to take a photo, so as to obtain the image of the pressure gauge area of ​​the fire extinguisher to be detected, and automatically call the pre-trained pressure gauge state recognition model to detect the fire extinguisher pressure gauge state image, and automatically write the fire extinguisher pressure state detection result and detection time into the corresponding column of the basic information page; S3. Through the data upload function, the basic information of the fire extinguisher is sent to the remote server based on wireless transmission to complete the automatic entry of fire extinguisher management data.

2. According to claim 1, a smart fire extinguisher management method based on RFID and machine vision integration is characterized in that: The pre-trained pressure gauge state recognition model described in step S2 is obtained by training the YOLOv7s network based on the attention mechanism using the collected historical pressure gauge state image set of the fire extinguisher, wherein the set model evaluation indicators are the accuracy, precision, and recall rate used for classification and recognition tasks.

3. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 2 is characterized in that: The fire extinguisher historical pressure gauge state image set is a large number of fire extinguisher historical pressure gauge state images collected in the power plant area, and a labeling tool is used to label the boundary box of each pressure area, pressure pointer and pressure dial in the image, and the pressure state is labeled with text.

4. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 2 is characterized in that: The YOLOv7s network based on the attention mechanism includes four modules, namely, an image initialization feature extraction module, a backbone network feature extraction module, a neck structure feature fusion module, and a head structure target detection module.

5. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 4 is characterized in that: The image initialization feature extraction module is used to preprocess the image and scale the image to a certain size; The backbone network feature extraction module includes CBS-2 module, CBS-3 module, ELAN module and MP-1 module, which are used to extract shallow graphic features and deep semantic features of images and obtain feature maps of different scales. The structure of the neck structure feature fusion module includes CBAM module, SPPCSPC module, ELAN-W module, MP-2 module, Upsample operation and Concat operation. It has an upsampling branch and a downsampling branch to achieve the fusion of shallow graphic features and deep semantic features, and obtain feature maps of different scales; The head structure consists of three branches, corresponding to three detection layers of different scales. Each detection layer is responsible for predicting the category and location information of the object; each branch consists of a REP module and a CBM module.

6. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 1 is characterized in that: The specific process of automatically calling the pre-trained pressure gauge state recognition model to detect the fire extinguisher pressure gauge state image in step S2 is as follows: Image of the pressure gauge to be tested After the image initialization feature extraction module is used, the initialization feature map is obtained. ; Initialize feature map Through the backbone network feature extraction module, image feature extraction from shallow to deep is performed to obtain a multi-scale feature map , , ; Multi-scale feature map , , Through the neck structure feature fusion module, the shallow graphic features and deep semantic features are better integrated to obtain a multi-scale fusion feature map. , , ; Multi-scale fusion feature map , , The pressure state recognition result is obtained through the head structure target detection module.

7. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 6 is characterized in that: The specific structure of the backbone network feature extraction module is: feature map Get the feature map through the first CBS-2 module of the backbone ; Feature map Get the feature map through the first CBS-3 module of the backbone ; Feature map Get the feature map through the backbone second CBS-2 module ; Feature map Get the feature map through the backbone second CBS-3 module ; Feature map Get the feature map through the first ELAN module of the backbone ; Feature map Get the feature map through the first MP-1 module of the backbone ; Feature map Get the feature map through the backbone second ELAN module ; Feature map Get the feature map through the backbone second MP-1 module ; Feature map Get the feature map through the backbone third ELAN module ; Feature map The feature map is obtained through the third MP-1 module of the backbone ; Feature map The feature map is obtained through the backbone fourth ELAN module ; In the feature extraction process, , , After being connected to a CBAM module in the unit, it is input into the neck structure, i.e., the feature map Through the first CBAM module in the connection unit, the feature map , feature map , the second CBAM module in the connection unit, feature map , feature map , the third CBAM module in the connection unit obtains the feature map .

8. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 6 is characterized in that: The specific structure of the neck structure feature fusion module is as follows: feature map The feature map is obtained by the SPPCSPC module in the neck upsampling branch ; Feature map The feature map is obtained by the first CBAM module in the neck upsampling branch ; Feature map The feature map is obtained by the first Upsample operation in the neck upsampling branch. ; Feature map and feature map The fused feature map is obtained through the first Concat operation in the neck upsampling branch. ; Fusion feature map The feature map is obtained by the first ELAN-W module in the neck upsampling branch. ; Feature map The feature map is obtained by the second CBAM module in the neck upsampling branch ; Feature map The feature map is obtained by the second Upsample operation in the neck upsampling branch. ; Feature map and feature map The fused feature map is obtained through the second Concat operation in the neck upsampling branch. ; Fusion feature map The feature map is obtained by the second ELAN-W module in the neck upsampling branch ; Feature map The feature map is obtained by the first MP-2 module in the neck downsampling branch ; Feature map and feature map The fused feature map is obtained through the first Concat operation in the neck downsampling branch. ; Fusion feature map The feature map is obtained by the first ELAN-W module in the neck downsampling branch. ; Feature map The feature map is obtained by the second MP-2 module in the neck downsampling branch ; Feature map and feature map The fused feature map is obtained through the second Concat operation in the neck downsampling branch. ; Fusion feature map The feature map is obtained by the second ELAN-W module in the neck downsampling branch .

9. The intelligent fire extinguisher management method based on RFID and machine vision integration according to claim 6 is characterized in that: The specific structure of the head structure target detection module is: feature map The large-scale feature map detection results are obtained through the REP module and CBM module of the first branch of the head ; Feature map The mid-scale feature map detection results are obtained through the REP module and CBM module of the second branch of the head ; Feature map The small-scale feature map detection results are obtained through the REP module and CBM module of the third branch of the head .

Images