Environmental monitoring method based on visual identification

Abstract

The invention discloses an environment monitoring method based on visual identification, and the method comprises the steps: S1, shooting an image of a monitoring region in real time through an image collection module, transmitting the image to a controller after the image is processed, and transmitting the image to a terminal platform through a data transmission module electrically connected with the controller; and S2, when the terminal platform detects that the flame and smoke information is marked in the received picture, alarm information is sent out. According to the invention, the image acquisition module is adopted to shoot the condition of the monitoring area in real time, the flame and smoke information of the monitoring area is accurately identified through the improved YOLOv8 algorithm, and the terminal platform sends out an alarm signal for the abnormal condition and takes corresponding measures in time, so that the harm caused by information hysteresis can be avoided, and the fire hazard can be reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual monitoring, and in particular relates to an environment monitoring method based on visual recognition. Background Art

[0002] The elevator machine room is generally located on the top floor. The elevator makes a lot of noise when running, and the traction wheel rotates at high speed. Therefore, in order to reduce noise and prevent non-staff from entering, the machine room is basically a closed environment. Secondly, the long-term operation of the high-power traction motor causes the temperature in the elevator machine room to rise, especially in summer, which can easily cause a fire due to high temperature causing the circuit to ignite.

[0003] The elevator machine room is an important place for emergency operation and rescue as it is the storage space for the drive host, control cabinet and speed limiter. The quality of its operating environment directly affects the operating life and quality of the equipment. Although the special equipment inspection regulations and national standards have set requirements for the elevator machine room environment, the monitoring of the machine room environment is mostly based on regular inspections and maintenance by elevator maintenance personnel, and real-time monitoring is not achieved. Therefore, there have always been safety hazards of high temperature and fire. Summary of the invention

[0004] The purpose of the present invention is to provide an environment monitoring method based on visual recognition, which can monitor the environment of the monitoring area in real time and solve the problems mentioned in the background technology.

[0005] In order to solve the above technical problems, the present invention adopts the following solutions: An environmental monitoring method based on visual recognition comprises the following steps: Step S1: The image acquisition module captures the image of the monitoring area in real time, and transmits it to the controller after processing, and then sends it to the terminal platform through the data transmission module electrically connected to the controller.

[0006] The image acquisition module is a K210 module equipped with an improved YOLOv8 algorithm.

[0007] The image acquisition module identifies the acquired image based on the improved YOLOv8 algorithm, and the specific steps include: Step S1.1: For multiple monitoring area images with different scale features taken by the image acquisition module, the improved YOLOv8 algorithm is used to extract image features to obtain feature maps. x ; Step S1.2: Feature map x The processing includes the following steps: Step S1.2.1: Feature map x After three maximum pooling layers are connected in series, the feature map is obtained xm 1. xm 2 and xm3. The specific formula is: ; (1) In formula (1), MaxPool () represents the maximum pooling function; Conv () represents the convolutional layer; Step S1.2.2: Introduction GAM Attention module, generating attention map xc 1. xc 2 and xc 3. The specific formula is: ; (2) In formula (2), GAM ()express GAM Attention function; Conv () represents the convolutional layer; Step S1.2.3: Concatenate the three attention maps along the channel dimension to form a fused feature map. The formula is: ; (3) In formula (3), xo Represents the fused feature map; Step S1.2.4: The fused feature map is expressed as follows using the downsampling module SPPFGAM module: ; (4) In formula (4), Concat () represents the connection function; Conv () represents the convolutional layer; Step S1.3: Design loss function; Step S1.4: Use the trained improved YOLOv8 algorithm to identify the output feature vector of the downsampling module SPPFGAM in step S1.2; wherein the improved YOLOv8 algorithm in the image acquisition module has been previously trained using a large number of existing flame and smoke images.

[0008] Step S2: When the terminal platform detects that the received image is marked with flame and smoke information, it issues an alarm message.

[0009] The present invention adopts an image acquisition module based on deep learning and machine vision to monitor the open flame and smoke in the elevator room in real time, so that the safety hazards in the elevator room can be discovered in time and the losses can be reduced.

[0010] Further optimization, the data transmission module is electrically connected to the terminal platform via the TCP / IP protocol, and the data transmission module communicates with the terminal platform via the 4G module.

[0011] Furthermore, in step S1.3, the loss function is calculated as follows: ; ; (5) In formula (5), X and y Indicates the coordinates of the center point of the anchor box; X gt and gt Indicates the center point coordinates of the target box; W g and H g Represents the length and height of the bounding box, W g and H g Separated from the computation graph, the superscript * indicates the separation operation; R WIoU For the penalty item, L CIOU represents the bounding box loss.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The present invention adopts an image acquisition module to shoot the situation in the monitoring area in real time, and accurately identifies the flame and smoke information in the monitoring area through the improved YOLOv8 algorithm. The terminal platform sends out an alarm signal for abnormal situations and takes corresponding measures in time, which can avoid the harm caused by information lag and reduce fire hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a module diagram of the environment monitoring system based on visual recognition described in Example 1; Figure 2 This is the improved YOLOv8 algorithm model diagram; Figure 3 This is the structure diagram of the improved SPPFGAM module; Figure 4 This is the flame graph identified by the improved YOLOv8 algorithm during outdoor testing; Figure 5 This is the smoke image recognized by the improved YOLOv8 algorithm during the outdoor test; Figure 6 This is the PR curve before the YOLOv8 algorithm was improved; Figure 7 This is the PR curve chart after the improvement of the YOLOv8 algorithm. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] Example 1: See Figure 1 , an environment monitoring system based on visual recognition, comprising: Microcontroller: In this embodiment, the core chip of the microcontroller is STM32F103RCT6 single chip microcomputer.

[0016] The image acquisition module is installed in the elevator room and is electrically connected to the microcontroller to capture the real-time image of the elevator room. In this embodiment, the image acquisition module is a K210 module, which is equipped with an improved YOLOv8 algorithm, has high face recognition efficiency and secondary development capabilities, and finally transmits the recognized face information to the single-chip microcomputer; and can recognize flames and smoke in the elevator room.

[0017] The data transmission module is electrically connected to the microcontroller and can send the monitoring data stored in the microcontroller to the terminal platform. In this embodiment, the data transmission module is an EC800M-CN core board with mobile communication function, and the EC800 series modules on it can be connected to the terminal platform through the TCP / IP protocol and send the data processed by the microprocessor to the terminal platform through 4G communication.

[0018] The power module supplies power to the data transmission module and the image acquisition module.

[0019] The terminal platform receives the monitoring data sent by the microcontroller and displays it. The terminal platform installs a TCP / IP server and connects to the data transmission module. After receiving the data, the terminal platform analyzes and displays it. When the image captured by the image acquisition module is marked with flame or smoke information in the elevator room, an alarm signal is issued.

[0020] Embodiment 2: A method for environmental monitoring based on visual recognition, comprising the following steps: Step S1: The image acquisition module captures the image of the monitoring area in real time, and transmits it to the controller after processing, and then sends it to the terminal platform through the data transmission module electrically connected to the controller.

[0021] The image acquisition module can identify and mark the flame and smoke information in the captured images based on the improved YOLOv8 algorithm.

[0022] Step S2: When the terminal platform detects that flames and smoke are marked in the picture taken by the acquisition module, an alarm message is issued.

[0023] As a classic mature algorithm, the YOLO algorithm has the advantages of fast detection speed, light model, and easy grafting and modification. The improvement and optimization based on the YOLO algorithm can make it strike a balance between detection speed and detection accuracy. The YOLOv8 algorithm is the most advanced YOLO algorithm at present. In addition, the flame and smoke recognition tasks in the elevator room have real-time and high accuracy. Therefore, YOLOv8 is selected as the basic algorithm model for classification and recognition. At the same time, in view of the large differences in the shape and color of flames and smoke, it is necessary to improve the YOLOv8 algorithm. For this reason, the present invention improves the traditional YOLOv8 algorithm.

[0024] The improvement strategy of the YOLOv8 algorithm is as follows: ① Replace the network structure with the Slimneck network structure in the Neck network to achieve lightweight overall network structure; ② In the backbone network Backbone, introduce the global attention mechanism GAM module to improve the downsampling module SPPF and name it the SPPFGAM module to enhance the significance of target features in the elevator room scene, thereby improving the detection accuracy; ③ Replace the loss function with the WIoUv1 function, accelerate the function convergence and improve the positioning accuracy of the anchor frame, and solve the problems of false detection, missed detection and repeated detection to a certain extent. The improved YOLOv8 algorithm model is shown in Figure 2 .

[0025] The process of image acquisition module using improved YOLOv8 algorithm to recognize images is as follows: First, for the image of the elevator room environment taken by the image acquisition module, the improved YOLOv8 algorithm is used for feature extraction, feature recognition and image detection. According to the structural characteristics, the YOLOv8 algorithm model is divided into three parts: the backbone network Backbone, the neck network Neck and the head network Head: the backbone network mainly includes the convolution module Conv, the C2f module and the downsampling / pooling module SPPF, which are mainly used to extract features from the input image. The C2f module includes two convolution modules and n bottleneck modules Bottleneck, which are used to extract richer gradient information. The downsampling module SPPF includes 3 maximum pooling operations to capture information and improve detection accuracy. The neck network part is used to fuse feature maps of different levels and scales to ensure feature extraction capabilities. The head network mainly completes detection and classification tasks at different scales and finally outputs detection information.

[0026] Specifically, the improved algorithm compared to the traditional YOLOv8 algorithm is further described as: based on GSConv replacing standard convolution, constructing a Slimneck network. First, it is processed through the GSConv network structure, which is lightweight while ensuring sufficient detection accuracy. The GSConv network structure transforms part of the images with different scale features through standard convolution, and selects the other part through DW convolution, and then splices the two parts of feature extraction. Finally, the channel shuffle operation is used, which ensures sufficient detection accuracy while being lightweight. The GSConv network will cause the loss of key information during the layer-by-layer downsampling process, so two GSConv convolutions are combined with the input shortcut operation to form GSBottleneck. Finally, a one-time aggregation is performed to form the VoVGSCSP module. The GSConv network and the VoVGSCSP module form a Slimneck network.

[0027] Then, the downsampling module is improved to construct the SPPFGAM module. x In the flame smoke detection task in the elevator room environment, due to the large changes in the shape of the flame smoke, in order to improve the detection field of the algorithm model and further improve the detection accuracy, the downsampling module SPPF is improved, and the feature extraction capability is enhanced by integrating the GAM global attention mechanism; three maximum pooling layers are operated on the information features of the C2f input, and a 3×3 convolution operation is performed in each maximum pooling layer, and then the GAM attention module is introduced, and finally splicing and fusion are performed. The improved module is named SPPFGAM. The structure diagram of the improved SPPFGAM module is shown below: Figure 3 shown.

[0028] The improved SPPFGAM module processes the input feature map x as follows: 1) The feature map x After three maximum pooling layers are connected in series, the feature map is obtained xm 1. xm 2 and xm 3. The specific formula is: ; (1) In formula (1), MaxPool () represents the maximum pooling function; Conv () represents a convolutional layer.

[0029] 2) Introduction GAM Attention module, generating attention map xc 1. xc 2 and xc 3. The specific formula is: ; (2) In formula (2), GAM ()express GAM Attention function; Conv () represents a convolutional layer.

[0030] GAM (F1) is expressed as: ;

[0031] In the above formula, M C Represents the channel attention mechanism; M S Represents the spatial attention mechanism; Indicates cascade.

[0032] 3) Concatenate the three attention maps along the channel dimension to form a fused feature map. The formula is: ; (3) In formula (3), xo Represents the fused feature map.

[0033] 4) The feature map output by the downsampling module SPPFGAM module is expressed as: ; (4) In formula (4), Concat () represents the connection function; Conv () represents a convolutional layer.

[0034] In the target recognition scenario in the elevator room, although the traditional attention mechanism has contributed to the preservation of channel and spatial information, it is still insufficient in enhancing cross-dimensional interaction. To this end, the present invention introduces a global attention mechanism module (Global Attention Mechanism, GAM) in the SPPF module, which reduces the diffusion of information in the network and amplifies the expression of global interaction features, highlights the characteristics of flames and smoke, retains key information while filtering out background noise, so as to effectively focus on the target objects in these small spaces, thereby improving the detection accuracy and the overall performance of the network.

[0035] Then, the loss function is designed. The YOLOv8 regression loss function is CIoU. CIoU introduces the aspect ratio between the predicted box and the real box boundary and the center distance between the two on the basis of the IoU loss function, so that the loss function pays more attention to the shape of the bounding box. However, since the aspect ratio describes a relative value rather than a true difference value, when the aspect ratio of the predicted box is equal to that of the real box, the penalty term of the CIoU aspect ratio will be invalid; at the same time, CIoU does not consider the balance between high and low quality samples. When the target detection training set contains low-quality examples, the model detection performance will be reduced. There are three versions of Wise-IoU. The Wise-IoUv1 (abbreviated as WIoUv1) loss function is used to replace the CIoU used by the original YOLOv8 model to balance the model training results of images of different qualities and obtain more accurate detection results. The WIoUv1 loss function is an existing technology and will not be described in detail.

[0036] The calculation formula of WIoUv1 is: ; (5) ; (6) In formula (5) and (6), X and y Indicates the coordinates of the center point of the anchor box; X gt and y gt Indicates the coordinates of the center point of the target box; W g and H g Represents the length and height of the bounding box, W g and H g Separated from the computation graph, the superscript * indicates the separation operation; R WIoU is the penalty item, L CIOU represents the bounding box loss.

[0037] Finally, the improved YOLOv8 algorithm in the image acquisition module has been trained using a large number of flame and smoke images. The output feature vector x0 of the downsampling module SPPFGAM in the second step is recognized using the trained improved YOLOv8 algorithm.

[0038] If the improved YOLOv8 algorithm can identify flames and smoke, and the corresponding images are marked by the host computer platform and sent to the terminal platform, the terminal platform will send an alarm signal based on the mark.

[0039] Test and verification: Due to the special environment of the elevator room, it is not possible to conduct real flame and smoke tests in the elevator room, so flame and smoke tests are conducted outdoors, and the image acquisition module based on the improved YOLOv8 algorithm is the K210 module, which collects and recognizes flames. The flame images recognized in the test are shown in Figure 4 , the identified smoke image is shown in Figure 5 ,in, Figure 4 (a), (b), (c), and (d) represent the flame images identified in different test processes. The numbers in the figure represent the probability of being identified as flames, for example Figure 4 In (a), the probability of identifying the flame is 0.95. Similarly, Figure 5 (a), (b), and (c) in the figure represent the smoke images identified in different test processes. The numbers in the figure represent the probability of identifying it as smoke, for example Figure 5 In (a), the probability of detecting smoke is 0.94. Figure 4 and Figure 5 It can be seen that the clarity and accuracy of both flames and smoke have been greatly improved, which shows that the improved YOLOv8 algorithm is very reliable for identifying flames and smoke in elevator rooms.

[0040] In order to verify the separate and combined application effects of the three modules of Slimneck, SPPFGAM and WIoUv1 in flame and smoke recognition, ablation tests were carried out, as shown in Table 1. "√" indicates that the method was used in the experiment, and "-" indicates that the method was not used in the experiment. Through ablation test comparison, compared with the traditional YOLOv8 algorithm, the present invention adds three modules respectively, and has a certain improvement in mAP0.5, mAP0.5:0.95 and FPS indicators. Among them, mAP0.5 represents the average accuracy when the threshold is 0.5, and mAP0.5:0.95 represents the average accuracy when the threshold is between 0.5-0.95; FPS represents the number of frames transmitted per second; Params represents the total number of parameters contained in the model, and Gflops represents the amount of floating-point operations.

[0041] Table 1 Ablation experiment results ; As can be seen from Table 1, the accuracy of mAP0.5 and mAP0.5:0.95 has been improved. The PR curves before and after the improvement are as follows: Figure 6 and Figure 7As shown. P is the precision (Precision, P), R is the recall (Recall, R), generally the recall is set as the horizontal axis, the precision is set as the vertical axis, the area under the PR curve is the average precision (Average Precision, AP), the mean average precision of all categories (mean Average Precision, mAP) is mAP, this curve is the most important performance indicator curve. Figure 6 , Figure 7 In the figure, the three lines represent the average accuracy of flame, smoke and the mean of their average accuracy. Specifically, Figure 6 The average accuracy of flame is 0.883, that is, the average accuracy of flame recognition before improvement is 0.883; the average accuracy of smoke is 0.725, that is, the average accuracy of smoke recognition before improvement is 0.725; the middle curve indicates that the average accuracy of flame and smoke recognition before improvement is 0.804. Figure 7 The average accuracy of flame is 0.902, that is, the average accuracy of flame recognition after improvement is 0.902; the average accuracy of smoke is 0.769, that is, the average accuracy of smoke recognition after improvement is 0.769; the middle curve indicates that the average accuracy of flame and smoke recognition is 0.835, that is, the average accuracy of flame and smoke recognition after improvement is 0.835. Comparing the two figures, we can see that the three indicators of the improved method have been improved.

[0042] In order to verify the performance of the improved algorithm, it is compared with mainstream algorithm models, including SSD, FR-CNN and YOLO series. The comparative experimental results are shown in Table 2.

[0043] Table 2 Comparative experimental results

[0044] The experimental results show that although the floating-point computing capacity Gflops of the improved algorithm model is slightly larger than that of the YOLOv8 algorithm, it performs best in many other key indicators, fully demonstrating the advantages of the improved algorithm in complex detection tasks.

[0045] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. An environmental monitoring method based on visual recognition, characterized in that: The following steps are involved: Step S1: The image acquisition module takes images of the monitoring area in real time, and transmits them to the controller after processing, and then sends them to the terminal platform through the data transmission module electrically connected to the controller; The image acquisition module is a K210 module equipped with an improved YOLOv8 algorithm; The image acquisition module identifies the acquired image based on the improved YOLOv8 algorithm, and the specific steps include: Step S1.1: For multiple monitoring area images with different scale features taken by the image acquisition module, the improved YOLOv8 algorithm is used to extract image features to obtain feature maps. x ; Step S1.2: Feature map x The processing includes the following steps: Step S1.2.1: Feature map x After three maximum pooling layers are connected in series, the feature map is obtained xm 1. xm 2 and xm 3. The specific formula is: ； (1) In formula (1), MaxPool () represents the maximum pooling function; Conv () represents the convolutional layer; Step S1.2.2: Introduction GAM Attention module, generating attention map xc 1. xc 2 and xc 3. The specific formula is: ； (2) In formula (2), GAM ()express GAM Attention function; Conv () represents the convolutional layer; Step S1.2.3: Concatenate the three attention maps along the channel dimension to form a fused feature map. The formula is: ； (3) In formula (3), xo Represents the fused feature map; Step S1.2.4: The fused feature map is expressed as follows using the downsampling module SPPFGAM module: ； (4) In formula (4), Concat () represents the connection function; Conv () represents the convolutional layer; Step S1.3: Design loss function; Step S1.4: using the trained improved YOLOv8 algorithm to identify the output feature vector of the downsampling module SPPFGAM in step S1.2; wherein the improved YOLOv8 algorithm in the image acquisition module has been previously trained using a large number of existing flame and smoke images; Step S2: When the terminal platform detects that the received image is marked with flame and smoke information, it issues an alarm message.

2. The method for environmental monitoring based on visual recognition according to claim 1, characterized in that: The data transmission module is electrically connected to the terminal platform via the TCP / IP protocol, and the data transmission module communicates with the terminal platform via the 4G module.

3. The method for environmental monitoring based on visual recognition according to claim 2, characterized in that: In step S1.3, the calculation formula of the loss function is: ； ； (5) In formula (5), X and y Indicates the coordinates of the center point of the anchor box; X gt and gt Indicates the coordinates of the center point of the target box; W g and H g Represents the length and height of the bounding box, W g and H g Separated from the computation graph, the superscript * indicates the separation operation; R WIoU is the penalty item, L CIOU represents the bounding box loss.

Images