Flame detection method and device based on YOLOv5s

Abstract

The invention discloses a flame detection method based on YOLOv5s, and belongs to the technical field of target detection. The method comprises the following steps: acquiring target data of a fuel cell microgrid acquired by first acquisition equipment, wherein the target data comprises at least one of image data or video data; inputting the target data into a trained flame detection model to obtain a first detection result, the flame detection model comprising a reference network sub-model, a feature fusion sub-model and a detection sub-model; hydrogen storage tank pressure data of the fuel cell microgrid collected by second collection equipment is obtained, and a second detection result is obtained; based on the first detection result and the second detection result, obtaining a flame detection result of the fuel cell microgrid; whether the flame detection result is larger than or equal to a preset threshold value or not is judged, and when the flame detection result is larger than or equal to the preset threshold value, early warning information is sent. According to the method, the accuracy and real-time performance of fuel cell microgrid flame detection are improved, and early warning of fuel cell microgrid fire accidents is realized.

Description

Technical Field

[0001] This application belongs to the technical field of object detection, and particularly relates to a flame detection method and device based on YOLOv5s. Background Art

[0002] As an important distributed energy supply device in the fields of new energy, power systems, etc., the operation safety and reliability of fuel cell microgrids are directly related to the stability and economy of energy equipment. With the rapid development of hydrogen energy technology, the fire risk of fuel cell microgrids under high-temperature and high-pressure conditions has increased significantly. In particular, the flames caused by hydrogen leakage have characteristics such as fast diffusion and strong concealment, and are extremely likely to pose a serious threat to the safety of equipment and personnel. Therefore, the early detection and accurate identification of flames have become the key links to ensure the safe operation of fuel cell microgrids.

[0003] Existing flame detection methods mainly rely on the mode of sensor monitoring. Although they can trigger alarms in a timely manner, they lack visual analysis of the morphological characteristics of flames and cannot accurately judge the location of the fire source, the scale of combustion, and the spread trend. In addition, for the unique hydrogen flame characteristics of fuel cell microgrids (such as light blue flames and low smoke concentration), existing sensors are easily affected by the environment, resulting in a high false alarm rate. In recent years, although the vision detection method based on YOLOv5s can achieve flame recognition, its general algorithm has not been optimized for the complex environment of microgrids (such as metal reflection, equipment occlusion, and multi-light source interference), and there are problems such as insufficient feature extraction, high missed detection rate of small targets, poor real-time performance, and low accuracy. Summary of the Invention

[0004] This application aims to at least solve one of the technical problems existing in the prior art. For this reason, this application proposes a flame detection method and device based on YOLOv5s, which improves the accuracy and real-time performance of flame detection in fuel cell microgrids and realizes early warning of fuel cell microgrid fire accidents.

[0005] In the first aspect, this application provides a flame detection method based on YOLOv5s, and the method includes:

[0006] Obtain the target data of the fuel cell microgrid collected by the first acquisition device, where the target data includes at least one of image data or video data;

[0007] Input the target data into the trained flame detection model to obtain a first detection result, where the flame detection model includes a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model;

[0008] Obtain the hydrogen storage tank pressure data of the fuel cell microgrid collected by the second acquisition device to obtain a second detection result;

[0009] Based on the first detection result and the second detection result, obtain the flame detection result of the fuel cell microgrid;

[0010] Determine whether the flame detection result is greater than or equal to a preset threshold, and send a warning message when the flame detection result is greater than or equal to the preset threshold.

[0011] According to an embodiment of the present application, the obtaining the second detection result by acquiring the pressure data of the hydrogen storage tank of the fuel cell microgrid collected by the second acquisition device includes:

[0012] Perform normalization processing on the pressure data of the hydrogen storage tank to obtain normalized pressure data;

[0013] Process the normalized pressure data based on the Sigmoid function, and obtain the second detection result through the following formula:

[0014]

[0015] where, Wp is the second detection result, P N is the normalized pressure data, and k is a parameter for controlling the weight change speed.

[0016] According to an embodiment of the present application, the obtaining the flame detection result of the fuel cell microgrid based on the first detection result and the second detection result includes:

[0017] Fuse the first detection result and the second detection result based on the fusion analysis formula to obtain the flame detection result of the fuel cell microgrid, where the fusion analysis formula is as follows:

[0018] F = V×(1 + αWp)

[0019] where, F is the flame detection result of the fuel cell microgrid, Wp is the second detection result, V is the first detection result, and α is an adjustment coefficient for the pressure influence.

[0020] According to an embodiment of the present application, the benchmark network sub-model includes a backbone network and an FLA attention mechanism unit. The backbone network is a RepConv convolutional network. The FLA attention mechanism unit is embedded before the last convolutional layer of the backbone network. The backbone network is used to extract multi-scale features of the target data, and the FLA attention mechanism unit is used to improve the target detection performance.

[0021] According to an embodiment of the present application, the feature fusion sub-model includes a neck network and a small target detection layer. The neck network is used to fuse the multi-scale features extracted by the backbone network, and the small target detection layer is used to detect small-sized flames.

[0022] According to an embodiment of the present application, the detection sub-model is a head network, which is used to perform object classification and bounding box regression on the multi-scale features fused by the neck network, and finally output the flame confidence of the target data.

[0023] According to an embodiment of the present application, the training process of the flame detection model includes:

[0024] Construct a preset flame detection model;

[0025] Based on the fuel cell microgrid fire video picture set and the hydrogen flame picture set, obtain a data set;

[0026] Based on the WloU-v3 bounding box loss function, train the preset flame detection model according to the data set to obtain the flame detection model.

[0027] In a second aspect, the present application provides a flame detection device based on YOLOv5s, and the device includes:

[0028] A first acquisition module, which is used to acquire the target data of the fuel cell microgrid collected by the first acquisition device, and the target data includes at least one of image data or video data;

[0029] A processing module, which is used to input the target data into the trained flame detection model to obtain a first detection result, and the flame detection model includes a reference network sub-model, a feature fusion sub-model and a detection sub-model;

[0030] A second acquisition module, which is used to acquire the hydrogen storage tank pressure data of the fuel cell microgrid collected by the second acquisition device to obtain a second detection result;

[0031] A detection module, which is used to obtain the flame detection result of the fuel cell microgrid based on the first detection result and the second detection result;

[0032] A judgment module, which is used to judge whether the flame detection result is greater than or equal to a preset threshold, and in the case where the flame detection result is greater than or equal to the preset threshold, send a warning message.

[0033] In a third aspect, the present application provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the YOLOv5s-based flame detection method described in the first aspect above.

[0034] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the YOLOv5s-based flame detection method described in the first aspect above.

[0035] In a fifth aspect, the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the flame detection method based on YOLOv5s as described in the first aspect.

[0036] In a sixth aspect, the present application provides a computer program product, including a computer program which, when executed by a processor, implements the flame detection method based on YOLOv5s as described in the first aspect above.

[0037] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application.

[0038] The flame detection method based on YOLOv5s provided by the present invention has the following beneficial effects compared with the prior art:

[0039] (1) By acquiring image data or video data from a first acquisition device and inputting it into a trained flame detection model, and combining a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model to obtain a first detection result, the present invention effectively improves the accuracy of flame detection. By simultaneously acquiring the hydrogen storage tank pressure data of a second acquisition device as a second detection result and fusing it with the first detection result to obtain a flame detection result, it is possible to judge and prevent the occurrence of a fire faster and more accurately, realize early warning of fuel cell microgrid fire accidents, improve the real-time performance and accuracy of early warning of fuel cell microgrid fire accidents, and reduce potential safety hazards.

[0040] (2) By fusing the first detection result and the second detection result based on a fusion analysis formula to obtain a flame detection result of the fuel cell microgrid, and introducing a regulation coefficient for the influence of pressure to perform weighted fusion on different detection results, comprehensively considering the influence of different detection results, it is possible to judge and prevent the occurrence of a fire faster and more accurately, realize early warning of fuel cell microgrid fire accidents, and improve the real-time performance and accuracy of early warning of fuel cell microgrid fire accidents.

[0041] (3) By embedding the FLA attention mechanism unit before the last convolutional layer of the RepConv backbone network, the present invention can adapt to the scenarios of different illumination conditions and background noises in the fuel cell microgrid, enable the flame detection model to more accurately locate and identify the target area, obtain information of a larger area while reducing the computational amount, improve the feature representation, detection accuracy, computational efficiency, and generalization ability of the flame detection model, and can judge and prevent the occurrence of a fire faster and more accurately, realizing early warning of fuel cell microgrid fire accidents. Description of the Drawings

[0042] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the description of the embodiments in conjunction with the following drawings, where:

[0043] Figure 1 is a schematic flow chart of a flame detection method based on YOLOv5s provided by an embodiment of the present application;

[0044] Figure 2 is a schematic structural diagram of a RepConv convolutional network provided by an embodiment of the present application;

[0045] Figure 3 is a schematic flow chart of a Softmax attention mechanism provided by an embodiment of the present application;

[0046] Figure 4 is a schematic flow chart of a FLA attention mechanism provided by an embodiment of the present application;

[0047] Figure 5 is a schematic structural diagram of a neck network provided by an embodiment of the present application;

[0048] Figure 6 is a schematic structural diagram of a flame detection device based on YOLOv5s provided by an embodiment of the present application;

[0049] Figure 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0050] The technical solutions in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application fall within the scope of protection of the present application.

[0051] Terms such as "first" and "second" in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such used data may be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order different from those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and the number of objects is not limited. For example, the first object may be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means an "or" relationship between the associated objects before and after.

[0052] The following will, in conjunction with the accompanying drawings, elaborate in detail on the flame detection method based on YOLOv5s, the flame detection device based on YOLOv5s, the electronic device, and the readable storage medium provided by the embodiments of the present application through specific embodiments and their application scenarios.

[0053] Among them, the flame detection method based on YOLOv5s can be applied to a terminal, and specifically can be executed by the hardware or software in the terminal.

[0054] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touchpad).

[0055] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0056] The flame detection method based on YOLOv5s provided by the embodiments of the present application. The execution subject of the flame detection method based on YOLOv5s can be an electronic device or a functional module or functional entity in the electronic device that can implement the flame detection method based on YOLOv5s. The electronic devices mentioned in the embodiments of the present application include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices, etc. The following takes the electronic device as the execution subject to illustrate the flame detection method based on YOLOv5s provided by the embodiments of the present application.

[0057] Currently, while the fuel cell microgrid is undergoing industrial upgrading, due to the increasingly complex and specific tasks of automated equipment, the risk of various industrial disasters has also been greatly increased. Among various industrial disasters, the most likely dangerous event is a fire. Once a fire occurs in the fuel cell microgrid, it is very likely to cause personal and property damage to relevant technical personnel as well as property damage to the fuel cell microgrid. If the flame can be timely identified and detected in the early stage of the fuel cell microgrid fire, the fire can be discovered as early as possible and measures can be taken, which can greatly reduce the casualties and property losses caused by the fire.

[0058] In traditional fuel cell microgrid fire detection, it is usually completed by manual patrols and various sensors, such as temperature sensors, smoke sensors and other alarms. Among them, the former method has low intelligent automation, few patrol personnel and irregular patrol time, making it difficult for relevant personnel to be informed in the first time when a fire occurs and deal with the fire in a timely manner, which makes it possible for the fire to spread. The latter method does not have video images of the fire scene and cannot give detailed fire information, such as the degree of fire combustion, the size of the flame and the specific location of the fire. In addition, there is little targeted research on fuel cell microgrid fire detection. In recent years, thanks to the development of deep learning and computer vision technology, it has become possible to build a reliable, efficient and stable fuel cell microgrid fire prevention and control system.

[0059] The existing YOLOv5s algorithm only focuses on fire detection in flame detection, without considering its specific application background, etc. The fuel cell microgrid environment is complex and there are many interference sources, which often have high requirements for the real-time performance of model detection, and also have certain requirements for detection accuracy. The existing methods have problems such as insufficient feature extraction, high missed detection rate of small targets, poor real-time performance and low accuracy.

[0060] Figure 1 It is a schematic flow chart of a flame detection method based on YOLOv5s provided by an embodiment of the present application. As Figure 1 shown, the flame detection method based on YOLOv5s includes: step 110, step 120, step 130, step 140 and step 150.

[0061] Step 110: Obtain target data of the fuel cell microgrid collected by the first acquisition device, where the target data includes at least one of image data or video data;

[0062] It is easy to understand that the fuel cell microgrid includes a fuel cell stack, an air supply system, a power conditioning system, an energy storage system, a power grid and a communication system, and the energy storage system includes a hydrogen storage tank.

[0063] In some embodiments, the first acquisition device is a camera, and the camera is installed inside the fuel cell microgrid for shooting image data or video data including the hydrogen storage tank.

[0064] Step 120: Input the target data into the trained flame detection model to obtain a first detection result, where the flame detection model includes a reference network sub-model, a feature fusion sub-model and a detection sub-model;

[0065] Exemplarily, the electronic device inputs the target data into the trained flame detection model to obtain a first detection result, and the first detection result includes the hydrogen flame of the hydrogen storage tank, the specific location of the flame and the confidence level.

[0066] Optionally, the flame detection model is based on the YOLOv5s model. The flame detection model includes a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model. The benchmark network sub-model is used to extract multi-scale features of target data, providing rich feature information for subsequent target detection. The feature fusion sub-model is used to fuse the multi-scale features. The detection sub-model shows the hydrogen flame, the specific position of the flame, and the confidence level through anchor boxes, obtaining the first detection result.

[0067] In some embodiments, the electronic device includes a UI interface, which is connected to the flame detection model. The electronic device receives the target data collected by the first acquisition device through the UI interface, and can display information such as the target position, the total number of targets, the confidence level, and the time used in real time. The electronic device inputs the target data into the flame detection model through the UI interface for real-time detection of hydrogen flames, and displays and saves the first detection result output by the flame detection model.

[0068] Step 130: Obtain the hydrogen storage tank pressure data of the fuel cell microgrid collected by the second acquisition device, obtaining the second detection result;

[0069] Optionally, the second acquisition device is a pressure sensor, which is installed inside the hydrogen storage tank and is used to collect the hydrogen storage tank pressure data in real time and send it to the electronic device. The electronic device preprocesses the hydrogen storage tank pressure data to obtain the second detection result.

[0070] Step 140: Based on the first detection result and the second detection result, obtain the flame detection result of the fuel cell microgrid;

[0071] In some embodiments, the electronic device includes a data processing unit, which is used to fuse and analyze the first detection result and the second detection result to obtain the flame detection result of the fuel cell microgrid, and can detect possible dangers.

[0072] Step 150: Determine whether the flame detection result is greater than or equal to a preset threshold. When the flame detection result is greater than or equal to the preset threshold, send a warning message.

[0073] It should be noted that the electronic device also includes an automatic warning unit. When the electronic device determines that the flame detection result is greater than or equal to the preset threshold, it can send a warning message in the form of sound, image, or pop-up window, etc.

[0074] In some embodiments, the electronic device further includes an adaptive learning unit for automatically adjusting the parameters of the flame detection model according to real-time data to adapt to changing environmental conditions and flame characteristics. The adaptive learning unit includes a data collection unit, a parameter adjustment unit, and a performance evaluation unit. The data collection unit is used to collect environmental condition and flame characteristic data. The parameter adjustment unit is used to automatically adjust the model parameters based on the collected data. The performance evaluation unit is used to evaluate the performance of the adjusted flame detection model and feedback it to the parameter adjustment unit for further optimization.

[0075] Exemplarily, the electronic device is deployed in relevant areas of the fuel cell microgrid, enabling it to effectively detect flames and hydrogen flames. By real-time detecting the flame and hydrogen flame conditions, when flames and hydrogen flames are detected, early warning information is sent in a timely manner, and the specific location of the flame is marked, which can assist the staff in preventing fire accidents and quickly handling them in the early stage.

[0076] According to the flame detection method based on YOLOv5s provided by the embodiments of the present application, by obtaining image data or video data from the first acquisition device and inputting it into the trained flame detection model, and combining the benchmark network sub-model, the feature fusion sub-model, and the detection sub-model to obtain the first detection result, the accuracy of flame detection is effectively improved. By simultaneously obtaining the hydrogen storage tank pressure data of the second acquisition device as the second detection result and fusing it with the first detection result to obtain the flame detection result, it is possible to judge and prevent the occurrence of fires faster and more accurately, realizing early warning of fuel cell microgrid fire accidents, improving the timeliness and accuracy of early warning of fuel cell microgrid fire accidents, and reducing potential safety hazards.

[0077] In some embodiments, obtaining the hydrogen storage tank pressure data of the fuel cell microgrid collected by the second acquisition device to obtain the second detection result includes:

[0078] Performing normalization processing on the hydrogen storage tank pressure data to obtain normalized pressure data;

[0079] Processing the normalized pressure data based on the Sigmoid function, and obtaining the second detection result through the following formula:

[0080]

[0081] where, Wp is the second detection result, p N is the normalized pressure data, and k is the parameter controlling the weight change speed.

[0082] It should be noted that the calculation formula of the hydrogen storage tank pressure data is as follows:

[0083]

[0084] where P is the pressure of the hydrogen storage tank, P normal is the normal pressure value, P threshold is the pressure threshold, and p N is the pressure data of the hydrogen storage tank.

[0085] In this embodiment, by introducing a parameter for controlling the change speed of the weight, the processing process of the pressure data of the hydrogen storage tank is optimized, making the pressure monitoring of the hydrogen storage tank more accurate, capable of reflecting the change trend of the pressure of the hydrogen storage tank in real time, effectively improving the accuracy and reliability of the pressure detection of the hydrogen storage tank in the fuel cell microgrid, realizing the early warning of the fire accident in the fuel cell microgrid, enhancing the real-time performance and accuracy of the early warning of the fire accident in the fuel cell microgrid, and reducing the risk brought by pressure fluctuations.

[0086] In some embodiments, obtaining the flame detection result of the fuel cell microgrid based on the first detection result and the second detection result includes:

[0087] Fusing the first detection result and the second detection result based on a fusion analysis formula to obtain the flame detection result of the fuel cell microgrid, where the fusion analysis formula is as follows:

[0088] F = V × (1 + αWp)

[0089] where F is the flame detection result of the fuel cell microgrid, Wp is the second detection result, V is the first detection result, and α is the adjustment coefficient of the pressure influence.

[0090] It is easy to understand that another form of the fusion analysis formula is as follows:

[0091]

[0092] In this embodiment, fusing the first detection result and the second detection result based on the fusion analysis formula to obtain the flame detection result of the fuel cell microgrid. By introducing the adjustment coefficient of the pressure influence, weighted fusion of different detection results is performed, comprehensively considering the influence of different detection results, capable of judging and preventing the occurrence of fire faster and more accurately, realizing the early warning of the fire accident in the fuel cell microgrid, and enhancing the real-time performance and accuracy of the early warning of the fire accident in the fuel cell microgrid.

[0093] In some embodiments, the reference network sub-model includes a backbone network and an FLA attention mechanism unit. The backbone network is a RepConv convolutional network. The FLA attention mechanism unit is embedded before the last convolutional layer of the backbone network. The backbone network is used to extract multi-scale features of the target data, and the FLA attention mechanism unit is used to improve the target detection performance.

[0094] Optionally, the backbone network includes a first unit, a second unit, a third unit, and a fourth unit. The first unit is a slice structure. The second unit includes a first convolutional layer, batch normalization, and a first activation function. The third unit includes a second convolutional layer and a bottleneck layer. The fourth unit includes a third convolutional layer and a pooling layer.

[0095] It should be noted that Figure 2 is a schematic structural diagram of the RepConv convolutional network provided by the embodiments of the present application. As Figure 2 shown, the first convolutional layer, the second convolutional layer, and the third convolutional layer of the backbone network replace the original 3×3 convolutional layer with the RepConv convolutional network. When the RepConv convolutional network is trained, it is a multi-branch structure, including a 3×3 convolution, a 1×1 convolution, and an identity mapping.

[0096] In some embodiments, the FLA attention mechanism unit is embedded before the third convolutional layer. The FLA attention mechanism is improved on the basis of the Softmax attention mechanism. Figure 3 is a schematic flow diagram of the Softmax attention mechanism provided by the embodiments of the present application. As Figure 3 shown, the calculation process of the Softmax attention mechanism is as follows: first, input the matrix. The size of the query matrix Q is N×d, the size of the key matrix KT is d×N, and the size of the value matrix V is N×d, where N is the sequence length and d is the feature dimension. Then, calculate the attention scores. By multiplying the matrix QKT, an attention weight matrix of size N×N is obtained. Next, perform Softmax normalization. The attention weights are normalized through the Softmax function. Finally, multiply the normalized attention weights by the value matrix V, and the final output is N×d.

[0097] Figure 4 is a schematic flow diagram of the FLA attention mechanism provided by the embodiments of the present application. As Figure 4 shown, different from the Softmax attention mechanism, the FLA attention mechanism first multiplies the key matrix KT and the value matrix V to obtain a matrix of size d×d. Then, calculate Q(KTV), and then multiply the query matrix Q by this d×d matrix. Finally, the output is N×d. The computational complexity of FLA is O(Nd2), which is reduced by one N dimension compared to O(N2d) of Softmax. The computational complexity of the FLA attention mechanism no longer depends on the sequence length N and is suitable for processing long-sequence tasks.

[0098] In this embodiment, by embedding the FLA attention mechanism unit before the last convolutional layer of the RepConv backbone network, it is possible to adapt to scenarios of fuel cell microgrids under different lighting conditions and background noises, enabling the flame detection model to more accurately locate and identify the target area, obtaining information of a larger area while reducing the computational amount, enhancing the feature representation, detection accuracy, computational efficiency, and generalization ability of the flame detection model, being able to judge and prevent the occurrence of fires faster and more accurately, and achieving early warning of fuel cell microgrid fire accidents.

[0099] In some embodiments, the feature fusion sub-model includes a neck network and a small target detection layer. The neck network is used to fuse multi-scale features extracted by the backbone network, and the small target detection layer is used to detect small-sized flames.

[0100] It should be noted that Figure 5 is a schematic structural diagram of the neck network provided by the embodiments of the present application. As Figure 5 shown, P3, P4, and P5 are feature maps of different scales obtained after being processed by the backbone network. The neck network downsamples and downsamples and splices adjacent multi-scale feature maps, then performs feature fusion on the spliced feature maps through the Fusion module, and finally performs small-sized target detection through the small target detection layer.

[0101] In this embodiment, by using the neck network to fuse multi-scale features extracted by the backbone network, it is possible to better capture target information of different scales. By introducing the small target detection layer, the sensitivity to small-sized flames is further enhanced, improving the real-time performance and accuracy of the flame detection model for detecting small-sized flames, and achieving early warning of fuel cell microgrid fire accidents.

[0102] In some embodiments, the detection sub-model is a head network, which is used to perform target classification and bounding box regression on the multi-scale features fused by the neck network, and finally output the flame confidence of the target data.

[0103] In some embodiments, the head network analyzes the fused feature map by using a fully connected layer or a convolutional layer, maps the features to a predefined class space, and outputs which class each prediction box belongs to. At the same time, the head network also performs bounding box regression prediction on the position of the flame for each prediction box through a regression algorithm. For example, the boundary of the flame is represented by four coordinates (the upper left corner and the lower right corner). The flame confidence is calculated through the output layer of the head network. The flame confidence usually combines the classification probability and the regression accuracy, and normalizes the confidence of each flame through a Sigmoid function or a Softmax layer, indicating the possibility of whether the flame really exists.

[0104] In this embodiment, by using the head network as the detection sub-model, object classification and bounding box regression are performed on the multi-scale features fused by the neck network, and finally the flame confidence is output, which improves the accuracy of flame detection, can more effectively distinguish the flame from the background, and realizes the early warning of fuel cell microgrid fire accidents.

[0105] In some embodiments, the training process of the flame detection model includes:

[0106] Construct a preset flame detection model;

[0107] Based on the fuel cell microgrid fire video image set and the hydrogen flame image set, obtain a data set;

[0108] Based on the WloU-v3 bounding box loss function, train the preset flame detection model according to the data set to obtain the flame detection model.

[0109] It is easy to understand that first, a preset flame detection model is constructed. In order to make the flame detection model better used in fuel cell microgrid fire detection, the public flame data set and the fuel cell microgrid fire video images and hydrogen flame images collected from the network are integrated to establish the data set used in the experiment, so that the trained flame detection model can improve the anti-interference ability of flame detection in the complex environment of fuel cell microgrid while ensuring its generalization ability.

[0110] In some embodiments, the establishment process of the data set includes the following steps:

[0111] (1) On the basis of the existing public flame data set, add the fuel cell microgrid fire video images and hydrogen flame images collected from the network, and integrate the two into the data set for training the flame detection model to obtain the first training set;

[0112] (2) De-duplicate the flame images in the first training set, and eliminate the duplicate data in the public data set and the data collected from the network. The method of calculating the histogram of the image and comparing can be used. If the histogram values of two images are the same, it means the images are duplicate, and one of them is deleted to obtain the second training set;

[0113] (3) For the unlabeled images in the flame images of the second training set, use the labeling software Labelling to label the data set. After labeling the flame with a rectangular bounding box, record its bounding box coordinates, and the labeling coordinates are fire. Then save the labeling coordinate file in the txt text format available for the YOLOv5s network model to obtain the third training set;

[0114] (4) Perform data augmentation on the flame images in the third training set. The data augmentation methods include flipping, mirroring, scaling, etc. on the images in the third training set, and then repositioning them to obtain multiple augmented flame images. Combine the multiple augmented flame images with the third training set to form a dataset.

[0115] Divide the dataset into a training set, a validation set, and a test set. The flame images in the test set are test images. Input the training set and the validation set into a preset flame detection model and train it based on the WloU-v3 bounding box loss function.

[0116] The calculation formula of the bounding box loss function is as follows:

[0117] L WIoU-v3 = rR WIoU L IoU

[0118] Among them, L WIoU-v3 is the difference between the predicted bounding box and the result bounding box of the flame image, L IoU is the parameter of the overlapping part of the correct and predicted anchor boxes, R WIoU is the difference parameter between the center coordinates of the predicted box and the correct box, and r is the non-monotonic dynamic focusing coefficient.

[0119] It should be noted that the role of R WIoU is to amplify L IoU of the ordinary-quality anchor boxes, and the role of L IoU is to reduce R WIoU of the high-quality anchor boxes and reduce the attention to the distance of the center point when the anchor box and the target box overlap well. The non-monotonic dynamic focusing coefficient can reduce the contribution of easily distinguishable samples to the loss value during training, so that the model focuses on difficult-to-distinguish samples. At the same time, it dynamically gives gradient gain to the bounding box, reduces the harmful gradients generated by low-quality anchor boxes in the later stage of training, and focuses more on ordinary-quality anchor boxes to improve the model's localization performance.

[0120] The calculation formula of L IoU is as follows:

[0121]

[0122] Among them, W i is the width of the overlapping area between the target box and the predicted box, H i is the height of the overlapping area between the target box and the predicted box, S u is the area of the union of the predicted box and the true box. ω is the width of the predicted box, h is the height of the predicted box, ω gt is the width of the true box, and h gt is the height of the true box.

[0123] R WIoUThe calculation formula is as follows:

[0124]

[0125] Among them, (x, y) is the center coordinate of the predicted bounding box, and (x gt , y gt ) is the center coordinate of the ground truth bounding box. W g is the width of the smallest surrounding box that contains both the predicted bounding box and the ground truth bounding box, and H g is the height of the smallest surrounding box that contains both the predicted bounding box and the ground truth bounding box.

[0126] It should be noted that the superscript * is separated from the computational graph, that is, its gradient does not need to be calculated, and the purpose is to eliminate the factors that hinder convergence.

[0127] In some embodiments, the trained flame detection model is connected to the UI interface, which can support inputting pictures, videos, and connecting to a camera to obtain the flame detection results. The UI interface is provided with a corresponding data import module for importing picture files, video files, and datasets stored locally that contain hydrogen flame scenarios or suspected hydrogen flame scenarios.

[0128] In this embodiment, by constructing a preset flame detection model, generating a dataset based on the fuel cell microgrid fire video picture set and the hydrogen flame picture set, and training the model using the WloU-v3 bounding box loss function, the accuracy and robustness of the flame detection model are improved, it can better identify different types of flames, is applicable to different scenarios, realizes early warning of fuel cell microgrid fire accidents, and improves the real-time performance and accuracy of early warning of fuel cell microgrid fire accidents.

[0129] The flame detection method based on YOLOv5s provided in the embodiments of the present application, the execution subject can be a flame detection device based on YOLOv5s. In the embodiments of the present application, taking the flame detection device based on YOLOv5s executing the flame detection method based on YOLOv5s as an example, the flame detection device based on YOLOv5s provided in the embodiments of the present application is described.

[0130] The embodiments of the present application also provide a flame detection device based on YOLOv5s, as Figure 6 shown. The flame detection device based on YOLOv5s includes: an acquisition module 610, an establishment module 620, a first processing module 630, a second processing module 640, and a third processing module 650.

[0131] A first acquisition module 610 is used to acquire target data of the fuel cell microgrid collected by a first acquisition device, and the target data includes at least one of image data or video data;

[0132] A processing module 620 is configured to input the target data into a trained flame detection model to obtain a first detection result. The flame detection model includes a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model;

[0133] A second acquisition module 630 is configured to acquire the hydrogen storage tank pressure data of the fuel cell microgrid collected by a second acquisition device to obtain a second detection result;

[0134] A detection module 640 is configured to obtain a flame detection result of the fuel cell microgrid based on the first detection result and the second detection result;

[0135] A judgment module 650 is configured to judge whether the flame detection result is greater than or equal to a preset threshold. When the flame detection result is greater than or equal to the preset threshold, a warning message is sent.

[0136] According to the flame detection method based on YOLOv5s provided in the embodiments of the present application, by acquiring image data or video data from a first acquisition device and inputting it into a trained flame detection model, and combining a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model to obtain a first detection result, the accuracy of flame detection is effectively improved. By simultaneously acquiring the hydrogen storage tank pressure data of a second acquisition device as a second detection result and fusing it with the first detection result to obtain a flame detection result, it is possible to judge and prevent the occurrence of a fire faster and more accurately, realizing early warning of fuel cell microgrid fire accidents, improving the timeliness and accuracy of early warning of fuel cell microgrid fire accidents, and reducing potential safety hazards.

[0137] The flame detection device based on YOLOv5s provided in the embodiments of the present application can implement Figures 1 to 5 each process implemented by the embodiments of the flame detection method based on YOLOv5s. To avoid repetition, it will not be elaborated here.

[0138] In some embodiments, as Figure 7 shown, the embodiments of the present application further provide an electronic device 700, including a processor 701, a memory 702, and a computer program stored on the memory 702 and executable on the processor 701. When the program is executed by the processor 701, it implements each process of the above-mentioned embodiments of the flame detection method based on YOLOv5s and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0139] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0140] The embodiments of the present application further provide a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above-mentioned embodiment of the flame detection method based on YOLOv5s and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0141] Among them, the processor is the processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks or optical discs, etc.

[0142] The embodiments of the present application further provide a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above-mentioned flame detection method based on YOLOv5s.

[0143] Among them, the processor is the processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0144] The embodiments of the present application further provide a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above-mentioned embodiment of the flame detection method based on YOLOv5s and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0145] It should be understood that the chip mentioned in the embodiments of the present application can also be called a device-level chip, a device chip, a chip device, or an on-chip device chip, etc.

[0146] It should be noted that in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such a process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0147] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the YOLOv5s-based flame detection method of each embodiment of the present application.

[0148] In the description of the present application, "the first feature", "the second feature" may include one or more of such features.

[0149] In the description of the present application, the meaning of "a plurality" is two or more.

[0150] The embodiments of the present application have been described above with reference to the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Those of ordinary skill in the art, under the inspiration of the present application and without departing from the purpose of the present application and the scope protected by the claims, can still make many forms, all of which fall within the protection scope of the present application.

[0151] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "schematic embodiments", "examples", "specific examples", or "some examples", etc., mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0152] Although the embodiments of this application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and purposes of this application, and the scope of this application is defined by the claims and their equivalents.

Claims

1. A flame detection method based on YOLOv5s, characterized in that, The method includes: Obtaining target data of a fuel cell microgrid collected by a first acquisition device, where the target data includes at least one of image data or video data; Inputting the target data into a trained flame detection model to obtain a first detection result, where the flame detection model includes a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model; Obtaining the hydrogen storage tank pressure data of the fuel cell microgrid collected by a second acquisition device to obtain a second detection result; Based on the first detection result and the second detection result, obtaining the flame detection result of the fuel cell microgrid; Judging whether the flame detection result is greater than or equal to a preset threshold, and sending a warning message when the flame detection result is greater than or equal to the preset threshold.

2. The flame detection method based on YOLOv5s according to claim 1, wherein, The obtaining the hydrogen storage tank pressure data of the fuel cell microgrid collected by the second acquisition device to obtain a second detection result includes: Performing normalization processing on the hydrogen storage tank pressure data to obtain normalized pressure data; Processing the normalized pressure data based on the Sigmoid function, and obtaining the second detection result through the following formula: where Wp is the second detection result, P N is the normalized pressure data, and k is the parameter for controlling the weight change speed.

3. The flame detection method based on YOLOv5s according to claim 1, characterized in that, The obtaining the flame detection result of the fuel cell microgrid based on the first detection result and the second detection result includes: Fusing the first detection result and the second detection result based on a fusion analysis formula to obtain the flame detection result of the fuel cell microgrid, where the fusion analysis formula is as follows: F = V×(1 + αWp) where F is the flame detection result of the fuel cell microgrid, Wp is the second detection result, V is the first detection result, and α is an adjustment coefficient for pressure influence.

4. The flame detection method based on YOLOv5s according to claim 1, characterized in that, The benchmark network sub-model includes a backbone network and an FLA attention mechanism unit. The backbone network is a RepConv convolutional network. The FLA attention mechanism unit is embedded before the last convolutional layer of the backbone network. The backbone network is used to extract multi-scale features of the target data, and the FLA attention mechanism unit is used to improve the target detection performance.

5. The flame detection method based on YOLOv5s according to claim 4, characterized in that, The feature fusion sub-model includes a neck network and a small target detection layer. The neck network is used to fuse the multi-scale features extracted by the backbone network, and the small target detection layer is used to detect small-sized flames.

6. The flame detection method based on YOLOv5s according to claim 5, wherein, The detection sub-model is a head network, which is used to perform target classification and bounding box regression on the multi-scale features fused by the neck network, and finally output the flame confidence of the target data.

7. The flame detection method based on YOLOv5s according to claim 1, wherein The training process of the flame detection model includes: Constructing a preset flame detection model; Based on the fuel cell microgrid fire video picture set and the hydrogen flame picture set, obtaining a data set; Training the preset flame detection model based on the WloU-v3 bounding box loss function according to the data set to obtain the flame detection model.

8. A flame detection device based on YOLOv5s, which is implemented by using the flame detection method based on YOLOv5s according to any one of claims 1 to 7, characterized in that, The device includes: A first acquisition module, configured to obtain target data of a fuel cell microgrid collected by a first acquisition device, where the target data includes at least one of image data or video data; A processing module, configured to input the target data into a trained flame detection model to obtain a first detection result, where the flame detection model includes a benchmark network sub-model, a feature fusion sub-model, and a detection sub-model; A second acquisition module, configured to acquire the hydrogen storage tank pressure data of the fuel cell microgrid collected by a second acquisition device to obtain a second detection result; A detection module, configured to obtain a flame detection result of the fuel cell microgrid based on the first detection result and the second detection result; A judgment module, configured to judge whether the flame detection result is greater than or equal to a preset threshold, and send a warning message when the flame detection result is greater than or equal to the preset threshold.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the YOLOv5s-based flame detection method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the YOLOv5s-based flame detection method according to any one of claims 1 to 7.

Images