Infrared thermal imaging flame detection method applied to building construction inspection robot

Abstract

The invention discloses an infrared thermal imaging flame detection method applied to a building construction inspection robot, and relates to the technical field of building construction. The method comprises the steps that an infrared thermal imaging image of a building construction site is obtained, flame detection is conducted on the infrared thermal imaging image based on a flame detection model, a flame detection result is obtained, the flame detection model is constructed based on an improved YOLOv5 model, and the flame detection result is obtained; the network structure of the YOLOv5 model is obtained by training after hyper-parameter optimization is carried out on the improved YOLOv5 model based on a hybrid strategy improved sparrow search algorithm, and the improvement on the network structure of the YOLOv5 model comprises the following steps: adding a triple attention mechanism on an input path of a C3 module of first and third downsampling in a backbone network, and replacing the network structure of a feature fusion network with a BiFPN; the flame detection precision is improved, and the omission ratio and the false detection ratio are reduced; therefore, a flame detection result is output, and the reliability of construction safety prevention and control is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of building construction, in particular to an infrared thermal imaging flame detection method applied to a building construction inspection robot. BACKGROUND

[0002] Building construction sites usually store or use a large amount of flammable building materials such as wood, paint, solvent, plastic, etc.; and building construction machinery, electric welding, temporary distribution box and other electrical equipment are frequently used, which have fire hazards such as short circuit and electric leakage, and at the same time, frequent hot work, such as electric welding, cutting and spraying, produces open fire or high-temperature sparks. Therefore, flame detection is an important detection content of the building construction site safety inspection robot, and is an important technical means and management measure for preventing fire and ensuring construction safety. The infrared thermal imaging technology has the characteristics of day and night monitoring and penetrating visual obstacles such as smoke and dust, and has become a key means for building construction inspection robot fire detection. However, in the construction scene, flames and high-temperature interference sources (such as electric welding sparks and mechanical heat radiation) coexist, and the flame heat characteristics present complex characteristics such as variable scale and fuzzy shape. The existing detection model is prone to problems such as insufficient feature extraction and poor scene adaptability in this scene, resulting in high false detection and missed detection rates, which makes it difficult to meet the actual needs of the construction site for early detection and accurate judgment of fire hazards, and restricts the improvement of the fire prevention and control efficiency of the inspection robot in the complex construction environment. SUMMARY

[0003] In view of the above shortcomings of the prior art, the purpose of the embodiments of the present application is to provide an infrared thermal imaging flame detection method applied to a building construction inspection robot.

[0004] In order to achieve the above-mentioned purpose, the first aspect of the present application provides an infrared thermal imaging flame detection method applied to a building construction inspection robot, comprising: obtaining an infrared thermal imaging image of a building construction site; performing flame detection on the infrared thermal imaging image based on a flame detection model to obtain a flame detection result, wherein the flame detection model is constructed based on an improved YOLOv5 model, and is trained after the improved YOLOv5 model is optimized in hyperparameters based on a hybrid strategy improved sparrow search algorithm, the improved YOLOv5 model is a model obtained by improving the network structure of the YOLOv5 model, and the improvement of the network structure of the YOLOv5 model includes adding a triple attention mechanism to the input path of the C3 module in the main network for the first and third times of downsampling, and replacing the network structure of the feature fusion network with BiFPN; outputting the flame detection result.

[0005] In the embodiment of the present application, the triple attention mechanism is used for parallel processing of three branches of the down-sampled feature map, wherein the down-sampled feature map is the feature map after the first down-sampling and the feature map after the third down-sampling in the backbone network, and the three branches include a first branch, a second branch and a third branch; The first branch is used for rotating the down-sampled feature map by ninety degrees counterclockwise along the height direction, reducing the dimension to two dimensions through a Z-pool layer, generating first attention weights after convolution fusion and activation processing, and rotating the first attention weights by ninety degrees clockwise along the height direction to restore the same size as the input feature map; The second branch is used for rotating the down-sampled feature map by ninety degrees counterclockwise along the width direction, reducing the dimension to two dimensions through a Z-pool layer, generating second attention weights after convolution fusion and activation processing, and rotating the second attention weights by ninety degrees clockwise along the width direction to restore the same size as the input feature map; The third branch is used for reducing the dimension of the down-sampled feature map to two dimensions through a Z-pool layer, generating third attention weights after convolution fusion and activation processing; The triple attention mechanism is also used for aggregating the first attention weights, the second attention weights and the third attention weights by average value, and multiplying the weights obtained after the average value aggregation with the down-sampled feature map element by element to obtain an enhanced feature map.

[0006] In the embodiment of the present application, the BiFPN is used for: obtaining a multi-scale feature map, wherein the multi-scale feature map includes a high-level feature map, a middle-level feature map and a bottom-level feature map, and the multi-scale feature map is a feature map of the backbone network at different down-sampling stages, wherein the middle-level feature map is a feature map enhanced by the triple attention mechanism; obtaining a first fusion result by weighting and fusing the high-level feature map and the middle-level feature map after upsampling the high-level feature map through a top-down path; obtaining a second intermediate result by weighting and fusing the middle-level feature map and the bottom-level feature map after down-sampling the bottom-level feature map through a bottom-up path, and obtaining a second fusion result by weighting and fusing the high-level feature map and the second intermediate result after down-sampling the second intermediate result, wherein the fusion weight of the bottom-level feature map is higher than a regular weight, and the regular weight is a default weight of the high-level feature map and the middle-level feature map in fusion; obtaining an optimized multi-scale feature map through cross-scale connection integration based on the first fusion result and the second fusion result.

[0007] In the embodiment of the present application, before the step of detecting the flame based on the flame detection model to obtain the flame detection result, the method further includes: constructing an initial model based on the improved YOLOv5 model; The initial model is optimized in hyperparameters based on a hybrid strategy improved sparrow search algorithm, wherein the target hyperparameters of the initial model optimized in hyperparameters include an initial learning rate, a cycle learning rate, momentum, a weight decay coefficient, a preheating learning number, a preheating learning momentum, a preheating initial learning rate, a target frame loss weight, a classification loss weight, a confidence loss weight, an IoU threshold, and a target frame threshold. The initial model optimized in hyperparameters is trained based on a preset flame infrared thermal imaging dataset to obtain a flame detection model.

[0008] In the embodiments of the present application, the initial model is optimized in hyperparameters based on a hybrid strategy improved sparrow search algorithm, including: Target hyperparameter information of the initial model is obtained, wherein the target hyperparameter information includes the number of target hyperparameters, the value range of each target hyperparameter, and a preset initial population size; Based on the target hyperparameter information, the sparrow population is initialized by the best point set theory to generate an initial population uniformly distributed, wherein each individual of the initial population corresponds to an initial candidate value of a group of target hyperparameters; The loss function of the initial model is taken as the fitness evaluation standard, and the target hyperparameters are iteratively optimized until the number of iterations reaches the maximum number of iterations to obtain the hyperparameter combination with the highest fitness, wherein the starting point of the iterative optimization is the initial population, and the step of iteratively optimizing the target hyperparameters includes: The producer position is updated by a nonlinear inertia weight; The scavenger position is updated by a double-sample learning strategy; The population search direction is monitored and adjusted by the lookout.

[0009] In the embodiments of the present application, the sparrow population is initialized by the best point set theory based on the target hyperparameter information to generate an initial population uniformly distributed, including: The dimension of a multi-dimensional search space is determined according to the number of target hyperparameters, and the smallest prime number meeting the preset dimension constraint condition is selected; Based on the smallest prime number, a group of uniformly distributed reference points are generated in the multi-dimensional search space by the best point set theory; According to the value range of each target hyperparameter, the relative positions of each reference point are mapped to specific target hyperparameter values to generate individuals consistent with the preset initial population size, each individual includes initial candidate values of all target hyperparameters, and all individuals are uniformly distributed in the multi-dimensional search space, collectively constituting the initial population.

[0010] In the embodiments of the present application, the producer position is updated by a nonlinear inertia weight, including: The current number of iterations, the maximum number of iterations, and the target hyperparameter combination currently corresponding to the producer are obtained; The inertia weight is determined based on the iteration progress and a nonlinear relationship based on a ratio of the current iteration number and the maximum iteration number, wherein the inertia weight has a weight value and a weight reduction rate in the early iteration stage that are greater than those in the later iteration stage; The search step of the producer is adjusted based on the inertia weight to update the position of the producer.

[0011] In the embodiments of the present application, the position of the scavenger is updated by a double-sample learning strategy, including: The scavenger simultaneously refers to a first position and a second position in the current population, wherein the first position is the position of the producer with the highest fitness, and the second position is the position of another randomly selected producer; The learning weight of the first position and the second position is adjusted according to the ratio of the current iteration number and the maximum iteration number; The position of the scavenger is updated based on the difference between the target hyperparameter value corresponding to the first position and the target hyperparameter value corresponding to the second position, and the adjusted learning weight.

[0012] The second aspect of the present application provides a building construction inspection robot, comprising: a memory configured to store instructions; a processor configured to call the instructions from the memory and capable of implementing the infrared thermal imaging flame detection method applied to the building construction inspection robot as described in the above embodiments when executing the instructions.

[0013] The third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to cause a machine to execute the infrared thermal imaging flame detection method applied to the building construction inspection robot as described in the above embodiments.

[0014] By the technical solution, the infrared thermal imaging image of the construction site is acquired, the heat radiation characteristics of the flame are accurately captured by breaking through the environmental restrictions such as illumination and smoke, reliable data basis is provided for subsequent detection, and effective perception of the flame target in a complex construction scene is ensured; and the infrared thermal imaging image is detected based on the flame detection model to obtain a flame detection result, wherein the flame detection model is constructed based on an improved YOLOv5 model, and is trained by optimizing the hyperparameters of the improved YOLOv5 model based on a hybrid strategy improved sparrow search algorithm, the improved YOLOv5 model is a model obtained by improving the network structure of the YOLOv5 model, and the improvement of the network structure of the YOLOv5 model includes adding a triple attention mechanism on the input path of the C3 module of the first and third down-sampling in the backbone network, and replacing the network structure of the feature fusion network with BiFPN; the accuracy of the flame detection is effectively improved, especially the recognition ability of small target flames and weak feature flames is significantly enhanced, and the miss detection rate and the false detection rate are reduced; and thus the flame detection result is output, real-time feedback of the fire hazard is realized, accurate basis is provided for safety warning and emergency disposal of the construction site, and the timeliness and reliability of the construction safety prevention and control are improved.

[0015] Other features and advantages of the embodiments of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the embodiments of the present application together with the following specific embodiments, but do not constitute a limitation of the embodiments of the present application. In the drawings: Figure 1 A flowchart of an infrared thermal imaging flame detection method applied to a construction inspection robot according to an embodiment of the present application is schematically shown; Figure 2 A network structure diagram of a flame detection model according to an embodiment of the present application is schematically shown; Figure 3 A refined network structure diagram of a flame detection model according to an embodiment of the present application is schematically shown; Figure 4 A network structure diagram of a triple attention mechanism according to an embodiment of the present application is schematically shown; Figure 5 A YOLOv5 feature fusion principle and a PANet network structure diagram according to an embodiment of the present application are schematically shown; Figure 6 An improved YOLOv5 weighted bidirectional feature fusion principle and a BiFPN network structure diagram according to an embodiment of the present application are schematically shown; Figure 7 Fig. 7 schematically shows a population distribution diagram of random initialization and local optimum initialization according to an embodiment of the present application; Figure 8 Fig. 8 schematically shows a curve diagram of adaptive inertia weight factor according to an embodiment of the present application; Figure 9 Fig. 9 schematically shows a contrast diagram of iteration curves of standard test functions F1-F8 according to an embodiment of the present application; Figure 10 Fig. 10 schematically shows a curve diagram of average loss value in flame detection model training process according to an embodiment of the present application; Figure 11 Fig. 11 schematically shows a flame detection result diagram of thermal imaging according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are merely used to explain and illustrate the embodiments of the present application and should not be used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0018] It should be noted that the acquisition, transmission, storage, use, processing and the like of data in the technical solutions of the present application comply with relevant provisions of laws and regulations. In the embodiments of the present application, some industry existing solutions such as software, components, models and the like may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility in the implementation of the technical solutions of the present application.

[0019] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, motion condition and the like between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional indications also change accordingly.

[0020] In addition, if the description of "first", "second" and the like is involved in the embodiments of the present application, the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor in the protection scope claimed by the present application.

[0021] Figure 1 The flowchart of the infrared thermal imaging flame detection method applied to the building construction inspection robot according to the embodiments of the present application is schematically shown. As shown in Figure 1 The infrared thermal imaging flame detection method applied to the building construction inspection robot is provided in the embodiments of the present application, which can include the following steps: Step 100, acquiring the infrared thermal imaging image of the building construction site; In the embodiments, it should be noted that the body or the holder of the building construction inspection robot integrates the infrared thermal imager, which captures the infrared radiation difference of different areas in the building construction site through the infrared thermal imager, and converts it into a gray scale or pseudo-color image, i.e. the infrared thermal imaging image. Infrared thermal imaging breaks through the limitation of visible light and can work in the daytime and at night, and can penetrate visual obstacles such as fog, smoke and dust, and is suitable for the complex and changeable environment of the building construction site. The image acquisition of the building construction inspection robot on the building construction site should have spatial coverage and time continuity. It is necessary to cover the whole area of the construction site, for example, from the building material yard to the temporary fire point, to avoid missing fire hazards; through continuous acquisition, the dynamic process of flame "temperature mutation → diffusion" can be captured, such as the heat spread of the fire spark and the gradual trend of the temperature rise of the electrical equipment, to provide complete data chain for subsequent detection.

[0022] Step 200, performing flame detection on the infrared thermal imaging image based on a flame detection model to obtain a flame detection result, wherein the flame detection model is constructed based on an improved YOLOv5 model, and is trained after the improved YOLOv5 model is optimized in hyperparameters based on a hybrid strategy improved sparrow search algorithm, the improved YOLOv5 model is a model obtained by improving the network structure of the YOLOv5 model, and the improvement of the network structure of the YOLOv5 model includes: adding a triple attention mechanism on the input path of the C3 module in the main network for the first and third times of downsampling, and replacing the network structure of the feature fusion network with BiFPN; It should be noted that in the original YOLOv5 model, the C3 module in the backbone network can extract target features through two branch structures, but the cross-stage connection structure causes the loss of feature information, resulting in a decrease in network feature extraction accuracy. The construction site fire hazards are complex, and the flame under infrared thermal imaging is easily interfered by smoke and mechanical heat radiation. The traditional target detection model based on the YOLOv5 model is difficult to accurately identify multi-scale and weak feature flames, so it is necessary to construct a flame detection model adapted to the construction site scene.

[0023] Please refer to Figure 2 and Figure 3 , the present embodiment proposes a flame detection model based on the YOLOv5 model, which improves the network structure, including: adding a triple attention mechanism (TAM) to the input path of the C3 module in the YOLOv5 backbone network at the first and third downsampling, strengthening feature expression, and learning more feature information through three branch structures to avoid information loss when directly inputting the C3 module. And the TAM attention module will not affect the parameter amount of the model.

[0024] It should be noted that the original YOLOv5 model uses a PANet (Path Aggregation Network) structure in the feature fusion network part (Neck), which mainly focuses on feature fusion and information transmission between layers, and learns less about small target feature information. In the feature fusion stage, part of the detail information will be lost. Therefore, in the present embodiment, the improvement of the YOLOv5 model also includes: replacing the PANet in the feature fusion network part with a BiFPN (Bi-directional Feature Pyramid Network, weighted bidirectional multi-scale feature fusion pyramid network), which can quickly realize multi-scale feature fusion. Through the weighted bidirectional structure, the details of different input features are fused, the loss of detail information is improved, the detection ability of small targets is improved, and the multi-scale feature integration is optimized.

[0025] It should be noted that in the present embodiment, the improved YOLOv5 model is optimized by using a hybrid strategy improved sparrow search algorithm for hyperparameters such as learning rate, anchor box size, etc. to improve the adaptability of the flame detection model to the construction site scene. The process is achieved through "network structure improvement → hyperparameter optimization → model training". The flame detection model in the present embodiment breaks through the detection bottleneck of the traditional model in complex scenes, improves the accuracy and efficiency of flame detection, and enhances the timeliness and reliability of fire warning.

[0026] Specifically, the triple attention mechanism is used for parallel processing of three branches of the down-sampled feature map, wherein the down-sampled feature map is the feature map after the first and third down-sampling in the backbone network, and the three branches include a first branch, a second branch and a third branch; The first branch is used for rotating the down-sampled feature map by ninety degrees counterclockwise along the height direction, dimensionally reducing the down-sampled feature map to two dimensions through a Z-pool layer, generating first attention weights after convolution fusion and activation processing, and rotating the first attention weights by ninety degrees clockwise along the height direction to restore the same size as the input feature map; The second branch is used for rotating the down-sampled feature map by ninety degrees counterclockwise along the width direction, dimensionally reducing the down-sampled feature map to two dimensions through a Z-pool layer, generating second attention weights after convolution fusion and activation processing, and rotating the second attention weights by ninety degrees clockwise along the width direction to restore the same size as the input feature map; The third branch is used for dimensionally reducing the down-sampled feature map to two dimensions through a Z-pool layer, generating third attention weights after convolution fusion and activation processing; The triple attention mechanism is also used for aggregating the first attention weights, the second attention weights and the third attention weights by an average value, and multiplying the weights obtained after the average value aggregation with the down-sampled feature map element by element to obtain an enhanced feature map.

[0027] In this embodiment, it should be noted that the spatial details of the features are easily lost by the down-sampling of the backbone network, and the triple attention mechanism enhances the spatial attention of the down-sampled feature map through multi-branch parallel processing.

[0028] Specifically, the attention mechanism is beneficial to enhance the attention degree of the network to the region of interest while suppressing the attention to the irrelevant region. The most commonly used attention mechanisms in computer vision and deep learning include channel attention mechanism and spatial attention mechanism. The triple attention mechanism can model the channel attention and spatial attention at the same time, encode the channel and spatial information with negligible additional computational overhead by introducing rotation operation and residual transformation to build the dependency relationship between different dimensions. The triple attention mechanism module is composed of three parallel branch structures, two of which are responsible for capturing the cross-dimensional interaction between the feature attribute dimension (channel C) and the spatial height (space H) or the spatial width (space W), and the last branch is used to build spatial attention. The outputs of the three branches are aggregated using the average.

[0029] It should be noted that the down-sampled feature map refers to the features output after the first and third down-sampling of the backbone network; the Z-pool layer is used to compress the spatial dimension of the feature map, such as reducing three-dimensional features to two-dimensional features; the structural diagram of the triple attention mechanism can be referred to Figure 4The input down-sampling feature map is passed to three branches in the module: in the first branch, the interaction between the space H and the channel C is established, the input down-sampling feature map is first rotated counterclockwise by 90° along the height direction, then simplified through the Z-pool layer to reduce the dimension of the channel C to two-dimensional reserved tensor content while reducing the depth to reduce the amount of calculation, then the feature fusion is performed through the 7*7 convolution with a large receptive field, in order to keep consistent with the input size, finally the attention weight is generated through the sigmoid activation layer, that is, the first attention weight, which is used for the 90° clockwise rotation and splicing superposition operation along the height direction, so that the obtained branch tensor returns to the original consistent size of the input feature map. In the second branch, the interaction between the space W and the channel C is established, the input down-sampling feature map is first rotated counterclockwise by 90° along the width direction, then simplified through the Z-pool layer to reduce the dimension of the channel C to two-dimensional reserved tensor content while reducing the depth to reduce the amount of calculation, then the feature fusion is performed through the 7*7 convolution with a large receptive field, in order to keep consistent with the input size, finally the attention weight is generated through the sigmoid activation layer, that is, the second attention weight, which is used for the 90° clockwise rotation and splicing superposition operation along the width direction, so that the obtained branch tensor returns to the original consistent size of the input feature map. The third branch does not perform rotation processing on the feature map, and the other structures are consistent with the first branch, which is used to construct the spatial attention, that is, the third attention weight. Finally, the refined shape tensors generated by the three branches are aggregated through the average method to output a fine tensor with the same shape. The triple attention mechanism enhances the attention of the target feature by strengthening different dimensional features, strengthens the effective features, and more fully learns the target feature information. And it can make up for the loss of down-sampling features, so that the flame detection model can more accurately extract the spatial features of the flame and reduce the influence of background interference on detection.

[0030] In this embodiment, by fusing the triple attention mechanism module in the backbone network part, the dependence between different dimensions is established through rotation operation and residual transformation, the channel information and spatial information related to the missing target in the feature map are increased, the receptive field is enhanced through the three-channel branch to improve the feature representation of the network structure, thereby enhancing the ability to suppress similar contour background interference and improving the flame feature extraction capability.

[0031] Specifically, the BiFPN is used for: obtaining a multi-scale feature map, wherein the multi-scale feature map includes a high-level feature map, a middle-level feature map, and a bottom-level feature map, the multi-scale feature map is a feature map of the backbone network at different down-sampling stages, and the middle-level feature map is a feature map enhanced by the triple attention mechanism; obtaining a first fusion result by weighting and fusing the high-level feature map and the middle-level feature map after upsampling the high-level feature map through a top-down path; After downsampling the bottom-level feature map through a bottom-up path, a second intermediate result is obtained by weighted fusion with the middle-level feature map. After downsampling the second intermediate result, a second fusion result is obtained by weighted fusion with the high-level feature map. The fusion weight of the bottom-level feature map is higher than the conventional weight. The conventional weight is the default weight when fusing the high-level feature map and the middle-level feature map. Based on the first fusion result and the second fusion result, an optimized multi-scale feature map is obtained through cross-scale connection integration.

[0032] It should be noted that the feature fusion network of the traditional YOLOv5 model uses PANet, such as Figure 5 As shown, Figure 5 include Figure 5 (a) and Figure 5 (b). PANet is an improvement on the Feature Pyramid Network (FPN). It consists of a bottom-up path and a top-down path. It is a feature fusion network for convolutional neural network models. The bottom-up path is responsible for extracting the underlying features layer by layer to generate a pyramid feature map. The top-down path starts with the highest-level features and fuses them layer by layer with the adjacent lower-level features. However, flames in construction scenes have multi-scale differences, such as small targets such as electric welding sparks and large targets such as open flames. Conventional feature fusion cannot take into account both details and semantics. Therefore, in this embodiment, BiFPN is used to replace PANet.

[0033] It should be noted that BiFPN's weighted bidirectional multi-scale feature fusion pyramid structure processes the bidirectional path structure in PANet into a feature network layer by fusing higher-level features in the multi-scale reinforcement path. Compared with PANet, BiFPN can differentiate and fuse different features according to their importance. BiFPN consists of a bottom-up original path aggregation network and an additional path from input to output node, such as Figure 6 As shown, Figure 6 include Figure 6 (a) and Figure 6 (b) BiFPN requires optimizing the fusion of multi-scale features. Multi-scale feature maps include high-level, mid-level, and low-level feature maps. High-level feature maps are rich in semantic information, such as the overall outline of a flame; mid-level feature maps are enhanced by triple attention, resulting in more focused features; and low-level feature maps contain ample detail, such as the edges of small flames.

[0034] Specifically, BiFPN fuses high layers by top-down up-sampling and fuses middle layers by top-down up-sampling, fuses low layers by bottom-up down-sampling and fuses middle and high layers by bottom-up down-sampling, and increases the fusion weight of low layer features, such as the weight of low layer is higher than the default weight when high and middle layers are fused; and the retention of small target details is strengthened. For example, after the weight of low layer features is increased, small flame details such as electric welding sparks can be captured more clearly. The imbalance problem of multi-scale flame detection is solved, multi-scale features are effectively integrated, the detection accuracy of large and small scale flames is improved, and the adaptability of the model to complex flame patterns is enhanced.

[0035] In this embodiment, by replacing the BiFPN module, the improved YOLOv5 model can learn more detailed information in the context information in the feature fusion network part, multi-scale fusion enables the model to better extract small-scale flame target feature information, and the bidirectional cross-scale weighted feature pyramid structure realizes higher level feature fusion, which can quickly and effectively realize target feature detection and positioning.

[0036] Step 300, outputting the flame detection result.

[0037] In this embodiment, it should be noted that the real-time response of the construction site fire hazard depends on the effective output of the detection result, therefore, outputting the flame detection result is the core link of the fire warning closed loop. The flame detection result usually includes target judgment, spatial positioning and confidence. Among them, the target judgment can include "flame", "high temperature interference" and other categories; the spatial positioning is marked by the boundary box coordinates, such as the upper left corner (x1, y1) and the lower right corner (x2, y2) in the image coordinate system mark the flame position; the confidence is the probability quantization of the model to the detection result, such as 0.9 represents 90% confidence. For example, when the inspection robot detects the open fire in the electric welding operation area, the flame detection result may be output as "category: flame, position: (200, 150)-(400, 350), confidence: 0.95".

[0038] In this embodiment, the infrared thermal imaging image of the construction site is obtained, breaking through the environmental restrictions such as light and smoke, accurately capturing the thermal radiation characteristics of the flame, and providing reliable data basis for subsequent detection, and ensuring effective perception of the flame target in a complex construction scene. The infrared thermal imaging image is detected based on a flame detection model to obtain a flame detection result, wherein the flame detection model is constructed based on an improved YOLOv5 model, and is trained based on a hybrid strategy improved sparrow search algorithm for super parameter optimization of the improved YOLOv5 model. The improved YOLOv5 model is a model obtained by improving the network structure of the YOLOv5 model. Improving the network structure of the YOLOv5 model includes adding a triple attention mechanism to the input path of the C3 module in the first and third downsampling of the backbone network, and replacing the network structure of the feature fusion network with BiFPN. The accuracy of flame detection is effectively improved, especially the recognition ability of small target flame and weak feature flame is significantly enhanced, and the miss detection rate and false detection rate are reduced. Thus, the flame detection result is output, real-time feedback of fire hazards is realized, accurate basis is provided for safety warning and emergency disposal of the construction site, and the timeliness and reliability of construction safety prevention and control are improved.

[0039] In one embodiment, before the step of detecting the infrared thermal imaging image based on the flame detection model to obtain the flame detection result, it further includes: constructing an initial model based on the improved YOLOv5 model; optimizing the super parameters of the initial model based on the hybrid strategy improved sparrow search algorithm, wherein the target super parameters of the initial model for super parameter optimization include the initial learning rate, the cycle learning rate, the momentum, the weight decay coefficient, the preheating learning number, the preheating learning momentum, the preheating initial learning rate, the target box loss weight, the classification loss weight, the confidence loss weight, the IoU threshold, and the target box threshold; training the initial model with optimized super parameters based on a preset flame infrared thermal imaging dataset to obtain the flame detection model.

[0040] In this embodiment, it should be noted that constructing a flame detection model adapted to the construction scene needs to consider the feature extraction capability of the network structure and the rationality of the super parameter configuration. The traditional model has fixed network structure and depends on experience setting of super parameters, and has problems such as insufficient detection accuracy and weak generalization ability in complex scenes. Therefore, the model performance needs to be improved by improving the network structure and optimizing the super parameters. The improved YOLOv5 model is a targeted adjustment of the original network structure, and the specific improved YOLOv5 model structure can refer to the above embodiment.

[0041] The hybrid strategy-improved sparrow search algorithm is used to optimize the target hyperparameters. It is understandable that the traditional YOLOv5 model has 25 hyperparameters, such as learning rate, weight decay coefficient, momentum, target box loss weight, data enhancement coefficient, etc., which are used for different training settings. The essence of the swarm intelligence algorithm to optimize the selection of hyperparameters is to combine multiple important hyperparameters into a multidimensional solution space, apply the hybrid strategy-improved sparrow search algorithm proposed in this embodiment to iteratively search, and obtain the multidimensional solution corresponding to the global optimal fitness value as the optimal hyperparameter combination of the improved YOLOv5 model. When the dimension of the hyperparameter combination is higher, the computational cost and time complexity of the optimization algorithm will increase. Since data enhancement processing has been performed in the dataset construction part of this embodiment, in order to improve the algorithm optimization efficiency, the 13 hyperparameters related to the data enhancement coefficient in the traditional YOLOv5 model are no longer optimized. These parameters use the default hyperparameter values ​​in the pre-training model for model training. Therefore, 12 hyperparameters including initial learning rate, cyclic learning rate, momentum, weight decay coefficient, number of warm-up learning times, warm-up learning momentum, warm-up initial learning rate, target box loss weight, classification loss weight, confidence loss weight, IoU threshold, and target box threshold are selected as target hyperparameters for optimization.

[0042] In this embodiment, the initial model after hyperparameter optimization is trained based on a preset flame infrared thermal imaging dataset, which can make the model more accurate in learning flame characteristics. The resulting flame detection model can better adapt to the complex environment of the construction scene.

[0043] In this embodiment, the improved network structure enhances the ability to extract and fuse flame features, and the precise optimization of hyperparameters improves the training efficiency and generalization performance of the model. The trained flame detection model can effectively identify multi-scale, weak-feature flame targets, meeting the high-precision requirements of fire detection in construction scenarios.

[0044] In one embodiment, the hyperparameter optimization of the initial model is performed based on the hybrid strategy-based improved sparrow search algorithm, including: Obtaining target hyperparameter information of the initial model, wherein the target hyperparameter information includes the number of target hyperparameters, the value range of each target hyperparameter, and the preset initial population size; Based on the target hyperparameter information, the sparrow population is initialized using the good point set theory to generate a uniformly distributed initial population, where each individual in the initial population corresponds to a set of initial candidate values ​​for the target hyperparameter. Using the loss function of the initial model as the fitness evaluation criterion, iteratively optimize the target hyperparameters until the number of iterations reaches the maximum number of iterations, and obtain the hyperparameter combination with the highest fitness. The starting point of the iterative optimization is the initial population. The steps of iteratively optimizing the target hyperparameters include: Update producer positions using nonlinear inertia weights; The scavenger position is updated by a two-sample learning strategy; The population search direction is monitored and adjusted by the sentinels.

[0045] It should be noted that the sparrow search algorithm (SSA, Sparrow Search Algorithm) is a swarm intelligence algorithm based on the social characteristics of sparrow population. Compared with commonly used genetic algorithm, particle swarm optimization algorithm, grey wolf optimization algorithm and the like, the sparrow search algorithm has the advantages of simple structure, few control parameters, strong local search ability and the like. The sparrow search algorithm defines the identity of the sparrow according to whether better food can be found, simulates the foraging and anti-predation behavior of sparrows, and divides the sparrow population into producers, scavengers and sentinels according to different divisions. The proportion of the number of sentinels to the number of the population is 10-20%, and the number of producers and scavengers is dynamically changed. In the foraging process of the sparrow population, the positions of the three are updated to complete the search and acquisition of food resources. The basic principle of the sparrow search algorithm is described as follows: Assuming that the population has n sparrows, the population X composed of all individuals and the fitness function F of each individual x can be represented as: ; ; wherein D represents the dimension of the input variable of the problem to be optimized, n represents the number of sparrows, and f(x) represents the fitness function.

[0046] The producer has a larger fitness value, is responsible for finding the direction of foraging for the population, and the position update formula of the producer in the behavior of searching for food resources is as follows:

[0047] wherein represents the position information of the i-th sparrow in the j-th dimension at the t+1-th iteration of the producer; represents the position information of the i-th sparrow in the j-th dimension at the t-th iteration of the producer; α represents a random number between 0 and 1; iter max represents the maximum number of iterations; R2 is the early warning value of the position of the sparrow population, R2 ∈ [0, 1]; ST is the safety threshold of the position of the sparrow population, ST ∈ [0.5, 1]; Q is a random number subject to normal distribution; and L is a D-dimensional matrix with all elements being 1. R2 < ST indicates that there may be a better solution in the region, triggering aggressive exploration, for example, refining the step size through an exponential term; and R2 ≥ ST indicates that the region is relatively safe, performing conservative update, for example, directly superimposing a random step size.

[0048] The scavenger will monitor the producer at all times due to the small fitness value set. When the producer finds better food, the scavenger will immediately leave the current position to compete for the food. If the scavenger wins the competition, the producer's food can be immediately obtained. The position update formula of the scavenger is as follows:

[0049] wherein, represents the position information of the ith sparrow in the jth dimension at the t+1th iteration of the scavenger; represents the position information of the ith sparrow in the jth dimension at the tth iteration of the scavenger; represents the current global worst position; n represents the population number of the scavenger; represents the current optimal position of the producer; A is a 1*D matrix, wherein the elements are randomly assigned as 1 or -1, and A + =A T (AA T ) -1 ; Q is a random number subject to a normal distribution; and L is a D-dimensional matrix with elements being 1.

[0050] The alert will monitor the safety environment of the foraging area. When encountering a threat from a natural enemy, the sparrow at the edge of the group will take anti-predation behavior and will quickly move to a safe area. The position update formula of the alert is as follows:

[0051] wherein, represents the position information of the ith sparrow in the jth dimension at the t+1th iteration of the alert; represents the position information of the ith sparrow in the jth dimension at the tth iteration of the alert; represents the current global optimal position; represents the current global worst position; β is a random number subject to a standard normal distribution; f i , f g and f w respectively represent the fitness value of the current individual, the global optimal fitness value and the global worst fitness value; K is a random number in [0, 1]; and ε is a minimum constant for avoiding f i =f g when the denominator is zero.

[0052] The standard sparrow search algorithm simply and randomly establishes an initial population, and individual sparrows are randomly distributed within the search space, which creates uncertainty in the algorithm's operation. The scavenger's position update is only related to the optimal and worst positions of the current population, which makes it easy for the algorithm to fall into a local optimal solution in the early stages. In the later stages of the algorithm, the diversity of the producer population decreases and gradually tends to be single, which affects the algorithm's global optimization ability and reduces its ability to discover the optimal position. Therefore, based on the above issues, this embodiment proposes a hybrid strategy to improve the sparrow search algorithm. Specifically, it involves a hybrid strategy involving three improvement measures to improve the sparrow search algorithm and enhance its optimization performance.

[0053] Specifically, obtaining the target hyperparameter information of the initial model and performing efficient optimization are the keys to improving model training results. Traditional hyperparameter optimization often suffers from low optimization efficiency and unsatisfactory results due to the lack of initial information or a single search strategy. Therefore, it is necessary to ensure the effectiveness of hyperparameter optimization through systematic information acquisition, population initialization and iterative strategies.

[0054] Target hyperparameter information is the basis of optimization. The number of target hyperparameters clearly defines the total number of parameters to be optimized, such as the 12 key parameters that affect model training given in the above embodiment; the value range of each target hyperparameter limits the reasonable interval of each parameter, such as the initial learning rate can be set between 0.001 and 0.1; the preset initial population size determines the number of initial samples for the search, such as 30 individuals; the three together provide clear search boundaries and starting scales for subsequent optimization.

[0055] First, we initialize the sparrow population based on the theory of good point sets, generating an initial population uniformly distributed across the hyperparameter space. Each individual corresponds to a set of initial candidate hyperparameter values. Compared to random initialization, this approach avoids localized initial sample concentration, ensuring that the search covers a wider range of potential optimal solutions, providing a diverse starting point for iterative optimization. Fitness is evaluated using the loss function of the initial model: the smaller the model loss corresponding to the hyperparameter combination, the higher the fitness, which serves as a measure of hyperparameter quality.

[0056] Second, iterative optimization begins with the initial population and optimizes hyperparameters through collaborative updates by producers, scavengers, and sentinels. Producers leverage nonlinear inertia weights to explore unknown areas with larger steps in the early stages of an iteration, and refine their search with smaller steps later, balancing global exploration with local exploitation. Scavengers employ a two-sample learning strategy, referencing both optimal and random hyperparameter combinations. Initially, they prioritize random samples to maintain diversity, while later, they prioritize optimal samples for accelerated convergence. Sentinels monitor the search area in real time, adjusting their search direction if no better solution is found, to prevent the population from falling into a local optimum. When the maximum number of iterations is reached, the hyperparameter combination with the highest fitness is output.

[0057] In this embodiment, the target hyperparameter information provides clear boundaries for optimization, the uniform initial population ensures comprehensive search, and the multi-strategy iterative optimization balances search efficiency and optimization accuracy. The final hyperparameter combination can significantly improve the training efficiency and performance stability of the initial model, laying a foundation for subsequent model adaptation to complex scenarios.

[0058] Specifically, in one embodiment, based on the target hyperparameter information, a sparrow population is initialized by the good point set theory to generate a uniformly distributed initial population, including: The dimension of the multi-dimensional search space is determined according to the number of target hyperparameters, and the smallest prime number that meets the preset dimension constraint condition is selected; Based on the smallest prime number, a set of uniformly distributed reference points in the multi-dimensional search space is generated by the good point set theory; According to the value range of each target hyperparameter, the relative position of each reference point is mapped to a specific target hyperparameter value, generating individuals consistent with the preset initial population size. Each individual contains initial candidate values of all target hyperparameters, and all individuals are uniformly distributed in the multi-dimensional search space, collectively forming the initial population.

[0059] The initialization of population individual position is the basis of swarm intelligence algorithm search, and the diversity of population distribution will affect the optimization ability of swarm intelligence algorithm. The more uniform the initial population distribution, the easier it is for the algorithm to search for the global optimal solution in a shorter time. In recent years, a large number of population initialization methods have been applied to the initial stage of swarm intelligence algorithms, and experiments have shown that compared with chaotic mapping, the frequency distribution error of good point set is smaller and the population distribution is significantly better. Therefore, in this embodiment, the first improvement measure involves initializing the search space of the sparrow search algorithm according to the good point set theory. The mathematical theory of the good point set of the sparrow initial population is as follows: (1) Assuming that G is a unit cube in a t-dimensional Euclidean space, the good point set P of n points in a t-dimensional space is n The expression of (k) is as follows:

[0060] Where P n (k) represents a good point set containing n points in a t-dimensional Euclidean space; t represents the hyperparameter dimension; n represents the number of points in the good point set, corresponding to the initial individual number of the sparrow population, i.e., the preset initial population size; K represents the iteration index, taking values in the range [1, n], and traversing to generate the dimension coordinates of each point; represents the value of the nth point in the tth dimension, represents the decimal part of the point , which maps the coordinates to the [0, 1) interval.

[0061] (2) Pn The deviation φ(n) of (k) satisfies the following formula:

[0062] Among them, φ(n) is a constant that is only related to the values ​​of x and σ, which represents the deviation of the good point set and measures the uniformity of the points in space; σ is an arbitrary positive number; n represents the number of points in the good point set; ε represents a small positive number to ensure that the deviation decreases as n increases.

[0063] (3) The calculation formula of the original value of the good point is as follows:

[0064] Where xi represents the original coordinates of the good point in the i-th dimension, which is (1) The base value of t; p is the smallest prime number that satisfies the condition (p-3) / 2≥ t.

[0065] (4) Use the following formula to convert the optimal point x i Mapping to the search space of the sparrow search algorithm:

[0066] Among them, X i (j) represents the initial position of the i-th sparrow individual in the j-th dimension; ub j and lb j Represent the upper and lower boundaries of the j-th dimension respectively; k represents the iteration index, with a value range of [1, n], which traverses to generate the dimension coordinates of each point; Represents the original value of the best point of the i-th point in the j-th dimension.

[0067] refer to Figure 7 , Figure 7 include Figure 7 (a) and Figure 7 (b). Figure 7 The population distribution diagram of the good point set initialization when the population size is 200, the dimension is 2, and the value boundary of each point is [0, 1] is shown in Figure 7 (b) and randomly initialized population distribution diagram Figure 7 (a). Comparison shows that the distribution of individuals in the population of the good point set is significantly more uniform than that of random initialization, fully covering the search space of the sparrow search algorithm. This gives the algorithm a better traversal advantage in the early stages and improves the algorithm's global search capability.

[0068] In one embodiment, updating the producer position by a nonlinear inertia weight includes: Get the current number of iterations, the maximum number of iterations, and the target hyperparameter combination corresponding to the producer; An inertia weight is determined based on an iteration progress and a nonlinear relationship based on a ratio of a current iteration number and a maximum iteration number, wherein the inertia weight has a weight value and a weight reduction rate in an early iteration stage that are greater than those in a late iteration stage; A search step of the producer is adjusted based on the inertia weight to update a position of the producer.

[0069] In the standard sparrow search algorithm, the producer is prone to deviate from the search direction and miss the optimal foraging area when updating the position due to lack of effective control of the step, resulting in the algorithm falling into a local optimum. Therefore, a nonlinear inertia weight factor ω is proposed in this paper to improve the position update of the producer, and the calculation formula is as follows:

[0070] where ω represents the nonlinear inertia weight factor; iter max represents the maximum iteration number, and t represents the current iteration number.

[0071] The iteration number is set to 500, and the change curve of the inertia weight factor ω is as shown in Figure 8 In the early iteration stage, the inertia weight factor ω has a larger value and a faster nonlinear change rate, which can enable the individual to quickly explore the unknown area of the population and maintain a high global search ability of the algorithm, thereby inhibiting the problem of premature fitting. In the late iteration stage, the inertia weight factor ω has a smaller value and a slower nonlinear change rate, which helps the individual to focus on finding a better position update scheme in the known area of the search population, thereby improving the local development ability of the algorithm and accelerating the convergence speed of the algorithm.

[0072] In the iteration process, the continuous adaptive change of the inertia weight ω will help to improve the balance between the search space and the development ability of the algorithm. The improved position update formula of the producer is as follows:

[0073] where, represents the position information of the ith sparrow in the jth dimension at the t+1th iteration of the producer; represents the position information of the ith sparrow in the jth dimension at the tth iteration of the producer; ω represents the nonlinear inertia weight factor; i represents the index of the sparrow individual, which distinguishes different producers, and the larger the index, the faster the exponential term decays; a represents a random number between 0 and 1; iter max is the maximum iteration number; R2 is a warning value of the sparrow population position, R2 ∈ [0, 1]; ST is a safety threshold of the sparrow population position, ST ∈ [0.5, 1]; Q is a random number subject to a normal distribution; and L is a D-dimensional matrix with all elements being 1.

[0074] In an embodiment, the gleaner position is updated by a double-sample learning strategy, including: The scavenger simultaneously refers to a first position and a second position in a current population, the first position is a position of a producer with the highest fitness, and the second position is a position of another randomly selected producer; According to the ratio of the current iteration number and the maximum iteration number, the learning weight of the first position and the second position is adjusted; Based on the difference between the target hyperparameter value corresponding to the first position and the target hyperparameter value corresponding to the second position, and the adjusted learning weight, the position of the scavenger is updated.

[0075] In the standard sparrow search algorithm, the position update formula of the scavenger is calculated by the position of the best individual and the position of the worst individual in the current population, and the scavenger can only learn from one individual in each iteration update, and the information exchange between sparrow populations is not fully utilized. If the best individual in the current population falls into local optimum at this time, other scavenger individuals will quickly gather, resulting in that the whole population falls into local optimum, and the algorithm converges into a local optimal solution.

[0076] The application proposes to improve the position update of the scavenger by using a double-sample learning strategy, and in each iteration, the scavenger moves to the optimal position of the current producer and the position of a randomly selected producer, and the two position samples are compared and learned to help the scavenger move faster to the global optimal position in the search space. The improved scavenger position update formula is as follows:

[0077] Wherein, represents the position information of the ith sparrow in the jth dimension at the t+1th iteration of the scavenger; represents the position information of the ith sparrow in the jth dimension at the tth iteration of the scavenger; represents the current global worst position; n represents the number of scavenger populations; represents the optimal position of the current producer, i.e. the optimal sample; represents the position of a randomly selected producer, i.e. a random sample; A is a 1*D matrix, wherein the elements are randomly assigned 1 or -1, and A + =A T (AA T ) -1 ; Q is a random number subject to normal distribution; L is a D-dimensional matrix with elements being 1.

[0078] r1 and r2 are double-sample learning step control parameters, which are only related to the current iteration number t and the maximum iteration number iter max , and satisfy the following formula:

[0079] wherein, r1 represents a dynamic step parameter of double sample learning; r2 is complementary to r1, guarantees the sum of double sample weights to be 1, realizes normalized allocation, iter max represents the maximum number of iterations; t represents the current number of iterations.

[0080] In this embodiment, in order to verify the performance of the improved algorithm, eight typical functions of the 23 standard test functions of the international general standard global optimization test data set IEEE CEC2005 are selected for performance test of the hybrid strategy improved sparrow search algorithm proposed in this embodiment. Functions F1 to F4 are unimodal functions, used to test local search accuracy; functions F5 and F6 are high-dimensional multimodal functions, used to test global search accuracy; functions F7 and F8 are fixed-dimensional multimodal functions, used to test the effectiveness of single-dimensional search in the later iteration. The specific function expression, domain, optimal value and other information of the test functions are shown in Table 1.

[0081] Table 1 Detailed information of eight test functions selected in IEEE CEC2005

[0082] Further, the hybrid strategy improved sparrow search algorithm (ISSA) proposed in this embodiment is compared with the standard sparrow search algorithm (SSA), the improved sparrow search algorithm (ASSA) and the grey wolf optimization algorithm (GWO) for comparative experiment to verify and analyze the optimization performance. The parameter settings of the four optimization algorithms are shown in Table 2, and the basic settings of all algorithms are kept consistent, the population size n is 30, the maximum number of iterations iter max is 500. Among them, the parameters of the three sparrow search algorithms are set the same: the producer individual proportion PD is 0.2, the sentinel individual proportion SD is 0.2, the sentinel threshold ST is 0.6, and the grey wolf algorithm convergence factor a linearly decreases from 2 to 0 with the number of iterations. The test runs in the same experimental environment, each optimization algorithm is independently run 10 times for each test function, the average value (ave) and the mean square error (std) of the output function value are calculated, and the total time (time) of running 10 times is recorded.

[0083]

[0084] Table 3 is the experimental results of four optimization algorithms running independently on eight standard test functions, from Table 3, it can be seen that the average value and standard deviation of ISSA algorithm running 10 times on each test function are better than other optimization algorithms. For unimodal functions F1 to F4, ISSA algorithm running 10 times can obtain the global optimal solution 0, which shows that compared with other three algorithms, ISSA has higher local search accuracy; for high-dimensional multi-peak functions F5 and F6, three sparrow search algorithms can calculate the optimal solution 0, and the global search accuracy of sparrow search algorithm is obviously higher than that of grey wolf optimization algorithm. Compared with standard SSA algorithm, ASSA algorithm and GWO algorithm, IGWO running time is reduced by 11.07%, 20.61% and 12.93% respectively, and ISSA has faster convergence speed; for fixed-dimensional multi-peak functions F7 and F8, the average value of ISSA is the smallest, which shows that in the later iteration, ISSA is more effective than other three algorithms in single-dimensional search. At the same time, for eight standard test functions, the standard deviation of ISSA calculation 10 times is 0, which shows that ISSA algorithm has higher stability. It can be understood that in order to show completely, a part of data is expressed by scientific notation.

[0085]

[0086] In 500 iterations, for unimodal test functions F1, F3, ASSA and ISSA converge to the same value 0, for high-dimensional multi-peak functions F5, F6, SSA, ASSA and ISSA three sparrow search algorithms also converge to the optimal value 0, and from the running results of Table 3, the convergence speed difference of four algorithms cannot be directly judged. Therefore Figure 9 The convergence curve of four optimization algorithms based on eight standard test functions is shown, Figure 9 including Figure 9 (a)-(h).

[0087] Figure 9 (a)-(d) are the convergence curves of unimodal test functions F1, F2, F3 and F4, it can be seen that ISSA has more ideal optimization accuracy than other optimization algorithms on four test functions, because the population position initialized by ISSA using the best point set theory is closer to the optimal solution of the function. For Figure 9 (c) function F3, in the case of reducing population diversity in the later iteration, ISSA can still further reduce the iteration curve, because the double sample learning strategy and the optimal and worst position updating strategy improve the global optimization ability of sparrow individuals in the later period and enhance the ability to explore the optimal position. Figure 9(e)-(h) are convergence curves of high-dimensional multimodal functions F5, F6 and fixed-dimensional multimodal functions F7, F8, it can be seen that with the increase of dimension, the ISSA algorithm can quickly converge to the optimal position, the algorithm stability is almost not affected, and the optimal solution accuracy calculated in all test functions is higher. In summary, the hybrid strategy improved sparrow search algorithm proposed in this embodiment has better global search ability and better robustness, and the improved algorithm has good optimization performance.

[0088] In one embodiment, program debugging is performed using the Visual Studio Code 2017 compiler, the YOLOv5-M algorithm is based on the PyTorch 2.0.6 deep learning framework, and model training is performed using an Intel(R) Core(TM) i9-12900K CPU @ 3.20 GHz processor, 64 GB of memory, and an NVIDIA GeForce RTX 3090 GPU in a Windows 10 operating system. Twelve hyperparameters are selected by the hybrid strategy improved sparrow search algorithm proposed in this embodiment, and the other hyperparameters use the default hyperparameter settings of the initial pre-trained model. The data set used for model training is a self-built flame target detection data set.

[0089]

[0090] The parameter settings of the improved sparrow search algorithm are shown in Table 4, the population size is set to 20, the maximum number of iterations is 30, the producer individual proportion PD and the sentinel individual proportion SD are both 0.2, and the sentinel threshold ST is 0.6. The hyperparameters and default values of the YOLOv5 model improved by ISSA iterative search are shown in Table 5.

[0091]

[0092] In one embodiment, the accuracy, recall rate, average precision mean, and model processing speed of the flame detection model are used as model evaluation indicators to evaluate the performance of the flame detection model.

[0093] Specifically, the model evaluation indicators include: (1) Accuracy (P): represents the proportion of correct predictions among all predictions of positive samples. That is, the proportion of the number of correctly identified flame targets to the number of all identified flame target categories in the data set, and the calculation formula is as follows:

[0094] Wherein, TP (True Positive) represents the number of correctly identified flames, which is the number of predicted boxes whose Intersection over Union (IoU) with the labeled boxes is greater than the IoU threshold; FP (False Positive) represents the number of incorrectly identified flames, which is the number of predicted boxes whose Intersection over Union with all labeled boxes is less than the IoU threshold.

[0095] (2) Recall rate (R): represents the proportion of the number of all correctly predicted positive samples to the number of actual positive samples. That is, the proportion of the number of correctly identified flame targets to the total number of flames, and the calculation formula is as follows:

[0096] Wherein, FN (False Negative) represents the number of missed identified flames, which is the number of predicted boxes whose Intersection over Union with the labeled boxes is greater than the IoU threshold, but the label of the predicted box is inconsistent with the label of the labeled box; TP represents the number of correctly identified flames.

[0097] (3) Average precision mean (mAP, Mean Average Precision): represents the average value of the average precision (AP, Average Precision) of each class classification of the prediction, and is used to calculate the average precision of multiple classes and evaluate the detection performance of the model. The greater the average precision mean mAP, the higher the precision of the target detection model, and the calculation formula is as follows: ; ; Wherein, the average precision AP represents the accuracy of each class being correctly detected and classified, which is the area of the precision-recall curve; P and R represent the accuracy and recall rate respectively; mAP is the average value of the accuracy AP of each class classification; n represents the number of target classes.

[0098] (4) Model processing speed FPS In addition to the accuracy of the model, the recognition processing speed of the model is also an important indicator for evaluating the detection performance. FPS (Frames Per Second) is the number of image frames that the model can process per second, which is used to evaluate the processing speed of the model on a given hardware system. The greater the FPS value represents the faster the speed of the model in detecting and processing pictures, which can better meet the real-time detection requirements.

[0099] Further, in one embodiment, the corresponding modifications are made based on the ISSA-optimized hyperparameter values in Table 5, and other parameters use the default values of the pre-trained model. After the preparation work is completed, the model training begins, and when the maximum number of iterations 100 is reached, the model training is completed. In order to compare the learning ability of the model during training, the YOLOv5 model is trained using the same data set and default hyperparameter settings, as shown in Figure 10 The average loss value change curve during the training of the two models is shown. From the results of the model training, it can be seen that the improved YOLOv5 model in this embodiment has a faster convergence speed of the loss value curve than the traditional YOLOv5, and the average loss value of the improved YOLOv5 model gradually decreases during the training process. When the number of iterations reaches about 40, the loss value of the model tends to be stable. Compared with the YOLOv5 model at the end of training, the improved YOLOv5 model has a smaller average loss value, indicating that the model learns the features of the flame target more fully, and the improvement measures help the improved YOLOv5 model to achieve good training results.

[0100] In order to comprehensively verify the performance of the improved model, the improved YOLOv5 model proposed in this embodiment is compared with YOLOv5, YOLOv5-TA which only adds a triple attention mechanism structure, YOLOv5-BiFPN which only changes the feature fusion network structure, YOLOv5-TB which adds two improvements but does not optimize the hyperparameters, and the latest YOLOv8. Except for the improved YOLOv5 model of this embodiment which uses the ISSA algorithm to optimize the hyperparameters, the remaining models use default hyperparameters for network training. During the training process of all network models, the hardware environment and other algorithmic settings remain the same, and Table 6 shows the calculation results of the four evaluation indicators.

[0101]

[0102] As can be seen from the indicators in Table 6, compared with the YOLOv5 model, the average accuracy mAP of YOLOv5-TA is improved by 3.6%, indicating that adding a triple attention mechanism module to the backbone network helps improve the detection accuracy of the model; the detection processing speed of the YOLOv5 model is 142.4FPS, and the detection speed is effectively improved to 178.2FPS in the YOLOv5-BiFPN model after the BiFPN improved feature fusion network, indicating that the BiFPN structure improves the model detection speed; the network structure improved model YOLOv5-TB with default hyperparameters improves the mAP value by 0.8% and 1.6% respectively compared with the YOLOv5-TA and YOLOv5-BiFPN models with only a single improvement, indicating that the simultaneous application of the two network structure improvement measures has better detection performance; on the same data set, the mAP value of the improved YOLOv5 model after the ISSA algorithm hyperparameter optimization in the embodiment is improved by 1.2% compared with the latest YOLOv8 model and by 5.8% compared with the traditional YOLOv5, indicating that the improvement measures proposed in the embodiment can effectively improve the model detection accuracy. In terms of model detection speed, the average precision of the improved YOLOv5 model in the embodiment reaches 97.6%, and the detection speed per second is 224.8FPS, which is improved by 57.9% compared with the traditional YOLOv5 model, and the detection accuracy and speed advantage is obvious. The thermal imaging flame detection result of the improved YOLOv5 model in the embodiment is shown in Figure 11 , Figure 11 comprising Figure 11 (a)-(d).

[0103] The embodiment of the application also provides a building construction inspection robot, comprising: a memory configured to store instructions; a processor configured to call the instructions from the memory and capable of realizing the infrared thermal imaging flame detection method for the building construction inspection robot as described in the above embodiment when executing the instructions.

[0104] The embodiment of the application also provides a machine readable storage medium, which stores instructions for causing a machine to execute the above-mentioned infrared thermal imaging flame detection method for the building construction inspection robot.

[0105] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0106] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems), computer program products according to embodiments of the application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0107] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0109] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0110] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM) for storing structural information and / or instruction code. Both can be within one or more memory devices 1225. Alternatively, some memory devices can provide a combination of one or more of the foregoing types of memories. Since memory is a computer readable medium, it can also include a computer program product for execution by the processors 1205. ​​​​​​

[0111] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0112] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0113] The above only is an embodiment of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. An infrared thermal imaging flame detection method applied to a construction inspection robot, characterized in that: include: Obtain infrared thermal imaging images of construction sites; Performing flame detection on the infrared thermal imaging image based on a flame detection model to obtain a flame detection result, wherein the flame detection model is constructed based on an improved YOLOv5 model and is trained after performing hyperparameter optimization on the improved YOLOv5 model based on a hybrid strategy improved sparrow search algorithm, and the improved YOLOv5 model is a model obtained by improving the network structure of the YOLOv5 model, and the improvement of the network structure of the YOLOv5 model includes: adding a triple attention mechanism to the input path of the C3 module of the first and third downsampling in the backbone network, and replacing the network structure of the feature fusion network with BiFPN; The flame detection result is output.

2. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 1, characterized in that: The triple attention mechanism is used to perform parallel processing of three branches on the downsampled feature map, wherein the downsampled feature map is the feature map after the first downsampling and the feature map after the third downsampling in the backbone network, and the three branches include a first branch, a second branch, and a third branch; The first branch is used to rotate the downsampled feature map counterclockwise by 90 degrees in the height direction, generate a first attention weight after dimensionality reduction to 2D by the Z-pool layer, convolution fusion and activation processing, and rotate the first attention weight clockwise by 90 degrees in the height direction to restore it to the same size as the input feature map; The second branch is used to rotate the downsampled feature map counterclockwise by 90 degrees along the width direction, generate a second attention weight after dimensionality reduction to 2D by the Z-pool layer, convolution fusion and activation processing, and rotate the second attention weight clockwise by 90 degrees along the width direction to restore it to the same size as the input feature map; The third branch is used to reduce the dimension of the downsampled feature map to two dimensions through the Z-pool layer, perform convolution fusion and activation processing, and then generate a third attention weight; The triple attention mechanism is also used to aggregate the first attention weight, the second attention weight, and the third attention weight through an average value, and multiply the weight obtained after the average value aggregation by the downsampled feature map element by element to obtain an enhanced feature map.

3. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 2, characterized in that: The BiFPN is used to: Obtain a multi-scale feature map, wherein the multi-scale feature map includes a high-level feature map, a middle-level feature map, and a bottom-level feature map, and the multi-scale feature map is a feature map of the backbone network at different downsampling stages, wherein the middle-level feature map is a feature map enhanced by the triple attention mechanism; After upsampling the high-level feature map through a top-down path, weighted fusion is performed with the middle-level feature map to obtain a first fusion result; After downsampling the bottom-level feature map through a bottom-up path, weighted fusion is performed with the middle-level feature map to obtain a second intermediate result. After downsampling the second intermediate result, weighted fusion is performed with the high-level feature map to obtain a second fusion result, wherein the fusion weight of the bottom-level feature map is higher than the conventional weight, and the conventional weight is the default weight when the high-level feature map and the middle-level feature map are fused; Based on the first fusion result and the second fusion result, an optimized multi-scale feature map is obtained through cross-scale connection integration.

4. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 1, characterized in that: Before the step of performing flame detection on the infrared thermal imaging image based on the flame detection model to obtain a flame detection result, the method further includes: Build an initial model based on the improved YOLOv5 model; Performing hyperparameter optimization on the initial model based on a hybrid strategy-improved sparrow search algorithm, wherein the target hyperparameters for hyperparameter optimization in the initial model include initial learning rate, cyclic learning rate, momentum, weight decay coefficient, number of warm-up learning times, warm-up learning momentum, warm-up initial learning rate, target box loss weight, classification loss weight, confidence loss weight, IoU threshold, and target box threshold; Based on a preset flame infrared thermal imaging data set, the initial model after hyperparameter optimization is trained to obtain the flame detection model.

5. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 4, characterized in that: The method of performing hyperparameter optimization on the initial model based on the hybrid strategy-based improved sparrow search algorithm includes: Obtaining target hyperparameter information of the initial model, wherein the target hyperparameter information includes the number of target hyperparameters, the value range of each target hyperparameter, and a preset initial population size; Based on the target hyperparameter information, a sparrow population is initialized by using the good point set theory to generate a uniformly distributed initial population, wherein each individual in the initial population corresponds to a set of initial candidate values ​​of the target hyperparameter; The target hyperparameters are iteratively optimized using the loss function of the initial model as a fitness evaluation criterion until the number of iterations reaches a maximum number of iterations, thereby obtaining a hyperparameter combination with the highest fitness. The starting point of the iterative optimization is the initial population, and the steps of iteratively optimizing the target hyperparameters include: Update producer positions using nonlinear inertia weights; Update the scavenger position through a two-sample learning strategy; Monitor and adjust the search direction of the population through sentinels.

6. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 5, characterized in that: Initializing the sparrow population based on the target hyperparameter information by using the good point set theory to generate a uniformly distributed initial population includes: Determining the dimension of the multidimensional search space according to the number of target hyperparameters, and selecting a minimum prime number that meets the preset dimensional constraints; Based on the minimum prime number, generating a set of evenly distributed reference points in the multidimensional search space by using the good point set theory; According to the value range of each target hyperparameter, the relative position of each reference point is mapped to a specific target hyperparameter value to generate individuals consistent with the preset initial population size. Each individual contains the initial candidate values ​​of all target hyperparameters, and all individuals are evenly distributed in the multidimensional search space, together constituting an initial population.

7. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 6, characterized in that: The updating of the producer position by using the nonlinear inertia weight comprises: Obtain the current number of iterations, the maximum number of iterations, and the target hyperparameter combination currently corresponding to the producer; determining an inertia weight based on a ratio of the current number of iterations to the maximum number of iterations, based on an iteration progress and a nonlinear relationship, wherein a weight value and a weight reduction rate of the inertia weight in an early iteration are both greater than those in a later iteration; The search step size of the producer is adjusted based on the inertia weight to update the position of the producer.

8. The infrared thermal imaging flame detection method applied to a construction inspection robot according to claim 7, characterized in that: The method of updating the scavenger position by using a two-sample learning strategy includes: The scavenger refers to the first position and the second position in the current population at the same time, wherein the first position is the position of the producer with the highest fitness, and the second position is the position of another randomly selected producer; Adjusting learning weights for the first position and the second position according to a ratio of the current number of iterations to the maximum number of iterations; The position of the scavenger is updated based on the difference between the target hyperparameter value corresponding to the first position and the target hyperparameter value corresponding to the second position, and the adjusted learning weight.

9. A construction inspection robot, characterized in that: include: a memory configured to store instructions; The processor is configured to call the instruction from the memory and implement the infrared thermal imaging flame detection method applied to a construction inspection robot according to any one of claims 1 to 8 when executing the instruction.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores instructions for enabling a machine to execute an infrared thermal imaging flame detection method for a construction inspection robot according to any one of claims 1 to 8.

Images