Target detection method of smart home in smog scene

Abstract

The present application relates to a kind of target detection method of intelligent home in smoke scene, belong to computer vision field.It includes the following steps: constructing the intelligent home dataset containing smoke scene image, constructs the FFA-Net defogging network framework based on CNN and VMamba;Indoor scene target detection network framework based on yolov9 is constructed;Intelligent home target detection model under smoke scene is constructed by combining defogging network framework and target detection network framework, and the indoor smoke scene image and haze-free scene image in intelligent home dataset are obtained by the target detection result of intelligent home target detection model under smoke scene.The present application method carries out target detection to indoor home articles with fog and without fog, provides more efficient, more accurate target detection capability for the intelligent home system under smoke scene, improves the security of intelligent home, increases the convenience and comfort of home environment, brings more intelligent, comfortable home experience for user.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a target detection method for smart homes in a smoke scene. Background Technology

[0002] With the rapid advancement of smart technology, smart homes have evolved into an indispensable element of modern life, serving as a new engine for cultivating new productivity. By interconnecting numerous smart devices and utilizing the internet for remote control and automated management, they significantly enhance the convenience, comfort, and security of daily life. In the many functions and applications of smart homes, such as Haier U+, object detection, as a core research direction in computer vision, plays a crucial role. However, under specific environmental conditions, such as in smoky scenes, object detection faces significant challenges. The obstruction and interference of smoke pose a severe test to the accuracy and efficiency of traditional object detection algorithms. Therefore, researching the problem of smart home object detection in smoky scenes and developing corresponding solutions has significant practical significance and application value. This will not only promote the further development of smart home technology but also provide users with more reliable and efficient security, further enhancing the practicality and intelligence level of smart home systems.

[0003] With the continuous advancement of image processing and artificial intelligence technologies, researchers have developed various object detection methods and algorithms for smoke scenes. These methods utilize machine learning and deep learning techniques to improve the accuracy and robustness of object detection. These innovative methods include deep learning-based object detection algorithms, multi-sensor fusion object detection technologies, and image enhancement and noise reduction techniques specifically designed for smoke characteristics. In the field of smart homes, object detection technology in smoke scenes has broad application prospects. For example, in fire alarm systems, object detection technology can monitor fire sources in smoke in real time and quickly trigger alarms, significantly improving the speed of fire detection and emergency response. Furthermore, this technology plays an important role in intelligent security systems, helping monitoring personnel to promptly identify suspicious behavior and enhance security capabilities. Nevertheless, current object detection algorithms still face some challenges in the application of smart home systems. Changes in lighting, diverse viewing angles, and complex backgrounds in the home environment place higher demands on the robustness of detection algorithms. To reduce the probability of false positives and false negatives, detection algorithms need to be able to adapt to different environmental conditions, exhibiting stronger adaptability and generalization capabilities. These challenges motivate researchers to continuously explore and improve technologies to promote the maturity and application of smart home object detection technology.

[0004] There are several problems that need to be solved in target detection methods in smoke scenes: 1) Traditional target detection algorithms are not accurate or efficient in dealing with smoke obscuration and interference. 2) Changes in lighting, diverse viewing angles, and complex backgrounds in home environments place higher demands on the robustness of detection algorithms; 3) The algorithm has high complexity and poor real-time performance. Summary of the Invention

[0005] The purpose of this invention is to provide a target detection method for smart homes in smoky environments, in order to solve the problem of difficult home detection in smoky environments.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a target detection method for smart homes in a smoke-filled environment, comprising the following steps: S1. Construct a smart home dataset containing smoke; S2. Construct an FFA-Net dehazing network framework based on CNN and VMamba. The FFA-Net dehazing network framework based on CNN and VMamba includes a preprocessing network and a dehazing network. The FFA-Net dehazing network framework is used to dehaze the smoke scene images in the dataset. S3. Construct an indoor scene target detection network framework based on YOLOv9, including an auxiliary reversible branch structure, a multi-level auxiliary information structure, a backbone structure, and a detection head, for target detection in dehazed and hazy images; S4. A smart home target detection model in a smoke scene is constructed by combining the FFA-Net dehazing network framework based on CNN and VMamba and the indoor scene target detection network framework based on YOLOv9. The indoor smoke scene images and fog-free scene images in the smart home dataset are processed by the smart home target detection model in a smoke scene to obtain the target detection results.

[0007] Furthermore, the smart home dataset mentioned in step S1 includes the ODSR-IHS dataset and the Haze4K dataset. The ODSR-IHS dataset is a fog-free image dataset, and the Haze4K dataset is a foggy image dataset. The fog-free image dataset serves as a comparison to ensure accurate recognition even in fog-free conditions. The fog-free images are also defogging processed in the smart home target detection model in smoky scenes.

[0008] Further, step S2 specifically includes: The preprocessing network comprises two sub-networks: a color remapping network and a normalization network. The original images in the dataset are converted from the RGB color space to the HSV color space via the color remapping network to obtain a mapped image XHSV. The mapped image is then passed through the normalization network to output a standardized image X with a mean of zero and a variance of one. The calculation process of the color remapping network is as follows: First, convert the RGB values ​​to a range of 0 to 1. , Where R represents the red component of the RGB value, G represents the green component of the RGB value, B represents the blue component of the RGB value, r represents the red component of the RGB value converted to a value between 0 and 1, g represents the green component of the RGB value converted to a value between 0 and 1, and b represents the blue component of the RGB value converted to a value between 0 and 1; calculate the V, S, and H values ​​respectively. , , , Where V represents the lightness value in the HSV color space, S represents the saturation value in the HSV color space, and H represents the hue value in the HSV color space. The expression indicates a multiplication operation; max represents the maximum value and min represents the minimum value. The normalized network calculation process is as follows: , in, X is the original data, and X is the normalized data. This represents the smallest data point in the original dataset. This represents the largest data point in the original dataset.

[0009] The dehazing network adopts the structure of Convolutional Neural Network (CNN) and Visual State Space Model (VMamba), including convolutional layers, pooling layers, fully connected layers, and Visual State Space Block (VSS Block). The input image X is processed by a 3×3 convolution to obtain feature map X1. X1 is processed by three Block Structures to obtain feature maps X2, X3, and X4 respectively. The three output results are combined by channels to obtain feature map X5, which is then processed by the VSS Block to obtain feature map X6. Finally, the result of X6 is processed by two 3×3 convolutions and added to the corresponding positions of the input image X to obtain the output result Xout. The Visual State Space Block (VSS Block) comprises two branches. The first branch includes a linear layer, a deep convolutional layer, a SiLU activation function, a 2D-SSM layer, and a layer normalization layer (LayerNorm). The second branch includes a linear layer and a SiLU activation function. X5 is obtained by passing X5 through the first branch. 1 X5 is obtained through the second branch. 2 For the X5 1 and X5 2 Perform a Hadamard product to obtain the Hadamard product result. Then, project the number of channels of the Hadamard product result onto the same number of channels as the input image X to obtain an output X with the same shape as the input image X. out The specific calculation process is as follows: , , , in, Presentation layer normalization operation, This indicates the operation of the 2D-SSM layer. express Activation function This represents a depthwise convolution operation. ⊙ represents a linear layer operation, and ⊙ represents a Hadamard product operation.

[0010] Furthermore, step S3 specifically includes: The YOLOv9-based indoor scene target detection network framework employs the existing YOLOv9 target detection algorithm to achieve real-time detection of dehazed and fog-free images.

[0011] Secondly, the present invention provides a target detection system for smart homes in a smoke-filled environment, comprising a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to implement the target detection method for smart homes in a smoke-filled environment of the present invention.

[0012] Thirdly, the present invention provides a computer medium storing a computer program that can be executed to implement the target detection method for smart homes in smoke scenarios according to the present invention.

[0013] The advantages of this invention are: Based on the YOLOv9 network architecture, this paper proposes a target detection method specifically for smart homes in smoky environments. By improving the attention mechanism of defogging methods, this invention proposes a smart home target detection method in smoky scenes. This method combines CNN and VMamba network architectures, removes the attention mechanism of traditional methods, reduces the number of parameters, and significantly improves real-time performance. Target detection experiments were conducted on the ODSR-IHS and Haze4K datasets in both foggy and fog-free indoor home environments, addressing the challenge of home detection in smoky scenes. Experimental results show that the proposed method achieves good detection results, providing more efficient and accurate target detection capabilities for smart home systems in smoky environments. This not only improves the security of smart homes but also increases the convenience and comfort of the home environment, bringing users a more intelligent and comfortable home experience. Attached Figure Description

[0014] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0015] Figure 1 This is a flowchart of the target detection method for smart homes in a smoke scene according to the present invention; Figure 2 This is a diagram of the overall YOLOv9 network framework of the present invention; Figure 3 This is a graph showing the experimental results of the ODSR-IHS dataset from this invention; Figure 4 The images shown are experimental results of the Haze4k dataset of this invention, where (a) is a foggy image and (b) is a defogging image. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1 In this embodiment, as Figure 1 As shown, a target detection method for smart homes in a smoke scene is provided, the specific steps of which include: S1. Construct a smart home dataset containing smoke; S2. Construct an FFA-Net dehazing network framework based on CNN and VMamba. The FFA-Net dehazing network framework based on CNN and VMamba includes a preprocessing network and a dehazing network. The FFA-Net dehazing network framework is used to dehaze the smoke scene images in the dataset. S3. For example Figure 2 As shown, an indoor scene target detection network framework based on YOLOv9 is constructed, including an auxiliary reversible branch structure, a multi-level auxiliary information structure, a backbone structure, and a detection head, which is used to perform target detection on dehazed and fog-free images. S4. A smart home target detection model in a smoke scene is constructed by combining the FFA-Net dehazing network framework based on CNN and VMamba and the indoor scene target detection network framework based on YOLOv9. The indoor smoke scene images and fog-free scene images in the smart home dataset are processed by the smart home target detection model in a smoke scene to obtain the target detection results.

[0018] Specifically, the smart home dataset includes the ODSR-IHS dataset and the Haze4K dataset. The ODSR-IHS dataset is a fog-free image dataset, and the Haze4K dataset is a foggy image dataset. The fog-free image dataset serves as a comparison to ensure accurate recognition even in fog-free conditions. The fog-free images are also defogging processed in the smart home target detection model in smoky scenes.

[0019] Specifically, the preprocessing network includes two sub-networks: a color remapping network and a normalization network. The original images in the dataset are converted from the RGB color space to the HSV color space by the color remapping network to obtain the mapped image XHSV. The mapped image is then passed through the normalization network to output a standardized image X with a mean of zero and a variance of one. The calculation process of the color remapping network is as follows: First, convert the RGB values ​​to a range of 0 to 1. , Where R represents the red component of the RGB value, G represents the green component of the RGB value, B represents the blue component of the RGB value, r represents the red component of the RGB value converted to a value between 0 and 1, g represents the green component of the RGB value converted to a value between 0 and 1, and b represents the blue component of the RGB value converted to a value between 0 and 1; calculate the V, S, and H values ​​respectively. , , , Where V represents the lightness value in the HSV color space, S represents the saturation value in the HSV color space, and H represents the hue value in the HSV color space. The expression indicates a multiplication operation; max represents the maximum value and min represents the minimum value. The normalized network calculation process is as follows: in, X is the original data, and X is the normalized data. This represents the smallest data point in the original dataset. This represents the largest data point in the original dataset.

[0020] Specifically, the dehazing network adopts the structure of Convolutional Neural Network (CNN) and Visual State Space Model (VMamba), including convolutional layers, pooling layers, fully connected layers, and Visual State Space Block (VSS Block). The input image X is processed by a 3×3 convolution to obtain feature map X1. X1 is processed by three Block Structures to obtain feature maps X2, X3, and X4 respectively. The three output results are combined by channels to obtain feature map X5, which is then processed by the VSS Block to obtain feature map X6. Finally, the result of X6 after two 3×3 convolutions is added to the input image X at corresponding positions to obtain the output result Xout. The Visual State Space Block (VSS Block) comprises two branches. The first branch includes a linear layer, a deep convolutional layer, a SiLU activation function, a 2D-SSM layer, and a layer normalization layer (LayerNorm). The second branch includes a linear layer and a SiLU activation function. X5 is obtained by passing X5 through the first branch. 1 X5 is obtained through the second branch. 2 For the X5 1 and X5 2 Perform a Hadamard product to obtain the Hadamard product result. Then, project the number of channels of the Hadamard product result onto the same number of channels as the input image X to obtain an output X with the same shape as the input image X. out The specific calculation process is as follows: , , , in, Presentation layer normalization operation, This indicates the operation of the 2D-SSM layer. express Activation function This represents a depthwise convolution operation. ⊙ represents a linear layer operation, and ⊙ represents a Hadamard product operation.

[0021] Specifically, the YOLOv9-based indoor scene target detection network framework uses the existing YOLOv9 target detection algorithm to achieve real-time detection of dehazed and fog-free images; The backbone structure includes a silence layer, a CBS convolutional layer, and a RepNCSPELAN4 layer. The primary function of the silence layer is to provide necessary input data for the auxiliary invertible branch. Furthermore, the 2nd, 3rd, and 4th RepNCSPELAN4 layers in the backbone are used to extract auxiliary information. This information provides a mapping between the data and the target during training, allowing the loss function to more effectively guide network training and helping to avoid establishing spurious associations between input and target data during training. The multi-level auxiliary information structure is designed to combine the return gradients from different prediction heads and aggregate the gradient information of all target objects, passing it to the main branch to update its parameters. This gradient information is crucial for driving parameter learning, helping the network extract correct and important information. It is worth noting that this auxiliary structure is only used during the training phase. During training, the network uses the three outputs of the detection head and the three auxiliary outputs of the auxiliary invertible branch to make predictions with the ground truth labels. During the inference phase, to reduce network complexity, the multi-level auxiliary information structure and the auxiliary invertible branch are not used. The detection head's role is to output the final detection result. This structural design does not require the main branch to retain all the original information. Instead, it generates useful gradients through an auxiliary supervision mechanism to update the original information. This design helps to alleviate the information loss problem commonly encountered during convolution.

[0022] GELAN is a generalized and efficient aggregation structure. The RepNCSPELAN 4-layer architecture combines two neural network architectures, CSPNet and ELAN, which employ gradient path planning, achieving a balance between lightweight design, inference speed, and accuracy. The GELAN layer extends the capabilities of ELAN, which initially only used stacked convolutional layers, to a new architecture that can utilize any computational block. GELAN's overall design integrates the cross-level partial connectivity of CSPNet and the efficient layer aggregation of ELAN to achieve effective gradient propagation and feature aggregation.

[0023] Example 2 In this embodiment, an indoor dataset containing smoke is collected and constructed; the dataset includes the ODSR-IHS dataset and the Haze4K dataset, in order to train and test the object detection model. The proposed method can effectively detect various objects in the scene.

[0024] To evaluate the model's performance, the experiment used four key metrics to help quantify its quality: precision, recall, F1 score, and average detection accuracy (mAP). The specific calculation process is as follows: , Where Precision refers to the number of samples predicted as positive that are actually positive samples, TP represents the number of positive samples predicted as positive, and FP represents the number of negative samples predicted as positive. , Recall, also known as recall rate, is a metric for evaluating the performance of a classification model. It is mainly used to measure the proportion of positive instances correctly identified by the model out of all actual positive instances. FN represents the number of positive samples predicted as negative examples. , F1 is the harmonic average of precision and recall; , mAP is a commonly used evaluation metric in object detection models. mAP 50 refers to the confidence level (IOU) when the ground truth bounding box and the predicted bounding box have no intersection at all (IOU = 0); when the confidence level (IOU) is 0.5, it means that the ground truth bounding box and the predicted bounding box have half of their intersection; and when the confidence level (IOU) is 1, it means that the predicted bounding box and the ground truth bounding box completely overlap. mAP 50-95 refers to the average of mAP at every 5 confidence levels from 50 to 95, thus obtaining a more comprehensive evaluation.

[0025] Table 1 In this embodiment, to verify the effectiveness of the target detection method for smart homes in smoky scenarios, indoor target prediction was performed on both the ODSR-IHS and ODSR-IHS datasets. Table 1 shows the prediction results of the method of this invention on the constructed datasets. As can be seen from Table 1, the prediction accuracy for toilets and socks is the highest, achieving a complete prediction accuracy. When the confidence level is 0.5, the mAP value can reach an accuracy of over 85% for all items except electrical wires. When the confidence level is 50-95, the accuracy for beds, toilets, and socks is the highest, reaching around 0.9.

[0026] In this embodiment, Figure 3 The figure shows the experimental results of the ODSR-IHS dataset of the present invention. As can be seen from the figure, the method of the present invention can accurately detect targets in fog-free images of smart homes and has a high accuracy. Figure 4The experimental results of the Haze4k dataset of the present invention are shown in the figure. (a) is a foggy image and (b) is the defogging image corresponding to (a). It can be seen that the method of the present invention has a good defogging effect and can accurately detect targets in foggy and fog-free images.

[0027] Example 3 In this embodiment, a target detection system for smart homes in a smoky environment is provided, including a processor and a memory. The memory stores a computer program, which can be executed by the processor to implement the target detection method for smart homes in a smoky environment of the present invention.

[0028] Example 4 In this embodiment, a computer medium is provided, which stores a computer program that can be executed to implement the target detection method for smart homes in smoke scenarios according to the present invention.

[0029] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A target detection method for smart homes in a smoky environment, characterized in that, Includes the following steps: S1. Construct a smart home dataset containing images of smoke scenes; S2. Construct an FFA-Net dehazing network framework based on CNN and VMamba. The FFA-Net dehazing network framework based on CNN and VMamba includes a preprocessing network and a dehazing network. The FFA-Net dehazing network framework is used to dehaze the smoke scene images in the dataset. The dehazing network adopts the structure of Convolutional Neural Network (CNN) and Visual State Space Model (VMamba), including convolutional layers, pooling layers, fully connected layers, and Visual State Space Block (VSS Block). The input image X is processed by a 3×3 convolution to obtain feature map X1. X1 is processed by three Block Structures to obtain feature maps X2, X3, and X4 respectively. The three output results are combined by channels to obtain feature map X5, which is then processed by the VSS Block to obtain feature map X6. Finally, the result of X6 after two 3×3 convolutions is added to the corresponding positions of the input image X to obtain the output result Xout. The Visual State Space Block (VSS Block) comprises two branches. The first branch includes a linear layer, a deep convolutional layer, a SiLU activation function, a 2D-SSM layer, and a layer normalization layer (LayerNorm). The second branch includes a linear layer and a SiLU activation function. X5 is obtained by passing X5 through the first branch. 1 X5 is obtained through the second branch. 2 For the X5 1 and X5 2 Perform a Hadamard product to obtain the Hadamard product result. Then, project the number of channels of the Hadamard product result onto the same number of channels as the input image X to obtain an output X with the same shape as the input image X. out The specific calculation process is as follows: ， ， ， in, Presentation layer normalization operation, This indicates the operation of the 2D-SSM layer. express Activation function This represents a depthwise convolution operation. ⊙ represents a linear layer operation, and ⊙ represents a Hadamard product operation. S3. Construct an indoor scene target detection network framework based on YOLOv9, including an auxiliary reversible branch structure, a multi-level auxiliary information structure, a backbone structure, and a detection head, for target detection in dehazed and hazy images; S4. A smart home target detection model in a smoke scene is constructed by combining the FFA-Net dehazing network framework based on CNN and VMamba and the indoor scene target detection network framework based on YOLOv9. The indoor smoke scene images and fog-free scene images in the smart home dataset are processed by the smart home target detection model in a smoke scene to obtain the target detection results.

2. The target detection method for smart homes in a smoke scene according to claim 1, characterized in that, The smart home dataset mentioned in step S1 includes the ODSR-IHS dataset and the Haze4K dataset. The ODSR-IHS dataset is a fog-free image dataset, and the Haze4K dataset is a foggy image dataset. The fog-free image dataset serves as a comparison to ensure accurate recognition even in fog-free conditions. The fog-free images are also defogging processed in the smart home target detection model in smoky scenes.

3. The target detection method for smart homes in a smoke scene according to claim 1, characterized in that, In S2, the preprocessing network includes two sub-networks: a color remapping network and a normalization network. The original images in the dataset are converted from the RGB color space to the HSV color space by the color remapping network to obtain the mapped image XHSV. The mapped image is then passed through the normalization network to output a standardized image X with a mean of zero and a variance of one. The calculation process of the color remapping network is as follows: Convert RGB values ​​to a range of 0 to 1. ， Where R represents the red component of the RGB value, G represents the green component of the RGB value, B represents the blue component of the RGB value, r represents the red component of the RGB value converted to a value between 0 and 1, g represents the green component of the RGB value converted to a value between 0 and 1, and b represents the blue component of the RGB value converted to a value between 0 and 1; calculate the V, S, and H values ​​respectively: ， ， ， Where V represents the lightness value in the HSV color space, S represents the saturation value in the HSV color space, and H represents the hue value in the HSV color space. The expression represents a multiplication operation, where `max` represents taking the maximum value and `min` represents taking the minimum value. The normalized network calculation process is as follows: ， in, X is the original data, and X is the normalized data. This represents the smallest data point in the original dataset. This represents the largest data point in the original dataset.

4. The target detection method for smart homes in a smoke scene according to claim 1, characterized in that, S3 specifically includes: The YOLOv9-based indoor scene target detection network framework employs the existing YOLOv9 target detection algorithm to achieve real-time detection of dehazed and fog-free images.

5. A target detection system for smart homes in a smoke-filled environment, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to implement the method as described in any one of claims 1-4.

6. A computer medium, characterized in that, The device contains a computer program that can be executed to implement the method as described in any one of claims 1-4.

Images