A hotel set door detection method based on YOLOv5s neural network
By adding coordinate attention mechanism and bidirectional feature pyramid structure to the YOLOv5s neural network, the model is optimized to adapt to edge-end equipment deployment, and the problems of strong computing power and difficult deployment in suite gate detection are solved, achieving efficient and accurate detection results.
Patent Information
- Application Number
- CN202210216832.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-07
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-03-07
AI Technical Summary
The existing deep learning-based object detection algorithm is difficult to apply to intelligent detection of set doors, and the trained models are difficult to deploy and use on site for building decoration, with strong computing power and difficult to deploy.
A hotel suit door detection method based on YOLOv5s neural network was designed. By adding coordinate attention mechanism, drawing on bidirectional feature pyramid structure and fine-tuning neck network, combining transfer learning, the model is optimized to adapt to edge device deployment, and through LabelImg and PyTorch implements data annotation and model training.
It improves detection accuracy and accuracy of border position, reduces the amount of calculation, and realizes efficient detection on Huawei Atlas200DK edge device, which can accurately identify different defects of hotel suit doors.
Smart Images

Figure CN114663366B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning, and in particular relates to a hotel set door detection method based on a YOLOv5s neural network. Background Art
[0002] With the disappearance of the demographic dividend, the development of the building decoration industry faces a difficult situation of hiring, and the age of decorators and decorators continues to rise. Nowadays, information technology has been rapidly developed, and the Internet of Things, big data and artificial intelligence have been gradually applied to all aspects of modern people's lives. The traditional building decoration and decoration industry needs to take advantage of the transformation and upgrading of information technology, and shift from manual work to industrialization and information development. In order to accelerate the industrial upgrading of the building decoration industry, it is necessary to vigorously promote the establishment of a modular and information-based prefabricated decoration component production system, and to improve the intelligent detection system of integrated building components and promote more standardized building components. Among them, the decoration quality of the set door as the entrance to a room will directly affect the consumer's subsequent consumption choices. Therefore, the intelligent detection task of hotel prefabricated decoration components represented by the set door has been increasingly valued, and the hotel decoration industry urgently needs to carry out research on defect detection of the decoration of the set door components.
[0003] At present, there are no intelligent measurement and quality inspection algorithms related to prefabricated set door decoration at home and abroad. The decoration site is still measured and inspected manually. The present invention attempts to introduce relevant algorithms of deep learning. Specifically, a target detection algorithm based on deep learning is used for set door components. However, most of the existing target detection algorithms based on deep learning are aimed at a variety of general targets or some special targets, and none of them can be directly applied to the special target of set doors. Moreover, most of these methods require equipment with powerful computing power for training and reasoning, and the optimal target detection model after training is difficult to deploy and use at the construction site of hotel prefabricated decoration. Therefore, how to build an intelligent detection model for prefabricated set door decoration that can accurately detect different defects of set doors, can be successfully trained, and can be deployed on edge devices is a problem that researchers in this field are very concerned about. Summary of the invention
[0004] Purpose of the invention: To solve the problems of special targets with different defects in the decoration of assembled door sets, the existing deep learning-based target detection algorithm requires strong computing power, and the trained optimal target detection model is not easy to deploy and use.
[0005] Technical solution: The present invention designs a hotel set door detection method based on YOLOv5s neural network. The effectiveness of the present invention lies in: collecting the set door images installed in the hotel; marking the data through the LabelImg target detection and annotation tool to generate a special target data set for different defects in the set door decoration; adding the coordinate attention mechanism Coordinate Attention to the YOLOv5s backbone network to enhance the network model's ability to extract spatial information and improve the detection accuracy; drawing on the bidirectional feature pyramid, fine-tuning the feature pyramid network FPN plus path aggregation network PAN structure in the neck network, thereby extracting multi-scale features to improve the accuracy of the detection category and the accuracy of the detection frame position and size; changing the three-branch detection head used for multi-scale feature prediction in the head network into two branches, removing the branch with the finest grid division, and reducing calculations and accelerating network convergence while meeting the needs for medium and large-scale targets; using transfer learning to reduce the amount of calculations required for model training; deploying the optimal model on Huawei Atlas200DK edge devices.
[0006] Technical solution: The present invention proposes a hotel suite door detection method based on the YOLOv5s neural network, and the method comprises the following steps:
[0007] Step 1: Collect the images of the door sets installed in the hotel, and use ACDSee to batch resize the images to 640×640;
[0008] Step 2: Label the data using the LabelImg target detection and annotation tool;
[0009] Step 3: Use the PyTorch deep learning framework to build a basic YOLOv5s neural network model and modify the model structure;
[0010] Step 4: Use the dataset to train and test the target detection model, and deploy the trained neural network model to the Huawei Atlas200DK edge device.
[0011] Further, the step 1 of collecting the images of the door sets installed in the hotel and using ACDSee to batch adjust the image size to 640×640 includes the following steps:
[0012] Step 1: Use the Huawei Atlas200DK device equipped with a camera to capture images of the door sets installed in the hotel. Each image contains a complete door frame structure.
[0013] Step 2: Divide the captured images into 8 categories based on the relationship between the two pairs of sides and the four corners of the quadrilateral:
[0014] The upper and lower frames are parallel, the left and right frames are parallel; the adjacent frames are vertical or the adjacent frames are not vertical;
[0015] The upper and lower borders are parallel, and the left and right borders are not parallel: only one of the left and right borders is perpendicular to the upper and lower borders, or the adjacent borders are not perpendicular;
[0016] The upper and lower borders are not parallel, and the left and right borders are parallel: only one of the upper and lower borders is perpendicular to the left and right borders, or the adjacent borders are not perpendicular;
[0017] The left and right sides are not parallel, and the upper and lower borders are not parallel: only one pair of adjacent borders is vertical or both adjacent borders are not vertical. In this order from 0 to 7, add the number corresponding to the category to the beginning of each image file name;
[0018] Step 3: Use ACDSee to open the folder containing the renamed images, then right-click and select "Batch Edit" under the "Batch" option, click Resize, and adjust the width and height to 640×640.
[0019] Furthermore, the step 2 of labeling the data using the LabelImg target detection and labeling tool includes the following steps:
[0020] Step 1: Install and run the LabelImg label detection and annotation tool and open the LabelImg tool interface;
[0021] Step 2: Set the input and output folders, click "Open Dir" to select the folder where the 640×640 image is stored, and then click "Change Save Dir" to select the save path;
[0022] Step 3: Set the marking format. First, check "Verify Image" to set the next image to have the same marking as the previous image. Then check the YOLO marking format. Finally, check the default save button.
[0023] Step 4: Label the image according to the number in front of the file name and frame the suit door in the image.
[0024] Furthermore, using the PyTorch deep learning framework as described in step 3, building a basic YOLOv5s neural network model and modifying the model structure include the following steps:
[0025] Step 1: Use the PyTorch framework to build a basic YOLOv5s neural network;
[0026] Step 2: Modify the backbone network of the YOLOv5s neural network and embed the coordinate attention mechanism in the cross-stage local structure of the backbone network. The coordinate attention mechanism converts the original feature x output by the channel attention into c(i,j) is decomposed into two one-dimensional transposed feature maps and
[0027]
[0028]
[0029] Among them, c, h, w represent the variable channel, variable height and variable width of the feature, H and W represent the maximum height and width of the feature, i and j represent the order of accumulating features in width and height, and the two one-dimensional feature codes are aggregated along two spatial directions respectively. Feature f:
[0030]
[0031] Among them, δ, Conv and Concat represent nonlinear activation function, convolution and feature concatenation respectively. Aggregate features are decomposed along a spatial direction to capture long-range dependencies. Preserve precise position information along another spatial direction
[0032]
[0033]
[0034] Among them, f h and f w They represent the feature decomposition of the aggregate feature f along the height and width directions, σ represents the Sigmoid activation function, and finally the features of these two directions are expanded and weighted with the original input features to obtain the coordinate attention feature y c (i,j):
[0035]
[0036] Among them, the expanded weighted sum is simplified and presented in the form of product;
[0037] Step 3: Modify the neck network of the original model. Refer to the bidirectional feature pyramid structure to establish a bidirectional connection in the fourth layer of the feature pyramid plus path aggregation structure. The intermediate output and final output of the second layer are expressed as and
[0038]
[0039]
[0040] Among them, Conv and Concat represent convolution and feature concatenation respectively. and They represent the inputs of the third, fourth, and fifth layers of the bidirectional feature pyramid in the neck network respectively. Resize means adjusting the parameter feature size to be consistent with the first parameter of Concat.
[0041] Step 4: Modify the head network of the original model, change the three-branch detection head used for multi-scale feature prediction in the head network into two branches, remove the branch with the smallest grid output feature size of 80×80×255, and retain the output feature sizes of the two branches of 40×40×255 and 20×20×255 respectively.
[0042] Furthermore, in step 4, the target detection model is trained and tested using the data set, and the trained neural network model is deployed on the Huawei Atlas200DK device, including the following steps:
[0043] Step 1: Before training, set the Adam optimizer to train the YOLOv5s model, with an initial learning rate of 0.001, a momentum of 0.937, and a weight decay of 0.0005;
[0044] Step 2: During training, use the labeled data in step 2 as the input of the YOLOv5s neural network model built in step 3, and use the labeled categories as the output of the neural network model to train the neural network model for a preset number of iterative trainings;
[0045] Step 3: After training, save the trained neural network model and deploy the trained neural network model to the Huawei Atlas200DK device.
[0046] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:
[0047] (1) Coordinate Attention is added to the YOLOv5s backbone network. This lightweight design can greatly improve the network model's ability to extract spatial information by adding a small amount of simple calculations.
[0048] (2) Drawing on the top-down and bottom-up dual weighted feature pyramid network BiFPN proposed in the EfficientDet target detection network, the feature pyramid network FPN plus path aggregation network PAN structure in the neck network is fine-tuned. The fine-tuned structure can efficiently fuse language information and location information, thereby extracting multi-scale features to improve the accuracy of detection categories and the accuracy of detection bounding box position and size.
[0049] (3) The three-branch detection head used for multi-scale feature prediction in the head network is changed into two branches. Because in the hotel set door data, all the targets to be detected occupy a large area in the entire image, there is no need to detect smaller-scale targets, and the branch with the finest grid division is removed, which can reduce calculations and speed up network convergence while meeting the needs. In short, the present invention has better performance than YOLOv5s in hotel set door detection, and can efficiently and accurately detect different types of defective or qualified hotel set doors. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Attached Figure 1 It is a flow chart of a hotel set door detection method based on YOLOv5s neural network described in the present invention. DETAILED DESCRIPTION
[0051] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Figure 1 As shown, a hotel set door detection method based on YOLOv5s neural network, the method comprises the following steps:
[0052] Step 1: Collect the images of the door sets installed in the hotel, and use ACDSee to batch resize the images to 640×640;
[0053] Step 2: Label the data using the LabelImg target detection and annotation tool;
[0054] Step 3: Use the PyTorch deep learning framework to build a basic YOLOv5s neural network model and modify the model structure;
[0055] Step 4: Use the dataset to train and test the target detection model, and deploy the trained neural network model to the Huawei Atlas200DK edge device.
[0056] Further, the step 1 of collecting the images of the door sets installed in the hotel and using ACDSee to batch adjust the image size to 640×640 includes the following steps:
[0057] Step 1: Use the Huawei Atlas200DK device equipped with a camera to capture images of the door sets installed in the hotel. Each image contains a complete door frame structure.
[0058] Step 2: Divide the captured images into 8 categories based on the relationship between the two pairs of sides and the four corners of the quadrilateral:
[0059] The upper and lower frames are parallel, the left and right frames are parallel; the adjacent frames are vertical or the adjacent frames are not vertical;
[0060] The upper and lower borders are parallel, and the left and right borders are not parallel: only one of the left and right borders is perpendicular to the upper and lower borders, or the adjacent borders are not perpendicular;
[0061] The upper and lower borders are not parallel, and the left and right borders are parallel: only one of the upper and lower borders is perpendicular to the left and right borders, or the adjacent borders are not perpendicular;
[0062] The left and right sides are not parallel, and the upper and lower borders are not parallel: only one pair of adjacent borders is vertical or both adjacent borders are not vertical. In this order from 0 to 7, add the number corresponding to the category to the beginning of each image file name;
[0063] Step 3: Use ACDSee to open the folder containing the renamed images, then right-click and select "Batch Edit" under the "Batch" option, click Resize, and adjust the width and height to 640×640.
[0064] Furthermore, the step 2 of labeling the data using the LabelImg target detection and labeling tool includes the following steps:
[0065] Step 1: Install and run the LabelImg label detection and annotation tool and open the LabelImg tool interface;
[0066] Step 2: Set the input and output folders, click "Open Dir" to select the folder where the 640×640 image is stored, and then click "Change Save Dir" to select the save path;
[0067] Step 3: Set the marking format. First, check "Verify Image" to set the next image to have the same marking as the previous image. Then check the YOLO marking format. Finally, check the default save button.
[0068] Step 4: Label the image according to the number in front of the file name and frame the suit door in the image.
[0069] Furthermore, using the PyTorch deep learning framework as described in step 3, building a basic YOLOv5s neural network model and modifying the model structure include the following steps:
[0070] Step 1: Use the PyTorch framework to build a basic YOLOv5s neural network;
[0071] Step 2: Modify the backbone network of the YOLOv5s neural network and embed the coordinate attention mechanism in the cross-stage local structure of the backbone network. The coordinate attention mechanism converts the original feature x output by the channel attention into c (i,j) is decomposed into two one-dimensional transposed feature maps and
[0072]
[0073]
[0074] Among them, c, h, w represent the variable channel, variable height and variable width of the feature, H and W represent the maximum height and width of the feature, i and j represent the order of accumulating features in width and height, and the two one-dimensional feature codes are aggregated along two spatial directions respectively. Feature f:
[0075]
[0076] Among them, δ, Conv and Concat represent nonlinear activation function, convolution and feature concatenation respectively. Aggregate features are decomposed along a spatial direction to capture long-range dependencies. Preserve precise position information along another spatial direction
[0077]
[0078]
[0079] Among them, f h and f w They represent the feature decomposition of the aggregate feature f along the height and width directions, σ represents the Sigmoid activation function, and finally the features of these two directions are expanded and weighted with the original input features to obtain the coordinate attention feature y c (i,j):
[0080]
[0081] Among them, the expanded weighted sum is simplified and presented in the form of product;
[0082] Step 3: Modify the neck network of the original model. Refer to the bidirectional feature pyramid structure to establish a bidirectional connection in the fourth layer of the feature pyramid plus path aggregation structure. The intermediate output and final output of the second layer are expressed as and
[0083]
[0084]
[0085] Among them, Conv and Concat represent convolution and feature concatenation respectively. and They represent the inputs of the third, fourth, and fifth layers of the bidirectional feature pyramid in the neck network respectively. Resize means adjusting the parameter feature size to be consistent with the first parameter of Concat.
[0086] Step 4: Modify the head network of the original model, change the three-branch detection head used for multi-scale feature prediction in the head network into two branches, remove the branch with the smallest grid output feature size of 80×80×255, and retain the output feature sizes of the two branches of 40×40×255 and 20×20×255 respectively.
[0087] Furthermore, the target detection model is trained and tested using the dataset, and the trained neural network model is deployed on the Huawei Atlas200DK device, including the following steps:
[0088] Step 1: Before training, set the Adam optimizer to train the YOLOv5s model, with an initial learning rate of 0.001, a momentum of 0.937, and a weight decay of 0.0005;
[0089] Step 2: During training, use the labeled data in step 2 as the input of the YOLOv5s neural network model built in step 3, and use the labeled categories as the output of the neural network model to train the neural network model for a preset number of iterative trainings;
[0090] Step 3: After training, save the trained neural network model and deploy the trained neural network model to the Huawei Atlas200DK device.
Claims
1. A hotel suite door detection method based on YOLOv5s neural network, characterized in that: The method comprises the following steps: Step 1: Collect the images of the door sets installed in the hotel, and use ACDSee to batch resize the images to 640×640; Step 2: Label the image data in step 1 using the LabelImg target detection and annotation tool; Step 3: Use the PyTorch deep learning framework to build a basic YOLOv5s neural network model and modify the model structure to obtain a target detection model; Step 4: Use the dataset to train and test the target detection model in step 3, and deploy the trained target detection model on the Huawei Atlas200DK device; Among them, in step 3, the use of the PyTorch deep learning framework to build a basic YOLOv5s neural network model and modify the model structure includes the following steps: Step 31: Use the PyTorch framework to build a basic YOLOv5s neural network; Step 32: Modify the backbone network of the basic YOLOv5s neural network, specifically: The coordinate attention mechanism is embedded in the cross-stage local structure in the backbone network. The coordinate attention mechanism converts the original feature x output by the channel attention c (i,j) is decomposed into two one-dimensional transposed feature maps and Among them, c, h, w represent the variable channel, variable height and variable width of the feature, H and W represent the maximum height and width of the feature, i and j represent the order of accumulating features in width and height, and the two one-dimensional feature codes are aggregated along two spatial directions respectively. Feature f: Among them, δ, Conv and Concat represent nonlinear activation function, convolution and feature concatenation respectively. Aggregate features are decomposed along a spatial direction to capture long-range dependencies. Preserve precise position information along another spatial direction Among them, f h and f w They represent the feature decomposition of the aggregate feature f along the height and width directions, σ represents the Sigmoid activation function, and finally the features of these two directions are expanded and weighted with the original input features to obtain the coordinate attention feature y c (i,j): Among them, the expanded weighted sum is simplified and presented in the form of product; Step 33: Modify the neck network of the original model, specifically: Referring to the bidirectional feature pyramid structure, a bidirectional connection is established on the fourth layer in the feature pyramid plus path aggregation structure. The intermediate output and final output of the second layer are expressed as and Among them, Conv and Concat represent convolution and feature concatenation respectively. and They represent the inputs of the third, fourth, and fifth layers of the bidirectional feature pyramid in the neck network respectively. Resize means adjusting the parameter feature size to be consistent with the first parameter of Concat. Step 34: Modify the head network of the original model, specifically: The three-branch detection head used for multi-scale feature prediction in the head network is changed into two branches, the branch with the smallest grid output feature size of 80×80×255 is removed, and the output feature sizes of the two branches retained are 40×40×255 and 20×20×255 respectively.
2. A hotel suite door detection method based on YOLOv5s neural network according to claim 1, characterized in that, The steps described in step 1 are to collect the images of the door sets installed in the hotel and use ACDSee to batch resize the images to 640×640, including the following steps: Step 11: Use the Huawei Atlas200DK device equipped with a camera to capture images of the door sets installed in the hotel. Each image contains a complete door frame structure. Step 12: According to the relationship between the two pairs of sides and the four corners of the quadrilateral, the captured images are divided into 8 categories: The upper and lower frames are parallel, the left and right frames are parallel; the adjacent frames are vertical or the adjacent frames are not vertical; The upper and lower borders are parallel, and the left and right borders are not parallel: only one of the left and right borders is perpendicular to the upper and lower borders, or the adjacent borders are not perpendicular; The upper and lower borders are not parallel, and the left and right borders are parallel: only one of the upper and lower borders is perpendicular to the left and right borders, or the adjacent borders are not perpendicular; The left and right sides are not parallel, and the upper and lower frames are not parallel: only one pair of adjacent frames is vertical or both adjacent frames are not vertical; In this order from 0 to 7, add the number corresponding to the category to the beginning of each image file name; Step 13: Use ACDSee to open the folder containing the renamed images, then right-click and select "Batch Edit" under the "Batch" option, click Resize, and adjust the width and height to 640×640.
3. A hotel suite door detection method based on YOLOv5s neural network according to claim 1, characterized in that, Step 2, labeling the data using the LabelImg target detection and annotation tool, includes the following steps: Step 21: Install and run the LabelImg label detection and annotation tool and open the LabelImg tool interface; Step 22: Set the input and output folders, click "Open Dir" to select the folder where the 640×640 image is stored, and then click "Change Save Dir" to select the save path; Step 23: Set the marking format. First, check "Verify Image" to set the next image to have the same marking as the previous image. Then check the YOLO marking format. Finally, check the default save button. Step 24: Label the image according to the number in front of the file name and frame the suit door in the image.
4. A hotel suite door detection method based on YOLOv5s neural network according to claim 1, characterized in that, In step 4, the target detection model in step 3 is trained and tested using the data set, and the trained target detection model is deployed on the Huawei Atlas200DK device, including the following steps: Step 41: Before training, set the Adam optimizer to train the target detection model, with an initial learning rate of 0.001, a momentum of 0.937, and a weight decay of 0.0005; Step 42: During training, the data labeled in step 2 is used as the input of the target detection model, and the labeled categories are used as the output of the target detection model, and the target detection model is iterated for a preset number of times; Step 43: After training, save the trained target detection model and deploy the trained target detection model to the Huawei Atlas200DK device.
