Elevator landing door safety detection method based on improved YOLOv5s network
By improving the YOLOv5s network, adding attention mechanism and feature pyramid modules, the problems of inaccurate bolt detection and inaccurate detection of bolts in elevator detection scenarios are solved, and efficient and accurate safety inspection of elevator floor doors is achieved.
Patent Information
- Application Number
- CN202411442625.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art cannot accurately identify the looseness or absence of bolts in elevator detection scenarios, and the detection efficiency is not real-time enough, especially when the space inside the elevator is limited and the camera viewing angle is blocked or distance limited.
Adopting the improved YOLOv5s network, by adding attention mechanism module, feature pyramid module and semantic segmentation model to Yolov5, the network's detection ability of small objects is enhanced, and feature extraction and attention are improved through Bi-FPN and NAM attention mechanism modules. mechanism to improve detection accuracy and efficiency.
It realizes accurate identification of bolt looseness and missing, improves the accuracy and efficiency of safety inspection of elevator floor doors, and can improve the detection speed without increasing calculation costs, adapting to different input resolutions and target sizes.
Smart Images
Figure CN119445468A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of safety detection based on machine vision and relates to a method for detecting loose and missing bolts based on a YOLOv5s network. Background Art
[0002] As the demand for high-rise buildings and urban transportation continues to increase, the demand for the elevator industry will also increase. This is both an opportunity and a challenge for the elevator industry. The first challenge to be faced is safety. Traditional manual elevator inspections are labor-intensive, time-consuming, and prone to problems such as missed inspections and false inspections.
[0003] Existing machine vision target detection algorithms are used in elevator inspection scenarios, where bolt safety problem detection includes how to quickly and accurately identify missing or loose bolts. However, traditional image processing methods cannot make accurate judgments in the working environment of elevators, and their detection efficiency cannot achieve real-time monitoring.
[0004] YOLOv5 is an object detection algorithm based on deep learning and neural networks. Its core idea is to transform the object detection task into a regression problem. It extracts features from images through convolutional neural networks and predicts the category and location information of objects in each grid cell. Compared with other object detection algorithms, YOLOv5 has a faster detection speed while ensuring high accuracy.
[0005] When the existing YOLOv5 algorithm is applied in the elevator visual inspection scenario, due to the relatively limited space inside the elevator and the possibility that the camera's view angle is blocked or limited by distance, some small objects (such as small tools, small items, etc.) occupy fewer pixels in the image. When Yolov5 processes these small targets, it may miss detections due to insufficient feature extraction, resulting in a decrease in the accuracy of elevator door safety detection. Summary of the invention
[0006] In view of the shortcomings of the prior art, the present invention provides an elevator door safety detection method based on an improved YOLOv5s network to solve the problem that some small objects (such as small tools, small items, etc.) occupy fewer pixels in the image due to the relatively limited space in the elevator and the camera viewing angle may be blocked or limited by distance. When Yolov5 processes these small targets, it misses detection due to insufficient feature extraction, resulting in a decrease in the accuracy of elevator door safety detection.
[0007] In view of the above situation, in order to achieve the expected effect, the present invention provides an elevator floor door safety detection technology based on deep learning target detection, which uses an industrial camera to collect corresponding floor door locks and bolt images, and improves the Yolov5 algorithm to accurately identify whether the bolts are missing or loose and whether the door locks are locked. For the detection of loose bolts, the anti-loosening line area is extracted and screened, the minimum circumscribed rectangle is calculated and the center point is found, and then the angle is used to determine whether they are aligned, thereby identifying whether the bolts are loose.
[0008] The present invention provides the following technical solution: The present invention proposes an elevator door safety detection method based on deep learning target detection, which specifically includes the following steps:
[0009] Step S1, collecting image data, using an industrial camera to collect images of bolts on the safety retainer and the door lock of the landing door;
[0010] Step S2, constructing a sample data set, marking the types of problem images and using them as labels, and constructing a problem data set with the problem images and the labels corresponding to the problem images;
[0011] Step S3, obtaining a pre-trained model, and obtaining a weight model of the Yolov5 network model after training on the sample data set;
[0012] Step S4, detection model improvement, adding an attention mechanism module, a feature pyramid module and a semantic segmentation model to Yolov5 to obtain an improved Yolov5 model;
[0013] Step S5, model construction, migrating the weight file obtained in step S3 to the improved Yolov5 network model, training the improved Yolov5 using the sample data set obtained in step S2, obtaining a model for detecting bolt loosening and floor door lock locking problems, and completing model construction;
[0014] Step S6, defect detection, using the trained model to detect and identify missing and loose bolts.
[0015] Furthermore, the present invention provides an elevator door safety detection method based on deep learning target detection, comprising:
[0016] Bi-FPN is used to replace the FPN in the original network, so that the network can adaptively learn semantic information and spatial weight parameters at different levels;
[0017] On one of the branches in the C3 module in the backbone network Backbone of the YOLOv5s network model, the standard convolution module Conv before the BottleNeck module is replaced with the NAM attention mechanism module to form an improved module.
[0018] Furthermore, the present invention provides an elevator floor door safety detection method based on deep learning target detection, including: using a 40×40 feature map after 16 times downsampling to establish a bidirectional path and cross-layer connection with a feature map of a higher level to determine whether the bolts are loose and whether the floor door is locked.
[0019] Furthermore, the present invention provides an elevator door safety detection method based on deep learning target detection, step S2, comprising:
[0020] Use industrial cameras to capture images of elevator door locks and door safety retainers. The images of elevator door locks and door safety retainers include images of loose bolts, missing bolts, unlocked door locks, and images in normal states for comparison;
[0021] Perform a preliminary screening of the collected images to remove blurry, dark or bright images, and images with inappropriate viewing angles. For each image, identify and annotate the problem type, including loose bolts, missing bolts, and unlocked door locks.
[0022] If the bolts are loose, mark the location of the bolts and the degree of looseness. If the door lock is not locked, mark the location of the door lock and its locking status.
[0023] The annotated images and their corresponding label information are stored to form a sample data set, and the sample data set is divided into a training set, a validation set, and a test set.
[0024] Furthermore, the present invention provides an elevator door safety detection method based on deep learning target detection, step S3, comprising:
[0025] Select a pre-trained Yolov5 network model and use the deep learning framework to load the weight file of the pre-trained model; the weight file includes all the parameters learned by the model during the training process;
[0026] The pre-trained model is initially evaluated on a dataset similar or related to the elevator door detection task. Since the elevator door detection task has a specific dataset and detection targets, the detection targets include bolts and door locks.
[0027] Furthermore, the present invention provides an elevator door safety detection method based on deep learning target detection, step S4, comprising:
[0028] In the backbone network Backbone of the YOLOv5s network model, find the C3 module and select the standard convolution module Conv before the BottleNeck module on one of the branches;
[0029] Replace the standard convolution module with the NAM attention mechanism module, and replace the FPN module in the original YOLOv5 network with Bi-FPN;
[0030] Bi-FPN introduces bidirectional connections in the feature pyramid structure. When constructing the feature pyramid, features are extracted from multiple layers of the backbone network, and bidirectional paths and cross-layer connections are established between adjacent layers.
[0031] A new segmentation head is added to the Head part of YOLOv5, integrating the BiSeNet semantic segmentation algorithm into the target detection model;
[0032] The attention mechanism module, Bi-FPN, and semantic segmentation model are seamlessly integrated into the YOLOv5 network, and the improved model is debugged to check for any incompatibility or performance degradation issues, and is adjusted and optimized.
[0033] Furthermore, the present invention provides an elevator door safety detection method based on deep learning target detection, step S5, comprising:
[0034] Extract the model weights from the weight file of the pre-trained Yolov5 network model obtained in step S3;
[0035] Migrate the weights to the improved Yolov5 network model in step S4;
[0036] Prepare the sample data set constructed in step S2, configure the parameters in the training process, such as learning rate, batch size, number of iterations, optimizer, and train the improved Yolov5 model using the training set prepared in step S2
[0037] Furthermore, the improved YOLOv5s target detection algorithm improves and optimizes the existing YOLOv5s target detection algorithm in the following three aspects:
[0038] Using Bi-FPN to replace the FPN in the original network can make the network focus on small targets such as bolts in the image:
[0039] When collecting images, it is impossible to take close-up photos of the detection target, so the pixel area of the detected object in the obtained image is very small, and the feature information contained is less and is easily disturbed by environmental factors. Bi-FPN is a feature bidirectional pyramid network, which expands the pyramid network and adds bidirectional connections between them, so that information can flow from bottom to top and from top to bottom. This can better capture multi-scale information and improve model performance.
[0040] The working principle of Bi-FPN is as follows:
[0041] Generating feature pyramid: Initially, feature pyramid is generated by extracting features from multiple layers of the backbone network.
[0042] Bidirectional links: Different from traditional FPN, the Bi-FPN network introduces bidirectional connections between adjacent levels of the pyramid, which means that information can flow from higher levels to lower levels (top-down) and from lower levels to lower levels (bottom-down)
[0043] Feature Integration: Bidirectionally connected networks allow for the integration of incoming information in both directions, which helps better capture features at different scales.
[0044] Weighted feature fusion: It adopts a weighted feature fusion mechanism to combine features at different levels. The fused weights are learned during the training process to ensure optimal feature integration.
[0045] Adding the NAM attention mechanism to the network can suppress some insignificant features, thereby improving the detection effect:
[0046] The working environment of elevators is complex, with many parts, and the images obtained during shooting also contain many unimportant targets. Adding an attention mechanism module can enable the network to increase or reduce its attention to some features. The NAM attention mechanism is a lightweight and efficient attention mechanism that adopts the module integration method of CBAM and redesigns the channel attention and spatial attention submodules. The attention mechanism proposed by previous researchers lacks consideration of the contribution factor of the weight, which can further suppress some insignificant features in the complex environment around the bolts. This can avoid adding fully connected layers and convolutional layers like SE and CBAM.
[0047] The core idea of the NAM attention mechanism is to normalize the attention weights and impose a sparse penalty on them so that less significant weights are suppressed, which makes the attention module more sparse and improves computational efficiency. The NAM attention mechanism uses the scaling factor in Batch Normalization. In formula 1, the scaling factor reflects the magnitude of the change in each channel and also indicates the importance of the channel. The scaling factor is the variance in BN. The larger the variance, the greater the change in the channel, that is, the richer the information contained.
[0048]
[0049] Based on Yolov5, a new segmentation head is added, and the BiSeNet semantic segmentation algorithm is added to the target detection, so that it can perform target detection and semantic segmentation at the same time.
[0050] There are very few methods for detecting bolts on elevator landing doors. The currently promoted related methods can be divided into two categories: contact and non-contact. Among them, non-contact mainly uses sound, light, electricity and other sensors. The above practical applications are extremely difficult.
[0051] Semantic segmentation is the ability to separate each part of the entire image so that each part has a certain category meaning. Unlike target detection, target detection only needs to find the target in the image, put a frame around it, and then classify it. Semantic segmentation is to divide the entire image into each area without leaving any gaps in the form of strokes. Each area is a category, and areas without categories are defaulted to background. It can help computing to achieve a more detailed and accurate understanding of the objects in the image.
[0052] The model used for semantic segmentation here is BiSeNet, which is added to the Head part of Yolov5.
[0053] In the semantic segmentation task, some existing methods attempt to retain the resolution of the input image by dilated convolution to encode sufficient spatial information, while a few use pyramid pooling modules and atrous spatial pyramid pooling to capture sufficient receptive field, which shows that both spatial information and receptive field are critical to achieving high accuracy, but it is difficult to meet this requirement at the same time.
[0054] Based on this result, BiSeNet uses a spatial path to preserve the spatial dimensions of the original input image and encode rich spatial information.
[0055] The spatial path encodes rich spatial information, but the context path aims to provide sufficient receptive field, which is very important for performance in semantic segmentation tasks. In order to expand the receptive field, some methods take advantage of "big kernels", but this method is computationally demanding and memory intensive, resulting in slow speed.
[0056] Based on this result, BiSeNet proposed a context path, which uses a lightweight model and global average pooling to provide a larger receptive field.
[0057] Semantic segmentation is an algorithm that predicts image pixels, so the calculation of evaluation indicators is based on pixels. Commonly used evaluation indicators for multi-class segmentation include category average pixel accuracy (Mean Accuracy, mAcc) and category average pixel intersection over union (Mean Iou, mIou).
[0058] In the k-category segmentation task, pixel accuracy represents the ratio of correctly predicted pixels to total pixels, and the calculation formula is as follows:
[0059]
[0060] Among them, pii represents the number of pixels whose true label is i and whose predicted result is also i, and pij represents the number of pixels whose true label is i and whose predicted result is j.
[0061] mAcc is the average sum of the pixel accuracy of each category. In the segmentation task, the background is usually also regarded as a category, which is category 0 and participates in the calculation of the evaluation index. The calculation formula of mAcc is as follows
[0062]
[0063] IoU is a standard metric in semantic segmentation, which represents the ratio of the intersection and union of the two pixel sets of the true label and the predicted result. The calculation formula is as follows:
[0064]
[0065] The category average pixel intersection over union (mIoU) is the average sum of the pixel intersection over union (IoU) of each category, and its calculation formula is as follows:
[0066]
[0067] In summary, the improved Yolov5 network adds a bidirectional feature pyramid structure, and uses a 40×40 feature map that is downsampled 16 times to establish a bidirectional path and cross-layer connection with a higher-level feature map, which can better capture target information, increase the focus on small targets, and improve the detection speed without increasing the computational cost. At the same time, it can adapt to different input resolutions and target sizes and retain the details of low-resolution feature maps. The BiSeNet model combines the advantages of the full convolutional network and the spatial pyramid pooling module, and can achieve high-efficiency and high-precision semantic segmentation at the same time. Its core idea is to divide the image into two parts: global information and local information. The advantage is that the spatial path SpatialPath uses more channels and a shallower network to retain rich spatial information to generate high-resolution features, and the context path Context Path uses fewer channels and a deeper network to quickly downsample to obtain sufficient context. Based on the output of these two networks, a Feature Fusion Module (FFM) is also designed in this paper to fuse the two features.
[0068] The beneficial effects achieved by the present invention using the above structure are as follows: The present invention proposes an elevator floor door safety detection method based on deep learning target detection, which can achieve real-time monitoring of the elevator floor door safety retainer. At the same time, its detection efficiency is also better than other methods, and it has the advantages of good effect, high performance, and accurate judgment. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0070] Figure 1 It is a schematic diagram of the method flow of the present invention;
[0071] Figure 2 It is the structural diagram of the improved Yolov5 of the present invention;
[0072] Figure 3 This is the original C3 structure diagram of the present invention;
[0073] Figure 4 The improved NAMC3 module of the present invention;
[0074] Figure 5 This is a diagram of the Bi-FPN network structure in the present invention;
[0075] Figure 6 This is the BiSeNet network structure diagram for semantic segmentation of the present invention;
[0076] Figure 7 It is a module structure diagram of the BiSeNet network in the present invention;
[0077] Figure 8 Schematic diagram for comparing performance indicators of different algorithms of the detection model in an embodiment of the present invention. DETAILED DESCRIPTION
[0078] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0079] See also Figure 1 The present invention proposes a method for elevator door safety detection based on deep learning target detection, which specifically includes the following steps:
[0080] Step S1, marking the anti-loosening line on the bolt of the floor door safety retainer, and then collecting its image through the image acquisition device to collect the image of the bolt missing and loose problem.
[0081] Step S2, constructing a sample data set: marking the type of the problem image and using it as a label, and constructing a problem data set with the problem image and the label corresponding to the problem image.
[0082] Step S3, obtaining a pre-trained model: obtaining a weight model of the Yolov5 network model after training on the sample data set.
[0083] Step S4, detection model improvement: adding an attention mechanism module and a feature pyramid module to Yolov5 to obtain an improved Yolov5 model.
[0084] Step S5, model construction: The weight file obtained in S3 is migrated to the improved Yolov5 network model, and the improved Yolov5 is trained using the sample data set obtained in S2 to obtain a model for detecting bolt loosening and floor door lock locking problems, thereby completing model construction.
[0085] Defect detection: Use the trained model to detect and identify missing and loose bolts.
[0086] Please also see Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , this embodiment provides an elevator floor door safety detection method based on deep learning target detection. For the floor door safety retainer: first, it is necessary to manually draw a bolt loosening mark line on it, and then collect the detection part image and the bolt item point image. The detection part image contains the bolts of the floor door safety retainer and the bolt loosening mark line. First, all the bolts in the image are detected. Secondly, the semantic segmentation model will segment the anti-loosening line painted on the bolt. Through analysis, it can be known that no matter what angle the shooting position is at, the anti-loosening line part of the bolt that is not loose is connected. After the anti-loosening line is segmented, its minimum circumscribed rectangle is made, and whether it is loose is determined by judging whether it is connected.
[0087] Regarding whether the floor door lock is locked: it is divided into a main lock and a secondary lock. We only need to detect whether they are engaged to determine whether it is locked. After making the data set, we first identify the door lock structure, and then add a semantic segmentation model to segment it. After fitting its edges into straight lines, we determine whether they are tangent to determine whether the door lock is locked.
[0088] In this solution, in order to improve the detection accuracy of the model for the target object, the following two improvements are made:
[0089] Using Bi-FPN to replace the FPN in the original network can make the network focus on small targets such as bolts in the image:
[0090] When collecting images, it is impossible to take close-up photos of the detection target, so the pixel area of the detected object in the obtained image is very small, and the feature information contained is less and is easily disturbed by environmental factors. Bi-FPN is a feature bidirectional pyramid network, which expands the pyramid network and adds bidirectional connections between them, so that information can flow from bottom to top and from top to bottom. This can better capture multi-scale information and improve model performance.
[0091] Adding the NAM attention mechanism to the network can suppress some insignificant features, thereby improving the detection effect. The working environment of the elevator is complex, with many parts, and the images obtained during shooting also contain many insignificant targets. Adding an attention mechanism module can enable the network to increase or reduce its attention to some features. The NAM attention mechanism is a lightweight and efficient attention mechanism that adopts the module integration method of CBAM and redesigns the channel attention and spatial attention submodules. The attention mechanism proposed by predecessors lacks consideration of the contribution factor of the weight, which can further suppress some insignificant features in the complex environment around the bolt. This can avoid adding fully connected layers and convolutional layers like SE and CBAM.
[0092] The core idea of the NAM attention mechanism is to normalize the attention weights and impose a sparse penalty on them so that less significant weights are suppressed, which makes the attention module more sparse and improves computational efficiency. The NAM attention mechanism uses the scaling factor in Batch Normalization. In formula 1, the scaling factor reflects the magnitude of the change in each channel and also indicates the importance of the channel. The scaling factor is the variance in BN. The larger the variance, the greater the change in the channel, that is, the richer the information contained.
[0093]
[0094] The embodiments of the present invention are as follows:
[0095] The hardware configuration used in this experiment is Intel 13400f-i5@4.6GHz processor, the graphics card is NVIDIA GeForce RTX4060Ti 8GB GDDR6, the operating system is Windows 10 Professional 64-bit system, and the deep learning framework is pyTorch. The development environment versions are python 3.9, pyTorch 1.10.1, CUDA 11.6, and CUDNN 8.4. In the process of training Yolov5s and improved Yolov5s, the epochs are 300, the batch-size is 8, the lr is set to 0.001, the workers are 4, the IoU threshold is 0.5, and the SGD method is used for gradient descent optimization.
[0096] The elevator door safety retainer and door lock data sets used in this embodiment are all independently constructed. A total of 854 images were taken and marked. At the same time, Pillow was used to expand the image data. Finally, there were 1500 images. After the image was expanded, the annotation software used in this embodiment was LabelImg. Due to the requirements of the YOLO algorithm, the annotation format of the data set is a txt file. Each txt file corresponds to the annotation information of an image. According to the annotation information in the txt file, the position pixel point of the target to be inspected can be determined. According to the training set: test set: validation set is 8:1:1.
[0097] In this embodiment, the area occupied by bolts in the collected image is small, and the proportion of anti-loosening wires is only a small part, which may cause extremely unbalanced sample distribution. In order to solve this problem and improve the marking accuracy, this solution reuses the Dice Loss function and the Alpha-IoU function. The Dice Loss function can solve the problem of uneven sample distribution, while the Alpha-IoU function can alleviate the problem of gradient disappearance and obtain more accurate bounding box regression. It can dynamically adjust the Loss under different IoU conditions. The form of Alpha-IoU is shown in the following formula. Its form varies, and only three common ones are listed here.
[0098] Therefore, this technical solution not only perfectly solves the above problems, improves the convergence speed and regression accuracy, but also achieves collective optimization.
[0099] When αα→0,
[0100] When α=1, L1-IoU=1-IoU=LIoU
[0101] When α=2,
[0102] This embodiment mainly uses mean average precision (mAP) and frame rate (FPS) to evaluate performance. The specific expression is as follows:
[0103]
[0104] In the formula, AP refers to the average precision of a single category, which measures the model's detection ability for that category and is the area under the PR curve. The larger the area under the PR curve, the higher the AP value, indicating that the detection model is more accurate. mAP is the average value of AP of all categories, indicating the comprehensive performance of the detection model. FPS refers to the number of image frames processed per second, which is used to measure the performance of YOLOv5 detection.
[0105] Analysis of experimental results of embodiments of the present invention:
[0106] In order to objectively evaluate the detection effect of the improved network, the evaluation indicators are used to evaluate the model results. 1 Ablation experiments are used to study the impact of key components of a system, model or algorithm on the overall performance of the system. In the field of machine learning, it is often used to evaluate the impact of different features or hyperparameters on model performance to determine which factors are essential for achieving good performance. This experiment uses ablation experiments to evaluate the contribution and importance of each component. When performing an ablation study, one or more components in the system or model are usually gradually "removed" and the performance after each change is compared. This helps to determine which components contribute most to the performance of the system and which components can be removed or improved to improve performance.
[0107] Ablation experiments are a very useful tool that can help researchers better understand the internal mechanisms of a model or algorithm and provide useful guidance for improving the model or algorithm.
[0108] It is equivalent to using the "control variable method" to test which structure or parameter has a greater impact on the result.
[0109] This embodiment first trains the original Yolov5 algorithm, and then adds the improved model networks one by one to analyze the impact of each change on the model performance. Figure 8 As shown in the figure: Compared with other algorithms, the improved elevator door safety detection model of the present invention, namely YOLOv5s+NAM+BiSeNet, has improved the accuracy P by 2.1%, and is even 0.5% higher than the accuracy of the advanced method YOLOv8. The average precision mAP@0.5 has increased by 2.0%, and the computational complexity is reduced compared with the original YOLOv5s. Although the number of parameters has increased, it is still within a reasonable range. The experimental data show that higher detection accuracy and precision can be achieved with less computation.
[0110] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. An elevator door safety detection method based on improved Yolov5s, characterized in that: The following steps are involved: Step S1, collecting image data, using an industrial camera to collect images of bolts on the safety retainer and the door lock of the landing door; Step S2, constructing a sample data set, marking the types of problem images and using them as labels, and constructing a problem data set with the problem images and the labels corresponding to the problem images; Step S3, obtaining a pre-trained model, and obtaining a weight model of the Yolov5 network model after training on the sample data set; Step S4, detection model improvement, adding an attention mechanism module, a feature pyramid module and a semantic segmentation model to Yolov5 to obtain an improved Yolov5 model; Step S5, model construction, migrating the weight file obtained in step S3 to the improved Yolov5 network model, training the improved Yolov5 using the sample data set obtained in step S2, obtaining a model for detecting bolt loosening and floor door lock locking problems, and completing model construction; Step S6, defect detection, using the trained model to detect and identify missing and loose bolts.
2. According to the method of claim 1, the elevator door safety detection method based on deep learning target detection is characterized in that: include: Bi-FPN is used to replace the FPN in the original network, so that the network can adaptively learn semantic information and spatial weight parameters at different levels; On one of the branches in the C3 module in the backbone network Backbone of the YOLOv5s network model, the standard convolution module Conv before the BottleNeck module is replaced with the NAM attention mechanism module to form an improved module.
3. The elevator door safety detection method based on deep learning target detection according to claim 1 is characterized in that: include: A bidirectional path and cross-layer connection are established between the 40×40 feature map after 16-fold downsampling and the feature map of the higher level to determine whether the bolts are loose and whether the door is locked.
4. The elevator door safety detection method based on deep learning target detection according to claim 1 is characterized in that: Step S2 comprises: Use industrial cameras to capture images of elevator door locks and door safety retainers. The images of elevator door locks and door safety retainers include images of loose bolts, missing bolts, unlocked door locks, and images in normal states for comparison; Perform a preliminary screening of the collected images to remove blurry, dark or bright images, and images with inappropriate viewing angles. For each image, identify and annotate the problem type, including loose bolts, missing bolts, and unlocked door locks. If the bolts are loose, mark the location of the bolts and the degree of looseness. If the door lock is not locked, mark the location of the door lock and its locking status. The annotated images and their corresponding label information are stored to form a sample data set, and the sample data set is divided into a training set, a validation set, and a test set.
5. The elevator door safety detection method based on deep learning target detection according to claim 1 is characterized in that: Step S3 includes: Select a pre-trained Yolov5 network model and use the deep learning framework to load the weight file of the pre-trained model; the weight file includes all the parameters learned by the model during the training process; The pre-trained model is initially evaluated on a dataset similar or related to the elevator door detection task. Since the elevator door detection task has a specific dataset and detection targets, the detection targets include bolts and door locks.
6. The elevator door safety detection method based on deep learning target detection according to claim 1 is characterized in that: Step S4 comprises: In the backbone network Backbone of the YOLOv5s network model, find the C3 module and select the standard convolution module Conv before the BottleNeck module on one of the branches; Replace the standard convolution module with the NAM attention mechanism module, and replace the FPN module in the original YOLOv5 network with Bi-FPN; Bi-FPN introduces bidirectional connections in the feature pyramid structure. When constructing the feature pyramid, features are extracted from multiple layers of the backbone network, and bidirectional paths and cross-layer connections are established between adjacent layers. A new segmentation head is added to the Head part of YOLOv5, integrating the BiSeNet semantic segmentation algorithm into the target detection model; The attention mechanism module, Bi-FPN, and semantic segmentation model are seamlessly integrated into the YOLOv5 network, and the improved model is debugged to check for any incompatibility or performance degradation issues, and is adjusted and optimized.
7. The elevator door safety detection method based on deep learning target detection according to claim 1 is characterized in that: Step S5 includes: Extract the model weights from the weight file of the pre-trained Yolov5 network model obtained in step S3; Migrate the weights to the improved Yolov5 network model in step S4; Prepare the sample data set constructed in step S2, configure the parameters in the training process, such as learning rate, batch size, number of iterations, optimizer, and train the improved Yolov5 model using the training set prepared in step S2.