Escalator defect identification method and system based on machine vision

Through machine vision technology, the defects of escalators are identified in real time, and the safety hazards in the existing technology are solved, which is difficult to monitor dynamic changes, and the safety and detection accuracy of escalators are improved.

CN119027364APending Publication Date: 2024-11-26ANHUI SPECIAL EQUIP INSPECTION INST
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410878694.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-02
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

During the operation of the escalator, due to abnormal step gaps, defects such as comb plates, poor meshing or sinking, resulting in passenger safety hazards. The existing detection methods mainly rely on static measurement and human inspection, making it difficult to monitor dynamic changes in real time, resulting in accidents. occur.

Method used

Using machine vision-based escalator defect identification method, by obtaining the image to be detected and template images of the escalator, generating different images, determining defect candidate boxes, and using the trained defect identification model to identify defect areas and types, The management platform sends fault prompt information for timely repairs.

Benefits of technology

The defect detection accuracy during escalator operation is improved, the safety of escalator riders is greatly improved, real-time monitoring and fault reminders of dynamic changes are achieved, and the frequency and error of human inspection is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027364A_ABST
    Figure CN119027364A_ABST
Patent Text Reader

Abstract

The invention discloses an escalator defect identification method and system based on machine vision. The method comprises the steps that a to-be-detected image and a corresponding template image of an escalator in the running process of the escalator are obtained; obtaining a difference image of the to-be-detected image and the corresponding template image according to the to-be-detected image and the corresponding template image; determining a defect candidate frame in the to-be-detected image according to the difference image; cutting the image of each defect candidate frame to obtain a regional image of each defect candidate frame; and performing defect identification on the area image of each defect candidate frame by using the trained defect identification model to obtain a defect area and a defect type of the defect area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of escalators, and in particular to a method and system for identifying defects of escalators based on machine vision. Background Art

[0002] Escalators are commonly used electromechanical equipment in people's daily lives, especially in public places such as airports, subway stations, and hospitals. Escalators have many passengers and are used frequently. Unlike vertical elevators, passengers on escalators are directly exposed to the moving steps and need to face the relative movement between the steps, between the steps and the apron plate, between the steps and the comb plate, and between the handrail and the wall plate. If you are not careful, you will be caught in the gaps between the relatively moving parts, causing serious personal injuries.

[0003] If the comb plate of an escalator is damaged, such as broken teeth, or the distance between the steps and the comb plate is too large, resulting in poor meshing between the steps and the comb plate, it is easy to drag and clamp the feet of passengers standing on the steps or the items placed on them. If the steps sink due to mechanical failure during the operation of the escalator, it will cause the passengers standing on them to fall into a dangerous space. If an escalator is put into operation with some steps removed due to negligence, it is very easy for passengers to fall into the operating escalator through the exposed gap and be squeezed by the moving parts. Such tragic accidents happen from time to time.

[0004] Therefore, the relevant national standards for escalators have strict regulations on the gap between steps, the meshing between steps and comb plates, and the escalator stop protection function when steps sink or are missing. The special equipment safety technical specifications also put forward specific requirements for the inspection and testing of the above items.

[0005] Special Equipment Safety Technical Specification TSG T7001-2023 "Rules for Elevator Supervision Inspection and Periodic Inspection" requires: At any position in the working section, the gap between two adjacent steps measured from the tread is not more than 6mm; the comb plate comb teeth are intact and without defects; the meshing depth between the comb plate comb teeth and the tread tooth groove is at least 4mm, and the gap between the root of the comb tooth groove and the tread is not more than 4mm; check whether the electrical safety device can automatically stop the inspected equipment when the step sinks and no longer engages with the comb teeth; check whether the electrical safety device can automatically stop the inspected equipment before the gap caused by the missing steps or pedals appears from the comb plate position. Among them, for the gap between the steps and the meshing depth between the steps and the comb plate, the conventional inspection method is static measurement when the escalator is stopped. For the stop protection function in the case of sinking or missing steps, the detection device set near the upper and lower ends of the escalator is functionally verified after the individual steps are manually removed.

[0006] The detection of the above gaps and verification of their functions are usually only carried out during maintenance or annual regular inspections, and the special equipment safety technical specifications exempt some old escalators from the inspection of the step missing protection function. However, the conditions of the escalator during daily operation are dynamically changing. Any abnormality may cause personal injury and property loss at any time. Summary of the invention

[0007] In order to solve the technical problems existing in the background technology, the present invention proposes a method and system for identifying escalator defects based on machine vision.

[0008] The present invention proposes a method for identifying escalator defects based on machine vision, comprising:

[0009] Acquire an image to be detected of the escalator and a corresponding template image during the operation of the escalator;

[0010] Obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image;

[0011] Determine defect candidate frames in the image to be detected according to the difference image;

[0012] Cropping the image of each defect candidate frame to obtain a regional image of each defect candidate frame;

[0013] The trained defect recognition model is used to perform defect recognition on the regional image of each defect candidate frame to obtain the defect area and the defect type of the defect area; wherein the defect type includes one or more of abnormal gap between steps, comb plate defect, abnormal meshing of the comb plate and steps, and step subsidence or missing.

[0014] Preferably, after using the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect region and the defect type of the defect region, the method further includes:

[0015] Perform maintenance alarm.

[0016] Preferably, after using the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect region and the defect type of the defect region, the method further includes:

[0017] Acquire multiple images to be tested and corresponding defect areas and defect types in time sequence;

[0018] Determine whether the defect types in multiple images are all abnormal gaps between steps;

[0019] If yes, determine whether the defective areas in the multiple images to be tested appear at the same position of the elevator;

[0020] If so, it is determined that there is a problem with the track between the two steps corresponding to the position, and a fault repair prompt message indicating that there is a problem with the track between the two steps corresponding to the position is sent to the management platform and / or the mobile client;

[0021] If not, determining whether the defective areas in the plurality of images to be tested are between the same two steps;

[0022] If so, it is determined that there is a problem with the connection structure between the two steps corresponding to the defect, and a fault repair prompt message indicating that there is a problem with the connection structure between the two steps corresponding to the defect is sent to the management platform and / or the mobile client.

[0023] Preferably, the escalator image to be detected is a plurality of escalator images, which are obtained by photographing the escalator with high-definition cameras arranged at a plurality of different shooting positions and shooting angles along the conveying direction of the escalator.

[0024] Preferably, obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image specifically includes:

[0025] Align the image to be detected and the corresponding template image;

[0026] The aligned image to be detected and the corresponding template image are respectively subjected to mean filtering and grayscale conversion operations to obtain the corresponding intermediate image to be detected and intermediate template image;

[0027] According to the intermediate image to be detected and the intermediate template image, a difference image between the image to be detected and the corresponding template image is obtained.

[0028] Preferably, determining the defect candidate frame in the image to be detected according to the difference image specifically includes:

[0029] determining contours in the difference image;

[0030] According to the contour, the defect candidate box in the image to be tested is determined.

[0031] Preferably, determining the contour in the difference image specifically includes:

[0032] Perform morphological operations on the difference image to obtain a de-interferenced image corresponding to the difference image;

[0033] Find the contours in the de-perturbed image and use them as contours in the difference image.

[0034] Preferably, determining a defect candidate frame in the image to be tested according to the contour specifically includes:

[0035] Filtering contours in the difference image according to preset conditions;

[0036] Determine the defect candidate frame in the image to be detected based on the remaining contours in the difference image after filtering.

[0037] Preferably, before using the trained defect recognition model to identify the escalator area image to obtain the defect area and the defect type of the defect area, the method further includes:

[0038] Obtaining a sample set of escalator defects;

[0039] The defect areas and defect types of the escalator defect sample set are labeled to obtain the escalator defect sample set with labels;

[0040] Perform data augmentation on the annotated escalator defect sample set and establish a training set;

[0041] The pre-built defect recognition model is trained using the training set to obtain a trained defect recognition model.

[0042] Preferably, the pre-built defect recognition model is trained using the training set to obtain a trained defect recognition model, specifically including:

[0043] Input the samples in the training set into the pre-built defect recognition model;

[0044] The defect recognition model extracts multi-scale features from the regional image of the defect candidate frame to obtain feature maps of multiple scales;

[0045] Generate defect target candidate boxes of each scale based on the feature map of each scale;

[0046] Perform non-maximum suppression on the defect target candidate boxes of each scale;

[0047] Combine the anchor points in the feature maps of different scales to identify the defect target candidate box, and obtain the predicted defect area and the defect type of the predicted defect area:

[0048] The loss function is calculated according to the predicted defect area, the defect type of the predicted defect area and the corresponding annotation; and the defect recognition model is optimized according to the loss function to obtain a trained defect recognition model.

[0049] Preferably, a non-maximum suppression operation is performed on the defect target candidate box of each scale, specifically including:

[0050] Calculate the overlap between each defect target candidate box and the actual defect area;

[0051] When the overlap between the defect target candidate frames in the feature maps of multiple scales and the actual defect area is lower than a preset value, the candidate frames are regenerated according to the feature maps of multiple scales;

[0052] When the overlap between the defect target candidate boxes in the feature maps of multiple scales and the real defect area is greater than a preset value, the defect target candidate box with the largest overlap among the candidate boxes is retained.

[0053] The present invention also proposes an escalator defect identification system based on machine vision, comprising:

[0054] An acquisition module, used for acquiring an image to be detected of the escalator and a corresponding template image during the operation of the escalator;

[0055] A difference image determination module, used for obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image;

[0056] A defect candidate frame generation module is used to determine a defect candidate frame in the image to be detected based on the difference image;

[0057] A cutting module is used to crop the image of each defect candidate frame to obtain a regional image of each defect candidate frame;

[0058] The defect recognition module is used to use the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect area and the defect type of the defect area.

[0059] Preferably, it also includes:

[0060] The alarm prompt module is used to identify defects in the regional image of each defect candidate frame using the trained defect recognition model, obtain the defect area and the defect type of the defect area, and then issue a maintenance alarm;

[0061] When defects of the same defect type appear in the same defect area in a plurality of adjacent images to be tested acquired in a time sequence, fault repair prompt information of the defect of the defect type appearing in the defect area is sent to the management platform and / or the mobile client.

[0062] In the present invention, the proposed escalator defect recognition method and system based on machine vision can obtain a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image; based on the difference image, preliminarily determine the defect candidate frame in the image to be detected to preliminarily determine the approximate location of the defect; then crop the image of each defect candidate frame to obtain a regional image of each defect candidate frame; use the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame, and finally obtain an accurate defect area and defect type of the defect area. The present invention can effectively improve the accuracy of defect detection during the operation of the escalator and greatly improve the safety of passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The present invention is a flowchart of an escalator defect identification method based on machine vision in one embodiment of the present invention.

[0064] Figure 2 The figure is a schematic diagram of the structure of an elevator in one embodiment of the present invention. DETAILED DESCRIPTION

[0065] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0066] Reference Figure 1 The present invention proposes a method for identifying escalator defects based on machine vision, comprising:

[0067] Acquire an image to be detected of the escalator and a corresponding template image during the operation of the escalator;

[0068] Obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image;

[0069] Determine defect candidate frames in the image to be detected according to the difference image;

[0070] Cropping the image of each defect candidate frame to obtain a regional image of each defect candidate frame;

[0071] The trained defect recognition model is used to perform defect recognition on the regional image of each defect candidate frame to obtain the defect area and the defect type of the defect area.

[0072] The present invention can obtain a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image; based on the difference image, preliminarily determine the defect candidate frame in the image to be detected to preliminarily determine the approximate location of the defect; then crop the image of each defect candidate frame to obtain a regional image of each defect candidate frame; use the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame, and finally obtain an accurate defect region and defect type of the defect region. The present invention can effectively improve the accuracy of defect detection during the operation of the escalator, and greatly improve the safety of passengers.

[0073] Among them, the defect types include one or more of abnormal gaps between steps, defects in the comb plate, abnormal engagement between the comb plate and the steps, and sinking or missing steps.

[0074] In order to further improve the safety of elevator passengers, in this embodiment, after using the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect region and the defect type of the defect region, the following is further included:

[0075] A maintenance alarm is triggered to alert staff of escalator failure.

[0076] In a further embodiment, after using the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect region and the defect type of the defect region, the method further includes:

[0077] Acquire multiple images to be tested and corresponding defect areas and defect types in time sequence;

[0078] Determine whether the defect types in multiple images are all abnormal gaps between steps;

[0079] If yes, determine whether the defective areas in the multiple images to be tested appear at the same position of the elevator;

[0080] If so, it is determined that there is a problem with the track between the two steps corresponding to the position, and a fault repair prompt message indicating that there is a problem with the track between the two steps corresponding to the position is sent to the management platform and / or the mobile client;

[0081] If not, determining whether the defective areas in the plurality of images to be tested are between the same two steps;

[0082] If so, it is determined that there is a problem with the connection structure between the two steps corresponding to the defect, and a fault repair prompt message indicating that there is a problem with the connection structure between the two steps corresponding to the defect is sent to the management platform and / or the mobile client.

[0083] When an abnormal gap occurs between the steps of an elevator, if the defective areas in multiple images to be tested appear at the same position of the elevator, that is, as the steps move, an abnormal gap always occurs between two steps that are relatively fixed to the comb plate, then it is determined that there is a problem with the track between the two steps. Figure 2 If the track between step A and the step B below it is broken, a fault repair prompt message indicating that there is a problem with the track between the two steps corresponding to the position is sent to the management platform or / and the mobile client. If the defect of abnormal gap between steps always occurs between the same two steps, and as steps A and B move, the abnormal gap between steps AB also moves with the movement of the steps, it is determined that there is a problem with the connection structure between the two steps. At this time, a fault repair prompt message indicating that there is a problem with the connection structure between the two steps corresponding to the defect can be sent to the management platform or / and the mobile client.

[0084] This embodiment can send corresponding fault maintenance prompt information to the management platform and / or the mobile client when an abnormal gap defect occurs between steps, so that the staff can obtain targeted fault information, bring corresponding maintenance tools, and avoid extra trips.

[0085] In this embodiment, the escalator images to be detected are multiple escalator images, which are obtained by photographing the escalator with high-definition cameras arranged at multiple different shooting positions and shooting angles along the conveying direction of the escalator.

[0086] In order to obtain an accurate difference image, in this embodiment, the difference image between the image to be detected and the corresponding template image is obtained according to the image to be detected and the corresponding template image, specifically including:

[0087] Align the image to be detected and the corresponding template image;

[0088] The aligned image to be detected and the corresponding template image are respectively subjected to mean filtering and grayscale conversion operations to obtain the corresponding intermediate image to be detected and intermediate template image;

[0089] According to the intermediate image to be detected and the intermediate template image, a difference image between the image to be detected and the corresponding template image is obtained.

[0090] In a further embodiment, obtaining a difference image between the image to be detected and the corresponding template image according to the intermediate image to be detected and the intermediate template image specifically includes:

[0091] The intermediate image to be tested and the intermediate template image are subtracted pixel by pixel to obtain a difference image.

[0092] Preferably, determining the defect candidate frame in the image to be detected according to the difference image specifically includes:

[0093] determining contours in the difference image;

[0094] According to the contour, the defect candidate box in the image to be tested is determined.

[0095] In a further embodiment, determining the contour in the difference image specifically includes:

[0096] Perform morphological operations on the difference image to obtain a de-interferenced image corresponding to the difference image;

[0097] Find the contours in the de-interfered image as contours in the difference image.

[0098] In a further embodiment, determining a defect candidate frame in the image to be tested according to the contour specifically includes:

[0099] Filtering contours in the difference image according to preset conditions;

[0100] Determine the defect candidate frame in the image to be detected based on the remaining contours in the difference image after filtering.

[0101] In order to improve the measurement accuracy and robustness of the defect recognition model, in this embodiment, before using the trained defect recognition model to recognize the escalator area image to obtain the defect area and the defect type of the defect area, it also includes:

[0102] Obtaining a sample set of escalator defects;

[0103] The defect areas and defect types of the escalator defect sample set are labeled to obtain the escalator defect sample set with labels;

[0104] Perform data augmentation on the annotated escalator defect sample set and establish a training set;

[0105] The pre-built defect recognition model is trained using the training set to obtain a trained defect recognition model.

[0106] In the identification of defects such as the gap between escalator steps, the comb plate defect, the meshing of the comb plate and the step, and the sinking or missing of the step, the defects have different shapes and sizes. The step gap defect is small and subtle, while the step sinking or missing defect is large and obvious. Therefore, relying only on a single-scale feature extraction method may not be able to fully capture the defect information of these different scales.

[0107] In order to solve this problem, in a further embodiment, a pre-built defect recognition model is trained using a training set to obtain a trained defect recognition model, which specifically includes:

[0108] Input the samples in the training set into the pre-built defect recognition model;

[0109] The defect recognition model extracts multi-scale features from the regional image of the defect candidate frame to obtain feature maps of multiple scales;

[0110] Generate defect target candidate boxes of each scale based on the feature map of each scale;

[0111] Perform non-maximum suppression on the defect target candidate boxes of each scale;

[0112] Combine the anchor points in the feature maps of different scales to identify the defect target candidate box, and obtain the predicted defect area and the defect type of the predicted defect area:

[0113] The loss function is calculated according to the predicted defect area, the defect type of the predicted defect area and the corresponding annotation; and the defect recognition model is optimized according to the loss function to obtain a trained defect recognition model.

[0114] In order to remove redundant defect target candidate frames, in a further embodiment, a non-maximum suppression operation is performed on the defect target candidate frames of each scale, specifically including:

[0115] Calculate the overlap between each defect target candidate box and the actual defect area;

[0116] When the overlap between the defect target candidate frames in the feature maps of multiple scales and the actual defect area is lower than a preset value, the candidate frames are regenerated according to the feature maps of multiple scales;

[0117] When the overlap between the defect target candidate boxes in the feature maps of multiple scales and the real defect area is greater than a preset value, the defect target candidate box with the largest overlap among the candidate boxes is retained.

[0118] In order to improve the accuracy and robustness of defect detection, in one specific embodiment, the defect recognition model is a Faster R-CNN model.

[0119] In another specific embodiment, the defect recognition model is an improved Faster R-CNN, and the improved Faster R-CNN uses the residual network ResNet50 instead of VGG16 as the feature extraction network.

[0120] The present invention also proposes an escalator defect identification system based on machine vision, comprising:

[0121] An acquisition module, used for acquiring an image to be detected of the escalator and a corresponding template image during the operation of the escalator;

[0122] A difference image determination module, used for obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image;

[0123] A defect candidate frame generation module is used to determine a defect candidate frame in the image to be detected based on the difference image;

[0124] A cutting module is used to crop the image of each defect candidate frame to obtain a regional image of each defect candidate frame;

[0125] The defect recognition module is used to use the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect area and the defect type of the defect area.

[0126] In this embodiment, it also includes:

[0127] The alarm prompt module is used to identify defects in the regional image of each defect candidate frame using the trained defect recognition model, obtain the defect area and the defect type of the defect area, and then issue a maintenance alarm;

[0128] When defects of the same defect type appear in the same defect area in a plurality of adjacent images to be tested acquired in a time sequence, fault repair prompt information of the defect of the defect type appearing in the defect area is sent to the management platform and / or the mobile client.

[0129] In this embodiment, the images to be detected of the escalator are multiple escalator images; wherein the acquisition module includes multiple high-definition cameras arranged at intervals along the transmission direction of the escalator, and the shooting angles of the multiple highly linear cameras are different, so as to obtain escalator images at different shooting positions and shooting angles, which is conducive to defect detection at various positions of the escalator, such as Figure 2 shown.

[0130] In order to improve the accuracy and robustness of defect detection, in one specific embodiment, the defect recognition model is a Faster R-CNN model.

[0131] In another specific embodiment, the defect recognition model is an improved Faster R-CNN, and the improved Faster R-CNN uses the residual network ResNet50 instead of VGG16 as the feature extraction network.

[0132] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for identifying escalator defects based on machine vision. It is characterized in that include: Acquire an image to be detected of the escalator and a corresponding template image during the operation of the escalator; Obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image; Determine defect candidate frames in the image to be detected according to the difference image; Cropping the image of each defect candidate frame to obtain a regional image of each defect candidate frame; The trained defect recognition model is used to perform defect recognition on the regional image of each defect candidate frame to obtain the defect area and the defect type of the defect area; wherein the defect type includes one or more of abnormal gap between steps, comb plate defect, abnormal meshing of the comb plate and steps, and step subsidence or missing.

2. The escalator defect identification method based on machine vision according to claim 1, It is characterized in that After using the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect region and the defect type of the defect region, the following steps are also included: Acquire multiple images to be tested and corresponding defect areas and defect types in time sequence; Determine whether the defect types in multiple images are all abnormal gaps between steps; If yes, determine whether the defective areas in the multiple images to be tested appear at the same position of the elevator; If so, it is determined that there is a problem with the track between the two steps corresponding to the position, and a fault repair prompt message indicating that there is a problem with the track between the two steps corresponding to the position is sent to the management platform and / or the mobile client; If not, determining whether the defective areas in the plurality of images to be tested are between the same two steps; If so, it is determined that there is a problem with the connection structure between the two steps corresponding to the defect, and a fault repair prompt message indicating that there is a problem with the connection structure between the two steps corresponding to the defect is sent to the management platform and / or the mobile client.

3. The escalator defect identification method based on machine vision according to claim 1, It is characterized in that The escalator images to be detected are a plurality of escalator images, which are obtained by photographing the escalator with high-definition cameras arranged at a plurality of different shooting positions and shooting angles along the conveying direction of the escalator.

4. The escalator defect identification method based on machine vision according to claim 1, It is characterized in that According to the image to be detected and the corresponding template image, a difference image between the image to be detected and the corresponding template image is obtained, which specifically includes: Align the image to be detected and the corresponding template image; The aligned image to be detected and the corresponding template image are respectively subjected to mean filtering and grayscale conversion operations to obtain the corresponding intermediate image to be detected and intermediate template image; According to the intermediate image to be detected and the intermediate template image, a difference image between the image to be detected and the corresponding template image is obtained.

5. The escalator defect identification method based on machine vision according to claim 1, It is characterized in that According to the difference image, the defect candidate frame in the image to be detected is determined, which specifically includes: Perform morphological operations on the difference image to obtain a de-interferenced image corresponding to the difference image; Find the contours in the de-interference image and use them as the contours in the difference image; Filtering contours in the difference image according to preset conditions; Determine the defect candidate frame in the image to be detected based on the remaining contours in the difference image after filtering.

6. The escalator defect identification method based on machine vision according to claim 1, It is characterized in that Before using the trained defect recognition model to identify the escalator area image and obtain the defect area and the defect type of the defect area, it also includes: Obtaining a sample set of escalator defects; The defect areas and defect types of the escalator defect sample set are labeled to obtain the escalator defect sample set with labels; Perform data augmentation on the annotated escalator defect sample set and establish a training set; The pre-built defect recognition model is trained using the training set to obtain a trained defect recognition model.

7. The escalator defect identification method based on machine vision according to claim 8, It is characterized in that The pre-built defect recognition model is trained using the training set to obtain a trained defect recognition model, which specifically includes: Input the samples in the training set into the pre-built defect recognition model; The defect recognition model extracts multi-scale features from the regional image of the defect candidate frame to obtain feature maps of multiple scales; Generate defect target candidate boxes of each scale based on the feature map of each scale; Perform non-maximum suppression on the defect target candidate boxes of each scale; Combine the anchor points in the feature maps of different scales to identify the defect target candidate box, and obtain the predicted defect area and the defect type of the predicted defect area: The loss function is calculated according to the predicted defect area, the defect type of the predicted defect area and the corresponding annotation; and the defect recognition model is optimized according to the loss function to obtain a trained defect recognition model.

8. The escalator defect identification method based on machine vision according to claim 9, It is characterized in that The non-maximum suppression operation is performed on the defect target candidate box of each scale, including: Calculate the overlap between each defect target candidate box and the actual defect area; When the overlap between the defect target candidate frames in the feature maps of multiple scales and the actual defect area is lower than a preset value, the candidate frames are regenerated according to the feature maps of multiple scales; When the overlap between the defect target candidate boxes in the feature maps of multiple scales and the real defect area is greater than a preset value, the defect target candidate box with the largest overlap among the candidate boxes is retained.

9. An escalator defect identification system based on machine vision, It is characterized in that include: An acquisition module, used for acquiring an image to be detected of the escalator and a corresponding template image during the operation of the escalator; A difference image determination module, used for obtaining a difference image between the image to be detected and the corresponding template image according to the image to be detected and the corresponding template image; A defect candidate frame generation module is used to determine a defect candidate frame in the image to be detected based on the difference image; A cutting module is used to crop the image of each defect candidate frame to obtain a regional image of each defect candidate frame; The defect recognition module is used to use the trained defect recognition model to perform defect recognition on the regional image of each defect candidate frame to obtain the defect area and the defect type of the defect area.

10. The escalator defect identification system based on machine vision according to claim 9, It is characterized in that Also includes: The alarm prompt module is used to identify defects in the regional image of each defect candidate frame using the trained defect recognition model, obtain the defect area and the defect type of the defect area, and then issue a maintenance alarm; When defects of the same defect type appear in the same defect area in a plurality of adjacent images to be tested acquired in a time sequence, fault repair prompt information of the defect of the defect type appearing in the defect area is sent to the management platform and / or the mobile client.

Citation Information

Patent Citations

  • Flame detection model training method, computer readable medium and system

    CN110348390A

  • Track defect detection system and method based on image recognition

    CN112208573A

  • Defect detection method and device, electronic equipment, storage medium and program product

    CN116228746A

  • Safety device for escalator

    JP2010024018A