A method for judging the remaining life of the handrail belt of an escalator based on image recognition

Abstract

The present invention discloses a method for judging the remaining life of the handrail belt of an escalator based on image recognition, including: installing a camera at each monitoring point of the handrail belt on the left and right sides of the escalator, and locally cropping the photo of the handrail belt taken by the camera; determining the area where the crack of the handrail belt is located in the picture, and cropping this area; performing an affine transformation operation to correct the graphics in the photo and performing another cropping; using image enhancement, binaryzation of the image, and morphological erosion and dilation to process the photo: scanning each row of data on the photo to calculate the size of the crack; establishing a crack model of the handrail belt with different service years based on the size, length, and quantity of the cracks, and then making a model through a convolutional neural network. By comparing the real-time data collected with the established model, the quantity and size of the cracks on the handrail belt can be observed in a timely manner, and the remaining life of the handrail belt can be judged, ensuring that maintenance personnel can replace the handrail belt in a timely manner and handling potential safety hazards.

Description

Technical Field

[0001] The present invention relates to the field of fault diagnosis, and particularly relates to a method for judging the remaining life of the handrail belt of an escalator based on image recognition. Background Art

[0002] As a large transportation machine, escalators are widely used in railways, urban rail transit, shopping malls, airports, etc. The handrail belt is located on the top surface of the handrail device and runs synchronously with the steps, pedals or tapes. It is a belt-shaped component for passengers to hold. As an important part of such a large rotary device as an escalator, the state of the handrail belt is directly related to the safety of passengers. As a vulnerable part of the escalator, the structure of the handrail belt is mainly composed of a rubber layer, a cord fabric layer, a steel wire layer, and a friction layer. The driving methods of the handrail belt mainly include:

[0003] 1) Friction wheel drive, 2) Pressure wheel drive, both of which use the friction of mechanical rollers to drive the handrail belt to run. When the handrail belt ages, cracks will first appear at the edge of the handrail belt. The number and size of these cracks will increase with the increase of the aging degree of the handrail belt. In severe cases, it will directly cause the handrail belt to break, which will directly affect the safe operation of the escalator.

[0004] The existing quality inspection of the handrail belt mainly relies on regular maintenance by maintenance personnel, and it is impossible to observe the number and size of cracks on the handrail belt in real time, resulting in maintenance personnel being unable to replace the handrail belt in time to deal with potential safety hazards. Summary of the Invention

[0005] To solve the deficiencies of the prior art, the present invention aims to solve the problem in the prior art that the existing quality inspection of the handrail belt mainly relies on regular maintenance by maintenance personnel, and it is impossible to observe the number and size of cracks on the handrail belt in real time, resulting in maintenance personnel being unable to replace the handrail belt in time to deal with potential safety hazards.

[0006] To achieve the above object, a method for judging the remaining life of the handrail belt of an escalator based on image recognition is characterized by including the following steps:

[0007] Step 1: Install a camera at each monitoring point of the left and right handrail belts of the escalator. When the handrail belt is in operation, the camera takes real-time pictures of the handrail belt.

[0008] Step 2: The camera takes a centered picture of the left or right handrail belt, and then crops the picture of the handrail belt taken by the camera in Step 1 to only retain the part of the handrail belt.

[0009] Step 3: Crop the photo after partial cropping in Step 2 again (reduce 1100 pixels on the left) to obtain a picture of the handrail belt, and intercept the lower part of the picture (reduce 500 pixels at the top);

[0010] Step 4: Due to the influence of the internal space of the elevator, the camera cannot vertically photograph the handrail belt, resulting in an inclined angle between the edge of the handrail belt and the edge of the picture in the picture. Therefore, perform a Hough transform operation on the picture. In order to facilitate screenshotting and retain as much data related to cracks as possible, calculate the inclination angle of the edge of the handrail belt relative to the edge of the picture, and perform an affine transform operation on the inclined handrail belt picture according to the inclination angle to correct the angle, so that the edge of the handrail belt is parallel to the edge of the picture;

[0011] Step 5: Use image enhancement, binaryzation of the image, and morphological erosion and dilation to process the noise points of the photo obtained in Step 4 to remove redundant interference factors;

[0012] Step 6: Scan the picture obtained in Step 5 to obtain the width and length of the cracks on the handrail belt;

[0013] Step 7: Based on the size, length, and quantity of the cracks, establish a crack model of the handrail belt with different service years, and then make a model through a convolutional neural network. Compare the real-time data collected with the established model to judge the remaining life of the handrail belt.

[0014] Further, the monitoring point in Step 1 refers to above the handrail belt pinch roller or friction wheel or tensioning wheel.

[0015] Further, the pixel of the camera is not less than 20 million.

[0016] Further, in Step 4, the method for calculating the inclination angle of the edge of the handrail belt relative to the edge of the picture is to perform a Hough transform operation on the picture in Step 3 to obtain the shape of the edge of the handrail belt, draw the edge line of the handrail belt, calculate the slope of the edge line, and then convert it into the inclination angle of the edge line, and finally obtain the inclination angle of the edge of the handrail belt relative to the edge of the picture.

[0017] Further, after the affine transform operation in Step 4 is corrected, if there is still redundant image interfering with the crack in the picture, it is necessary to crop the corrected picture again to obtain a cracked picture after secondary cropping.

[0018] Further, the method for scanning the image obtained in step 6 to obtain the width and length of the crack on the handrail belt is as follows: For the image obtained in step 5, mark the white pixel points as 0 and the black pixel points as 1. Gradually scan each row of pixel points on the photo obtained in step 5. When a black pixel point is scanned, record the ordinate position y1 here, and then continue to scan down until a pixel point marked as 0 is scanned, and record the ordinate position y2 here. At this time, the width of the crack affecting the handrail belt is y2 - y1; according to y1 and y2, intercept the image of this part, and scan the image column by column from left to right. When a pixel point marked as 1 is encountered, record the abscissa x1 here, and continue to scan until a pixel point marked as 0 is scanned, and record the abscissa x2 here. At this time, the length of the crack affecting the handrail belt is x2 - x1.

[0019] Further, the specific method of step 7 is as follows:

[0020] Step 7.1: Group the handrail belts by the number of years of use (for example, group the handrail belts used for one year into one group, the handrail belts used for two years into one group, the handrail belts used for three years into one group, the handrail belts used for four years into one group, and the handrail belts used for five years into one group, a total of five groups). Take no less than 300 photos of each group of handrail belts. Through the above steps, the affected area s, length l, and quantity n of the handrail belt in each photo can be obtained. Therefore, a 1*3 matrix [s1 l1 n1] can be obtained from each photo;

[0021] Step 7.2: Next, set the parameters of logistic regression in the machine learning model. The regression formula is Y = w*X + b. Take the above 1*3 matrix [s1 l1 n1] as X input, Y as the calculation result, w as the weight parameter, and b as the correction parameter. Among them, w and b are the parameters to be trained and corrected. The initial value of w is a random variable with a standard deviation of 0.01, and the initial value of b is 0. To judge the quality of the calculation result during training is to judge how much the calculated Y loss value and the actual Y loss value (loss) are, and reduce the Y loss value;

[0022] Step 7.3: Set the optimization method, use the gradient descent method, initialize the parameters, and run the machine learning model; then a model that can test the handrail belts of different years of use can be obtained. By inputting the handrail belt photo, the year of use of the handrail belt can be analyzed.

[0023] Generally speaking, compared with the prior art through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

[0024] The number and size of the cracks on the handrail belt can be observed in time, the remaining life of the handrail belt can be judged, and it is ensured that the maintenance personnel can replace the handrail belt in time to handle potential safety hazards. Description of the Drawings

[0025] Figure 1 is a schematic diagram of the installation structure of a preferred embodiment of the present invention;

[0026] Figure 2 is another schematic diagram of the installation structure of a preferred embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of the change of the picture after partial cropping of a preferred embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of the change of the picture after the Hough transform operation of a preferred embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of the change of the picture after the affine transform operation of a preferred embodiment of the present invention;

[0030] Figure 6 is a schematic diagram of the change of the picture after the processes of image enhancement, binarization of the picture, and erosion and dilation of morphology of a preferred embodiment of the present invention (the color is retained because the color change phenomenon occurs during the processing); Detailed Description of the Invention

[0031] A method for judging the remaining life of the handrail belt of an escalator based on image recognition includes the following steps:

[0032] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0033] Please refer to Figures 1-3 , the camera takes a centered photo of the handrail belt, and then crops the photo of the handrail belt (reducing 300 pixel units on the left and 300 pixel units on the right), only retaining the part of the handrail belt, as shown in the figure

[0034] Please refer to Figure 4 , taking the right side of the handrail belt as an example, for the photo in the previous step, crop it again (reducing 1100 pixels on the left) to obtain a picture of the right part of the handrail belt, intercept the lower part of the picture (reducing 500 pixels at the top) to obtain a picture, perform the Hough transform operation on this picture, draw the white edge line of the handrail belt, calculate the slope of the white line, and then convert it into the inclination angle of the straight line, and finally obtain the inclination angle of the right part of the handrail belt;

[0035] Please refer to Figure 5, since the shooting angle may cause the visual inclination of the handrail belt in the photo, perform an affine transformation operation on the photo finally cropped in step 3 to correct the graphics in the photo;

[0036] Please refer to Figure 6 , after the picture is corrected, since there are still redundant images interfering with the cracks in the picture, so, crop the corrected photo in step 4 again to obtain the picture;

[0037] Use image enhancement, image binarization, and morphological erosion and dilation to process the noise points of the photo in step 5, remove the redundant interference factors, and obtain the picture;

[0038] The picture in step 6 is binarized, with white pixel points being 0 and black pixel points being 1. Gradually scan each row of data on the photo obtained in step 6. When a black pixel point is scanned, record the vertical coordinate position y1 here, and then continue to scan down until a pixel point value of 0 is scanned, and record the vertical coordinate position y2 here. At this time, the width of the crack affecting the handrail belt is y2 - y1. According to y1 and y2, crop the picture of this part, scan it column by column from left to right, when a pixel point of 1 is encountered, record the horizontal coordinate x1 here, and continue to scan until a pixel point of 0 or the end of the picture x1 is scanned. At this time, the length of the crack affecting the handrail belt is x2 - x1;

[0039] According to the operation of the previous step, the number and influence range of the cracks can be obtained. The calculation method for the left part of the handrail belt is the same as that for the right part. Obtain the crack number and crack size data on both sides of the handrail belt.

[0040] Those skilled in the art can easily understand that the above is only a preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for judging the remaining life of the handrail belt of an escalator based on image recognition, characterized in that The steps include: Step 1: Install a camera at each monitoring point of the escalator handrail on the left and right sides. When the handrail is in operation, the camera will take real-time photos of the handrail; Step 2: The camera takes a photo of the center of the left or right handrail, and then the photo of the handrail taken by the camera in step 1 is cropped to retain only the handrail portion; Step 3: Crop the partially cropped photo in step 2 again to obtain a picture of the handrail, and cut off the lower part of the picture; Step 4: Due to the influence of the internal space of the elevator, the camera cannot shoot the handrail vertically, resulting in the handrail edge in the picture being tilted at an angle to the picture edge. Therefore, a Hough transform operation is performed on the picture. In order to facilitate screenshots and retain crack-related data as much as possible, the tilt angle of the handrail edge relative to the picture edge is calculated. Based on the tilt angle, an affine transform operation is performed on the tilted handrail picture to correct the angle so that the handrail edge remains parallel to the picture edge. Step 5: Use image enhancement, image binarization, and morphological corrosion and expansion to process step 4 to get the noise of the photo and remove unnecessary interference factors; Step 6: Scan the image obtained in step 5 to obtain the width and length of the cracks on the handrail; Step 7: Establish handrail crack models of different service life based on the size, length and number of cracks, and then make a model through convolutional neural network. The collected real-time data is compared with the established model to determine the remaining life of the handrail; Step 7: Step 7.1: Group the handrails by the number of years of use. For each group of handrails, take no less than 300 photos. Through the above steps, the affected area s, length l, and quantity n of the handrail in each picture can be obtained. Therefore, a 1 * 3 matrix can be obtained from each photo ; Step 7.2: Next, set the parameters of logistic regression in the machine learning model. The regression formula is Y = w * X + b. Take the above 1 * 3 matrix as the input of X, Y as the calculation result, w as the weight parameter, and b as the correction parameter. Among them, w and b are the parameters to be trained and corrected. The initial value of w is a random variable with a standard deviation of 0.01, and the initial value of b is 0. In the training, to judge the quality of the calculation result is to determine the loss value of the calculated Y and the actual Y loss value, and reduce the Y loss value; Step 7.3: Set the optimization method, use the gradient descent method, initialize the parameters, and run the machine learning model; then you can get a model that can test handrails of different years of use. By passing in a photo of the handrail, you can analyze the year of use of the handrail.

2. The method for judging the remaining life of the handrail belt of an escalator based on image recognition according to claim 1, characterized in that, The monitoring point in step 1 refers to the handrail belt pressure pulley, friction pulley or tension pulley above.

3. The method for judging the remaining life of the handrail belt of an escalator based on image recognition according to claim 1, characterized in that, The pixel of the camera is not less than 20 million.

4. The method for judging the remaining life of the handrail belt of an escalator based on image recognition according to claim 1, characterized in that, In the step 4, the method for calculating the inclination angle of the handrail edge relative to the image edge is to perform a Hough transform operation on the image in step 3 to obtain the edge shape of the handrail, draw the edge line of the handrail, and calculate the slope of the edge line, which is then converted into the inclination angle of the edge line, and finally obtain the inclination angle of the handrail edge relative to the image edge.

5. The method for judging the remaining life of the handrail belt of an escalator based on image recognition according to claim 1, characterized in that, After the affine transformation operation correction in step 4, if there are still redundant images of interfering cracks in the image, the corrected image needs to be cropped again to obtain a secondary cropped crack image.

6. The method for judging the remaining life of the handrail belt of an escalator based on image recognition according to claim 1, characterized in that, The method of step 6 is as follows: For the picture obtained in step 5, mark the white pixel points as 0 and the black pixel points as 1, and gradually scan each row of pixel points on the photo obtained in step 5. When a black pixel point is scanned, record the vertical coordinate position y1 here, and then continue to scan down until a pixel point marked as 0 is scanned, and record the vertical coordinate position y2 here. At this time, the width of the crack affecting the handrail belt is y2 - y1; according to y1 and y2, intercept the picture of this part, scan the picture column by column from left to right. When a pixel point marked as 1 is encountered, record the horizontal coordinate x1 here, and continue to scan until a pixel point marked as 0 is scanned, and record the horizontal coordinate x2 here. At this time, the length of the crack affecting the handrail belt is x2 - x1.

Images