Escalator drive chain elongation early warning method and system

Abstract

The application discloses an escalator driving chain elongation early warning method and system, the method comprises the following steps: a first video stream is divided into a plurality of first images; image processing is performed on the plurality of first images to synthesize a first key frame image; first area features and first perimeter features of the contour in the first key frame image are extracted to construct an abnormal shaking prediction model with area and perimeter as features; image processing is performed on a plurality of second images to synthesize a second key frame image; second area features and second perimeter features of the contour in the second key frame image are extracted, the second area features and the second perimeter features are input into the abnormal shaking prediction model to determine whether abnormal shaking of the escalator occurs; a stereo matching method is used to obtain disparity map data of the driving chain image when the escalator stops running; the length of the driving chain is obtained; early warning information is obtained and sent to a client. The application can prevent the driving chain from breaking in advance and improve the safety of the escalator.

Description

Technical Field

[0001] This invention relates to the field of escalator safety detection technology, and in particular to an escalator drive chain elongation early warning method and system. Background Technology

[0002] Escalators are typical devices used to transport passengers in public places. The escalator drive chain is the connecting component between the escalator's main unit and the main drive wheel. If the drive chain elongates or even breaks during normal escalator operation, the escalator will reverse or accelerate downwards, seriously affecting passenger safety and likely resulting in passenger injury.

[0003] Under current technology, monitoring whether the drive chain is broken involves reacting to a series of phenomena that occur after the drive chain breaks; these are merely protective measures after the chain breaks. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a method and system for early warning of escalator drive chain elongation, which can reduce the occurrence rate of drive chain breakage, predict and locate escalators prone to drive chain breakage in advance, and improve the safety of escalators.

[0005] In a first aspect, embodiments of the present invention provide an escalator drive chain elongation early warning method, the escalator drive chain elongation early warning method comprising:

[0006] Acquire the first video stream of the escalator during operation and segment the first video stream into multiple first images;

[0007] Image processing is performed on multiple first images to obtain multiple first contour images of the driving chain, and the multiple first contour images of the driving chain are combined to form a first keyframe image;

[0008] Based on the first keyframe image, extract the first area feature and the first perimeter feature of the contour in the first keyframe image, and input the first area feature and the first perimeter feature into the deep learning model for training to construct an abnormal jitter prediction model with area and perimeter as features.

[0009] Obtain the second video stream to be predicted, and segment the second video stream into multiple second images;

[0010] Image processing is performed on multiple second images to obtain multiple second contour images of the driving chain, and the multiple second contour images of the driving chain are combined into a second keyframe image;

[0011] Based on the second keyframe image, extract the second area feature and the second perimeter feature of the contour in the second keyframe image, and input the second area feature and the second perimeter feature into the abnormal jitter prediction model to determine whether the escalator has abnormal jitter; if abnormal jitter continues to occur within a preset time, control the escalator to stop running.

[0012] The drive chain image of the escalator when it is stopped is obtained, and the disparity map data of the drive chain image of the escalator when it is stopped is obtained by using a stereo matching method;

[0013] The length of the drive chain is obtained based on the disparity map data;

[0014] The length of the drive chain is compared with the initial length of the drive chain. If the length of the drive chain is longer than the initial length, an early warning message is obtained and sent to the client.

[0015] Compared with the prior art, the first aspect of the present invention has the following beneficial effects:

[0016] This method constructs an abnormal vibration prediction model based on area and perimeter. The second area feature and the second perimeter feature are input into the abnormal vibration prediction model to determine whether the escalator is experiencing abnormal vibration. If abnormal vibration continues for a preset time, the escalator is stopped, reducing the incidence of drive chain breakage and preventing it in advance, thus improving escalator safety. By comparing the length of the drive chain with its initial length, if the length is longer than the initial length, an early warning is obtained and sent to the client. This allows for early prediction and location of escalators prone to drive chain breakage, enabling timely escalator maintenance.

[0017] According to some embodiments of the present invention, the step of performing image processing on multiple first images to obtain multiple first contour images of the driving chain, and synthesizing the multiple first contour images of the driving chain into a first keyframe image, includes:

[0018] Multiple first images are processed to grayscale to obtain multiple first grayscale images;

[0019] The first grayscale images are binarized using a threshold segmentation method to separate the foreground and background of the first grayscale images, thereby obtaining the first contour images of the driving chain.

[0020] The first contour images of the drive chain are processed by multi-frame synthesis to obtain the first keyframe image.

[0021] According to some embodiments of the present invention, extracting the first area feature and the first perimeter feature of the contour in the first keyframe image based on the first keyframe image includes:

[0022] Extract the first contour of the drive chain during motion from the first keyframe image;

[0023] The first contour is drawn, and the first area feature and the first perimeter feature corresponding to the contour region after contour drawing are calculated using the calculation function in OpenCV.

[0024] According to some embodiments of the present invention, the step of performing image processing on multiple second images to obtain multiple second contour images of the driving chain, and synthesizing the multiple second contour images of the driving chain into a second keyframe image, includes:

[0025] Multiple second images are converted to grayscale to obtain multiple second grayscale images;

[0026] Multiple grayscale images of the second grayscale are binarized using a threshold segmentation method to separate the foreground and background of the second grayscale images, thereby obtaining multiple second contour images of the driving chain.

[0027] The second contour images of the drive chain are processed by multi-frame synthesis to obtain the second keyframe image.

[0028] According to some embodiments of the present invention, extracting the second area feature and the second perimeter feature of the contour in the second keyframe image based on the second keyframe image includes:

[0029] Extract the second contour of the drive chain during motion from the second keyframe image;

[0030] The second contour is drawn, and the second area feature and the second perimeter feature corresponding to the contour region after contour drawing are calculated using the calculation function in OpenCV.

[0031] According to some embodiments of the present invention, the step of inputting the second area feature and the second perimeter feature into the abnormal shaking prediction model to determine whether the escalator is experiencing abnormal shaking; and controlling the escalator to stop operating if abnormal shaking continues for a preset time, includes:

[0032] The second area feature and the second perimeter feature are input into the abnormal jitter prediction model. The abnormal jitter prediction model determines whether the second area feature exceeds the range of the first area feature and whether the second perimeter feature exceeds the range of the first perimeter feature.

[0033] If the second area feature does not exceed the range of the first area feature and the second perimeter feature does not exceed the range of the first perimeter feature, then it is determined that the escalator has not experienced abnormal shaking; if the second area feature exceeds the range of the first area feature or the second perimeter feature exceeds the range of the first perimeter feature, then it is determined that the escalator has experienced abnormal shaking.

[0034] The system continuously monitors for a preset time. If the escalator continues to vibrate abnormally at the end of the preset time, the system controls the escalator to stop operating. If the abnormal vibration of the escalator stops within the preset time, the abnormal vibration signal of the escalator is canceled, and the escalator is controlled to operate normally.

[0035] According to some embodiments of the present invention, the step of acquiring the drive chain image of the escalator when it is stopped and obtaining the disparity map data of the drive chain image of the escalator when it is stopped using a stereo matching method includes:

[0036] Images of the drive chain of the escalator when it is stopped are captured using a binocular camera. The drive chain images include a left view and a right view of the chain.

[0037] The binocular camera is calibrated, and the internal parameters of each camera are obtained through calibration, as well as the relative position between the two cameras of the binocular camera is measured.

[0038] Based on the internal parameters and the relative position, distortion elimination and row alignment are performed on the left view and the right view of the chain, respectively, so that the imaging coordinates of the left view and the right view of the chain are consistent, and the visual axes of the two cameras of the binocular camera are parallel, and the left imaging plane and the right imaging plane are coplanar.

[0039] The corresponding pixels on the left and right views of the chain in the same scene are matched using the stereo matching method to obtain disparity map data of the drive chain image when the escalator is stopped.

[0040] According to some embodiments of the present invention, obtaining the length of the drive chain based on the disparity map data includes:

[0041] Based on the disparity map data, the offset between the pixels is calculated using the triangulation principle to obtain the three-dimensional information of the driving chain;

[0042] Based on the three-dimensional information of the drive chain, the distance between adjacent links and the critical length of the chain plate in the drive chain are calculated using the calculation function in OpenCV.

[0043] The length of the drive chain is calculated based on the distance between adjacent chain links and the critical length of the chain plate.

[0044] Secondly, embodiments of the present invention also provide an escalator drive chain elongation early warning system, which includes a visual acquisition device, a visual computing device, an escalator main control subsystem, and an escalator early warning cloud platform.

[0045] The visual acquisition device is communicatively connected to the visual computing device. The visual acquisition device is used to acquire a first video stream during escalator operation, a second video stream to be predicted, and a drive chain image during escalator stop operation, and transmits the first video stream during escalator operation, the second video stream to be predicted, and the drive chain image during escalator stop operation to the visual computing device.

[0046] The visual computing device is communicatively connected to the visual acquisition device. The visual computing device is used to receive a first video stream during the operation of the escalator, a second video stream to be predicted, and an image of the drive chain during the operation of the escalator when it stops. It is also used to execute the escalator drive chain elongation early warning method as described above to obtain early warning information and abnormal jitter information.

[0047] The escalator main control subsystem is communicatively connected to the visual computing device. The escalator main control subsystem is used to receive the abnormal jitter information and control the escalator to stop running.

[0048] The escalator early warning cloud platform is communicatively connected to the visual computing device. The escalator early warning cloud platform is used to receive the early warning information and send the early warning information to the client.

[0049] Compared with the prior art, the second aspect of the present invention has the following beneficial effects:

[0050] This system can obtain early warning and abnormal vibration information through visual computing devices; receive abnormal vibration information through the escalator main control subsystem, causing the escalator to stop or resume normal operation; and receive early warning information through the escalator early warning cloud platform and send it to the client. Therefore, this system can reduce the occurrence rate of drive chain breakage, prevent drive chain breakage in advance, improve escalator safety, and also predict and locate escalators prone to drive chain breakage risks in advance, allowing for timely escalator maintenance.

[0051] According to some embodiments of the present invention, the visual acquisition device includes one or more of the following: a surveillance camera, a camera, a binocular camera, a depth camera, and a structured light camera. Attached Figure Description

[0052] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0053] Figure 1 This is a flowchart of an escalator drive chain elongation early warning method according to an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the distance between adjacent chain links and the critical length of the chain plate according to an embodiment of the present invention;

[0055] Figure 3 This is a diagram of a station-level drive chain early warning structure according to an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the drive chain state according to an embodiment of the present invention;

[0057] Figure 5 This is a structural diagram of an escalator drive chain elongation early warning system according to an embodiment of the present invention;

[0058] Figure 6 This is a top view of an escalator drive chain elongation early warning system according to an embodiment of the present invention. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0060] In the description of this invention, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or implicitly indicating the order of the technical features indicated.

[0061] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the drawings and are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0062] In the description of this invention, it should be noted that, unless otherwise explicitly defined, terms such as "setting," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0063] Escalators are typical devices used to transport passengers in public places. The escalator drive chain is the connecting component between the escalator's main unit and the main drive wheel. If the drive chain elongates or even breaks during normal escalator operation, the escalator will reverse or accelerate downwards, seriously affecting passenger safety and likely resulting in passenger injury.

[0064] Under current technology, monitoring whether the drive chain is broken involves reacting to a series of phenomena that occur after the drive chain breaks; these are merely protective measures after the chain breaks.

[0065] To address the aforementioned issues, this invention constructs an abnormal vibration prediction model based on area and perimeter. The model inputs the second area feature and the second perimeter feature to determine if the escalator is experiencing abnormal vibration. If abnormal vibration persists for a preset time, the escalator is stopped, reducing the incidence of drive chain breakage and improving escalator safety. By comparing the length of the drive chain with its initial length, a warning is issued if the drive chain length is longer than the initial length. This warning is then sent to the client, enabling early prediction and location of escalators prone to drive chain breakage, allowing for timely escalator maintenance.

[0066] Reference Figure 1 This invention provides an escalator drive chain elongation early warning method, which includes:

[0067] Step S100: Obtain the first video stream during the operation of the escalator and divide the first video stream into multiple first images.

[0068] Specifically, a visual acquisition device is used to acquire a first video stream containing the operating status of the escalator drive chain, and the first video stream is sent to a visual computing device. OpenCV is used for frame splitting processing to divide the first video stream into many single-frame images, that is, to divide the first video stream into multiple first images. For example, a 4-second video shot at 25 frames per second can be split into 100 first images.

[0069] It should be noted that the visual acquisition device in this embodiment includes one or more of the following: surveillance camera, camera, binocular camera, depth camera, and structured light camera.

[0070] Step S200: Perform image processing on multiple first images to obtain multiple first contour images of the driving chain, and synthesize the multiple first contour images of the driving chain into a first keyframe image.

[0071] Specifically, the visual computing device acquires multiple first images, performs grayscale processing on the multiple first images, and obtains multiple first grayscale images;

[0072] Multiple first grayscale images are binarized using a threshold segmentation method to separate the foreground and background of the first grayscale images, thereby obtaining multiple first contour images of the driving chain.

[0073] The first keyframe image is obtained by multi-frame synthesis of multiple first contour images of the driving chain.

[0074] In this embodiment, synthesizing images within a time period t yields all the superimposed trajectory images of the drive chain within that time period. The more images provided for synthesis, the higher the calculation accuracy and the more precise the synthesized superimposed trajectory images. This is because if a segmented frame image is used directly, the jitter generated during drive chain operation only reflects the trajectory of that single frame, making it impossible to determine whether the drive chain is continuously jittering, resulting in low accuracy in predicting drive chain breakage.

[0075] Step S300: Based on the first keyframe image, extract the first area feature and the first perimeter feature of the contour in the first keyframe image, and input the first area feature and the first perimeter feature into the deep learning model for training to construct an abnormal jitter prediction model with area and perimeter as features.

[0076] Specifically, the first contour of the driving chain during motion is extracted from the first keyframe image; the first contour is drawn, and the first area feature and first perimeter feature corresponding to the contour region after contour drawing are calculated using the calculation function in OpenCV. Specifically:

[0077] The first keyframe image is used to extract the contour of the drive chain using the contour detection function findContours(). An image may have multiple contours, such as the contour of the drive sprocket, the contour of the escalator motor, and the contour generated by the movement of the drive chain. In this embodiment, only the contour generated by the movement of the drive chain is detected.

[0078] Use the contour drawing function drawContours() to draw the contour of the drive chain;

[0079] The first area feature of the contour region generated by the driving chain motion is calculated using the ContourArea() function.

[0080] The first perimeter feature of the profile generated by the driving chain motion is calculated using the arcLength() function.

[0081] Step S400: Obtain the second video stream to be predicted and segment the second video stream into multiple second images.

[0082] Specifically, a visual acquisition device is used to acquire a second video stream containing the operating status of the escalator drive chain, and the second video stream is sent to a visual computing device. OpenCV is used for frame splitting to divide the second video stream into many single-frame images.

[0083] It should be noted that the visual acquisition device in this embodiment includes one or more of the following: surveillance camera, camera, binocular camera, depth camera, and structured light camera.

[0084] Step S500: Perform image processing on multiple second images to obtain multiple second contour images of the driving chain, and synthesize the multiple second contour images of the driving chain into a second keyframe image.

[0085] Specifically, multiple second images are processed to grayscale to obtain multiple second grayscale images;

[0086] Multiple second grayscale images are binarized using a threshold segmentation method to separate the foreground and background of the second grayscale images, thereby obtaining multiple second contour images of the driving chain.

[0087] Multiple second contour images of the driving chain are composited in multiple frames to obtain a second keyframe image. This second keyframe image contains the superimposed contour shapes of the driving chain at different times.

[0088] Step S600: Based on the second keyframe image, extract the second area feature and the second perimeter feature of the contour in the second keyframe image, and input the second area feature and the second perimeter feature into the abnormal shaking prediction model to determine whether the escalator has abnormal shaking; if abnormal shaking continues to occur within a preset time, control the escalator to stop running.

[0089] Specifically, the second contour during the driving chain motion is extracted from the second keyframe image; the second contour is drawn, and the second area feature and second perimeter feature corresponding to the drawn contour region are calculated using calculation functions in OpenCV. Specifically:

[0090] The contour of the drive chain is extracted from the second keyframe image using the contour detection function findContours(). An image may have multiple contours, such as the contour of the drive sprocket, the contour of the escalator motor, and the contour generated by the movement of the drive chain. In this embodiment, only the contour generated by the movement of the drive chain is detected.

[0091] Use the contour drawing function drawContours() to draw the contour of the drive chain;

[0092] The second area feature of the contour region generated by the driving chain motion is calculated using the ContourArea() function.

[0093] The second perimeter feature of the profile generated by the driving chain motion is calculated using the arcLength() function.

[0094] The second area feature and the second perimeter feature are input into the abnormal jitter prediction model. The abnormal jitter prediction model determines whether the second area feature exceeds the range of the first area feature and whether the second perimeter feature exceeds the range of the first perimeter feature.

[0095] If the second area feature does not exceed the range of the first area feature and the second perimeter feature does not exceed the range of the first perimeter feature, then it is determined that the escalator has not experienced abnormal shaking; if the second area feature exceeds the range of the first area feature or the second perimeter feature exceeds the range of the first perimeter feature, then it is determined that the escalator has experienced abnormal shaking, and abnormal shaking information is sent to the escalator early warning cloud platform to remind personnel to check.

[0096] The system continuously monitors the escalator for a preset time. If the escalator continues to vibrate abnormally at the end of the preset time, the system will stop the escalator from operating. If the abnormal vibration of the escalator stops within the preset time, the abnormal vibration signal of the escalator will be canceled, the visual computing device will re-enter the analysis state, and the system will control the escalator to resume normal operation.

[0097] It should be noted that the preset time in this embodiment can be changed according to actual needs, and this embodiment does not impose any specific limitations.

[0098] Step S700: Obtain the drive chain image of the escalator when it is stopped, and use a stereo matching method to obtain the disparity map data of the drive chain image of the escalator when it is stopped.

[0099] Specifically, a stereo camera is used to capture images of the drive chain of the escalator when it is stopped. These images include a left view and a right view of the chain. The stereo camera is calibrated to obtain the internal parameters of each camera and to measure the relative position between the two cameras. Based on the internal parameters and relative position, distortion correction and row alignment are performed on the left and right views of the chain to ensure that their imaging coordinates are consistent, their visual axes are parallel, and their left and right imaging planes are coplanar. Corresponding pixels in the left and right views of the chain within the same scene are matched using a stereo matching method to obtain disparity map data for the drive chain image of the escalator when it is stopped.

[0100] In this embodiment, the visual acquisition device uses a binocular camera. Binocular vision simulates the principles of human vision, using a computer to passively perceive distance and object length. The binocular camera captures images of the drive chain from two different observation points, obtaining images of the drive chain from different perspectives, including a left view and a right view of the chain. Due to the characteristics of the optical lenses, the images produced by the cameras are distorted, and due to assembly errors, the drive chain and the binocular camera are not perfectly parallel, which also causes image distortion. Therefore, the binocular camera must first be calibrated to obtain the internal parameters of each camera, such as focal length, radial distortion, tangential distortion, and imaging origin coordinates. Furthermore, calibration is also required to measure the relative position between the two cameras, such as the rotation matrix and translation amount of camera 1 relative to camera 2. Next, binocular calibration is performed. Based on the camera intrinsic parameter data obtained after camera calibration and the relative position between the two cameras, distortion correction and row alignment are performed on the left and right views of the chain, respectively, to ensure that the imaging coordinates of the left and right views of the chain are consistent, the visual axes of the two cameras of the binocular camera are parallel, and the left and right imaging planes are coplanar. After this operation, any pixel on the left view of the chain will necessarily have the same row number as its corresponding point on the right view. A search within that row is sufficient to find the corresponding chain pixel. Then, binocular matching is performed, using a stereo matching method to match corresponding pixels on the left and right views of the chain in the same scene, obtaining disparity map data of the drive chain image when the escalator is stopped.

[0101] Step S800: Obtain the length of the drive chain based on the disparity map data.

[0102] Specifically, based on the disparity map data, the offset between pixels is calculated using the triangulation principle to obtain the three-dimensional information of the driving chain;

[0103] Based on the three-dimensional information of the drive chain, the distance between adjacent links and the critical length of the chain plate are calculated using the calculation functions in OpenCV.

[0104] The length of the drive chain is calculated based on the distance between adjacent links and the critical length of the chain plate. Specifically:

[0105] Reference Figure 2 , Figure 2 In this equation, L1 represents the distance between adjacent links and L2 represents the critical length of the chain plate. The length of the drive chain is obtained by summing the distance between each adjacent link and the critical length of the chain plate.

[0106] Step S900: Compare the length of the drive chain with the initial length of the drive chain. If the length of the drive chain is longer than the initial length, obtain the warning information and send the warning information to the client.

[0107] Specifically, the length of the drive chain is compared with its initial length. If the drive chain length is longer than the initial length, a warning is issued and sent to the client.

[0108] Reference Figures 3 to 4 For example, to obtain the length of the drive chain of an escalator in a station, which typically has multiple escalators, the length of the drive chain of the same escalator at different times is compared multiple times. The result is determined by comparing the length of the drive chain at different times with the initial length of the drive chain (i.e.,...). Figure 4 The elongation of the drive chain is calculated by subtracting its normal length from its elongated length. If the elongation is longer than the initial length, an early warning is issued. The elongation is then provided to the escalator early warning cloud platform, which analyzes the elongation trend curve of the drive chain for each escalator and sends the early warning information and the elongation trend curve report to the client, providing maintenance personnel with accurate chain early warning information.

[0109] In this embodiment, an abnormal vibration prediction model is constructed using area and perimeter as features. The second area feature and the second perimeter feature are input into the abnormal vibration prediction model to determine whether the escalator is experiencing abnormal vibration. If abnormal vibration continues for a preset time, the escalator is stopped, reducing the incidence of drive chain breakage and preventing it in advance, thus improving escalator safety. By comparing the length of the drive chain with its initial length, if the length is longer than the initial length, a warning is issued and sent to the client. This allows for early prediction and location of escalators prone to drive chain breakage, enabling timely maintenance. Therefore, this embodiment can alert maintenance personnel to intervene before drive chain breakage occurs, achieving a preventative effect rather than a post-failure remedy. This minimizes the incidence of drive chain breakage and avoids accidents caused by escalator reversal due to chain breakage. This embodiment can calculate the elongation of the escalator drive chain and upload it to the escalator early warning cloud platform. The escalator early warning cloud platform allows users to view the length and elongation of the drive chain at the time of the last escalator stop. By analyzing the trend of elongation changes, a curve can be plotted, enabling early prediction and location of escalators prone to drive chain breakage, allowing for timely escalator maintenance.

[0110] Reference Figure 5 This invention also provides an escalator drive chain elongation early warning system, which includes a visual acquisition device, a visual computing device, an escalator main control subsystem, and an escalator early warning cloud platform.

[0111] The visual acquisition device is connected to the visual computing device. The visual acquisition device is used to acquire the first video stream when the escalator is running, the second video stream to be predicted, and the drive chain image when the escalator is stopped. The visual acquisition device transmits the first video stream when the escalator is running, the second video stream to be predicted, and the drive chain image when the escalator is stopped to the visual computing device.

[0112] The visual computing device is communicatively connected to the visual acquisition device. The visual computing device is used to receive a first video stream when the escalator is running, a second video stream to be predicted, and an image of the drive chain when the escalator is stopped. It is also used to execute an escalator drive chain elongation early warning method in the above embodiment to obtain early warning information and abnormal jitter information.

[0113] The escalator main control subsystem is connected to the visual computing device. The escalator main control subsystem is used to receive abnormal shaking information and control the escalator to stop running.

[0114] The escalator early warning cloud platform communicates with visual computing devices. The escalator early warning cloud platform is used to receive early warning information and send the early warning information to the client.

[0115] Reference Figure 6 , Figure 6 The system includes a visual acquisition device, a visual computing device, an escalator main control subsystem, and an escalator early warning cloud platform. The visual acquisition device is fixed in the escalator machine room truss by a bracket, and the visual acquisition lens is facing the drive chain of the escalator.

[0116] This embodiment can obtain early warning information and abnormal vibration information through visual computing devices; receive abnormal vibration information through the escalator main control subsystem, causing the escalator to stop or resume normal operation; receive early warning information through the escalator early warning cloud platform and send the early warning information to the client. Therefore, the system in this embodiment can reduce the occurrence rate of drive chain breakage, prevent drive chain breakage in advance, improve the safety of escalators, and also predict and locate escalators prone to drive chain breakage risks in advance, allowing for early escalator maintenance.

[0117] In some embodiments, the visual acquisition device includes one or more of a surveillance camera, a camera, a binocular camera, a depth camera, and a structured light camera.

[0118] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for early warning of escalator drive chain elongation, characterized in that, The escalator drive chain elongation early warning method includes: Acquire the first video stream of the escalator during operation and segment the first video stream into multiple first images; Image processing is performed on multiple first images to obtain multiple first contour images of the driving chain, and the multiple first contour images of the driving chain are combined to form a first keyframe image; Based on the first keyframe image, extract the first area feature and the first perimeter feature of the contour in the first keyframe image, and input the first area feature and the first perimeter feature into the deep learning model for training to construct an abnormal jitter prediction model with area and perimeter as features. Obtain the second video stream to be predicted, and segment the second video stream into multiple second images; Image processing is performed on multiple second images to obtain multiple second contour images of the driving chain, and the multiple second contour images of the driving chain are combined into a second keyframe image; Based on the second keyframe image, extract the second area feature and the second perimeter feature of the contour in the second keyframe image, and input the second area feature and the second perimeter feature into the abnormal jitter prediction model to determine whether the escalator has abnormal jitter; if abnormal jitter continues to occur within a preset time, control the escalator to stop running. The drive chain image of the escalator when it is stopped is obtained, and the disparity map data of the drive chain image of the escalator when it is stopped is obtained by using a stereo matching method; The length of the drive chain is obtained based on the disparity map data; The length of the drive chain is compared with the initial length of the drive chain. If the length of the drive chain is longer than the initial length, an early warning message is obtained and sent to the client.

2. The escalator drive chain elongation early warning method according to claim 1, characterized in that, The step of processing multiple first images to obtain multiple first contour images of the driving chain, and then synthesizing the multiple first contour images of the driving chain into a first keyframe image, includes: Multiple first images are processed to grayscale to obtain multiple first grayscale images; The first grayscale images are binarized using a threshold segmentation method to separate the foreground and background of the first grayscale images, thereby obtaining the first contour images of the driving chain. The first contour images of the drive chain are processed by multi-frame synthesis to obtain the first keyframe image.

3. The escalator drive chain elongation early warning method according to claim 1, characterized in that, The step of extracting the first area feature and the first perimeter feature of the contour in the first keyframe image based on the first keyframe image includes: Extract the first contour of the drive chain during motion from the first keyframe image; The first contour is drawn, and the first area feature and the first perimeter feature corresponding to the contour region after contour drawing are calculated using the calculation function in OpenCV.

4. The escalator drive chain elongation early warning method according to claim 1, characterized in that, The step of processing multiple second images to obtain multiple second contour images of the driving chain, and then synthesizing the multiple second contour images of the driving chain into a second keyframe image, includes: Multiple second images are converted to grayscale to obtain multiple second grayscale images; Multiple grayscale images of the second grayscale are binarized using a threshold segmentation method to separate the foreground and background of the second grayscale images, thereby obtaining multiple second contour images of the driving chain. The second contour images of the drive chain are processed by multi-frame synthesis to obtain the second keyframe image.

5. The escalator drive chain elongation early warning method according to claim 1, characterized in that, The step of extracting the second area feature and the second perimeter feature of the contour in the second keyframe image based on the second keyframe image includes: Extract the second contour of the drive chain during motion from the second keyframe image; The second contour is drawn, and the second area feature and the second perimeter feature corresponding to the contour region after contour drawing are calculated using the calculation function in OpenCV.

6. The escalator drive chain elongation early warning method according to claim 1, characterized in that, The second area feature and the second perimeter feature are input into the abnormal shaking prediction model to determine whether the escalator is experiencing abnormal shaking. If abnormal shaking continues for a preset time, the escalator will be stopped, including: The second area feature and the second perimeter feature are input into the abnormal jitter prediction model. The abnormal jitter prediction model determines whether the second area feature exceeds the range of the first area feature and whether the second perimeter feature exceeds the range of the first perimeter feature. If the second area feature does not exceed the range of the first area feature and the second perimeter feature does not exceed the range of the first perimeter feature, then it is determined that the escalator has not experienced abnormal shaking; if the second area feature exceeds the range of the first area feature or the second perimeter feature exceeds the range of the first perimeter feature, then it is determined that the escalator has experienced abnormal shaking. The system continuously monitors for a preset time. If the escalator continues to vibrate abnormally at the end of the preset time, the system controls the escalator to stop operating. If the abnormal vibration of the escalator stops within the preset time, the abnormal vibration signal of the escalator is canceled, and the escalator is controlled to operate normally.

7. The escalator drive chain elongation early warning method according to claim 1, characterized in that, The step of acquiring the drive chain image of the escalator when it is stopped and obtaining the disparity map data of the drive chain image of the escalator when it is stopped using a stereo matching method includes: Images of the drive chain of the escalator when it is stopped are captured using a binocular camera. The drive chain images include a left view and a right view of the chain. The binocular camera is calibrated, and the internal parameters of each camera are obtained through calibration, as well as the relative position between the two cameras of the binocular camera is measured. Based on the internal parameters and the relative position, distortion elimination and row alignment are performed on the left view and the right view of the chain, respectively, so that the imaging coordinates of the left view and the right view of the chain are consistent, and the visual axes of the two cameras of the binocular camera are parallel, and the left imaging plane and the right imaging plane are coplanar. The corresponding pixels on the left and right views of the chain in the same scene are matched using the stereo matching method to obtain disparity map data of the drive chain image when the escalator is stopped.

8. The escalator drive chain elongation early warning method according to claim 7, characterized in that, The step of obtaining the length of the drive chain based on the disparity map data includes: Based on the disparity map data, the offset between the pixels is calculated using the triangulation principle to obtain the three-dimensional information of the driving chain; Based on the three-dimensional information of the drive chain, the distance between adjacent links and the critical length of the chain plate in the drive chain are calculated using the calculation function in OpenCV. The length of the drive chain is calculated based on the distance between adjacent chain links and the critical length of the chain plate.

9. An escalator drive chain elongation early warning system, characterized in that, The escalator drive chain elongation early warning system includes visual acquisition equipment, visual computing equipment, escalator main control subsystem, and escalator early warning cloud platform: The visual acquisition device is communicatively connected to the visual computing device. The visual acquisition device is used to acquire a first video stream during escalator operation, a second video stream to be predicted, and a drive chain image during escalator stop operation, and transmits the first video stream during escalator operation, the second video stream to be predicted, and the drive chain image during escalator stop operation to the visual computing device. The visual computing device is communicatively connected to the visual acquisition device. The visual computing device is used to receive a first video stream during the operation of the escalator, a second video stream to be predicted, and an image of the drive chain during the operation of the escalator when it stops. It is also used to execute the escalator drive chain elongation early warning method as described in any one of claims 1 to 8 to obtain early warning information and abnormal jitter information. The escalator main control subsystem is communicatively connected to the visual computing device. The escalator main control subsystem is used to receive the abnormal jitter information and control the escalator to stop running. The escalator early warning cloud platform is communicatively connected to the visual computing device. The escalator early warning cloud platform is used to receive the early warning information and send the early warning information to the client.

10. The escalator drive chain elongation early warning system according to claim 9, characterized in that, The visual acquisition device includes one or more of the following: surveillance camera, camera, binocular camera, depth camera, and structured light camera.

Images