Method and system to improve pattern recognition of damaged elevator components

Abstract

Embodiments of the present disclosure are directed to a computing device comprising processors configured to receive vibration data associated with an elevator assembly. The processors may identify one or more moving phases when elevator doors of the elevator assembly are moving, and one or more nonmoving phases when the elevator doors of the elevator assembly are not moving. The processors may generate modified vibration data comprising one or more portions of the received vibration data collected during the one or more moving phases, generate features based on the modified vibration data, input the features into a trained neural network, and identify damaged components of the elevator assembly based on an output of the trained neural network.

Description

METHOD AND SYSTEM TO IMPROVE PATTERN RECOGNITION OFDAMAGED ELEVATOR COMPONENTSTECHNICAL FIELD

[0001] The present disclosure generally relates to elevator systems, and more particularly, to a method and system to improve pattern recognition of damaged elevator components.BACKGROUND

[0002] A neural network may be trained to predict damage to elevator components based on vibration data associated with an elevator cab when the elevators doors are opening or closing. This vibration data may be collected using an accelerometer or similar devices attached to the elevator cab. However, this may produce noisy data that may be difficult for the neural network to use. That is, the data may be unreliable. As such, a need exists for an improved method of pattern recognition of damaged elevator components.SUMMARY

[0003] In one embodiment, a computing device may include one or more processors configured to receive vibration data associated with an elevator assembly; identify one or more moving phases when elevator doors of the elevator assembly are moving, and one or more nonmoving phases when the elevator doors of the elevator assembly are not moving; generate modified vibration data comprising one or more portions of the received vibration data collected during the one or more moving phases; generate features based on the modified vibration data; input the features into a trained neural network; and identify one or more damaged components of the elevator assembly based on an output of the trained neural network.

[0004] In another embodiment a method may include receiving vibration data associated with an elevator assembly; identifying one or more moving phases when elevator doors of the elevator assembly are moving, and one or more nonmoving phases when the elevator doors of the elevator assembly are not moving; generating modified vibration data comprising one or more portions of the received vibration data collected during the one or more moving phases; generatingfeatures based on the modified vibration data; inputting the features into a trained neural network; and identifying one or more damaged components of the elevator assembly based on an output of the trained neural network.

[0005] In another embodiment, a system may include an elevator assembly including an elevator car and elevator doors, and a computing device. The computing device may include one or more processors configured to identify one or more moving phases when the elevator doors of the elevator assembly are moving, and one or more nonmoving phases when the elevator doors of the elevator assembly are not moving; generate modified vibration data comprising one or more portions of the received vibration data collected during the one or more moving phases; generate features based on the modified vibration data; input the features into a trained neural network; and identify one or more damaged components of the elevator assembly based on an output of the trained neural network.

[0006] These and additional features provided by the embodiments described herein will be more fully understood in view of the following detailed description, in conjunction with the drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The embodiments set forth in the drawings are illustrative and exemplary in nature and not intended to limit the subject matter defined by the claims. The following detailed description of the illustrative embodiments can be understood when read in conjunction with the following drawings, wherein like structure is indicated with like reference numerals and in which:

[0008] FIG. 1A schematically depicts a first aspect of an example elevator assembly schematic, according to one or more embodiments shown and described herein;

[0009] FIG. IB schematically depicts a second aspect of an example elevator assembly schematic, according to one or more embodiments shown and described herein;

[0010] FIG. 2A depicts a portion of the elevator doors of the elevator assembly of FIGS. 1A and IB, according to one or more embodiments shown and described herein;

[0011] FIG. 2B depicts a portion of the elevator doors of the elevator assembly of FIGS. 1A and IB, according to one or more embodiments shown and described herein;

[0012] FIG. 3 schematically depicts an example computing device, according to one or more embodiments shown and described herein;

[0013] FIG. 4A depicts example vibration data, according to one or more embodiments shown and described herein;

[0014] FIG. 4B depicts example modified vibration data, according to one or more embodiments shown and described herein;

[0015] FIG. 4C depicts an example spectrogram, according to one or more embodiments shown and described herein;

[0016] FIG. 4D depicts an example Cepstrogram, according to one or more embodiments shown and described herein;

[0017] FIG. 5 depicts a flow chart of an example method for operating the computing device of FIG. 3, according to one or more embodiments shown and described herein; and

[0018] FIG. 6 depicts a flow chart of an example method for operating the computing device of FIG. 3, according to one or more embodiments shown and described herein.DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure are directed to methods and systems to improve pattern recognition of damaged elevator components. A neural network may be trained to predict damage to elevator components based on vibration data associated with elevator doors that are moving such as opening or closing. In particular, training data may be collected that includes vibration data from a plurality of elevator cabs during operation of the elevator doors, such as opening and closing of elevator doors, along with ground truth target data indicating damaged components of elevator assemblies associated with the training data. Each training example of the training data may be preprocessed to determine a Fourier Transform (e.g., a Short Time Fourier Transform) of the time domain vibration data, and atleast one of a plurality of Cepstral coefficients, which may be input to the neural network as input features. The neural network may be trained to predict damaged components based on the training data and ground truth values.

[0020] After the neural network is trained, vibration data may be collected from an elevator cab in operation while the elevator doors are opening or closing. The vibration data may be preprocessed, as discussed above, and input to the trained neural network. The neural network may output a prediction as to components of the elevator that may be damaged.

[0021] However, vibration data captured from the elevator cab may include not only data from when the elevator doors are opening and closing, but also from time period before the elevator doors start to open, a time period after the elevators doors open, but before the start to close, and a time period after the elevator doors have finished closing. Because the doors are not moving during these periods of time, any vibration collected during these periods of time are not useful to the neural network predictions.

[0022] Accordingly, in embodiments disclosed herein, a time of flight sensor may be positioned to sense movements of the elevator door. That is, the time of flight sensor may predict when elevator doors start to open, when the elevator doors have finished opening, when the elevator doors start to close, and when the elevator doors have finished closing, as disclosed herein. This data may be used to modify or trim the vibration data associated with the elevator doors to adjust the time periods when the elevator doors are moving. This modified data may then be preprocessed and input to the neural network to improve the functioning of the neural network by improving the data quality input thereto.

[0023] The phrase “communicatively coupled” is used herein to describe the interconnectivity of various components of the monitoring system for elevator assemblies and means that the components are connected either through wires, optical fibers, or wirelessly such that electrical, optical, data, and / or electromagnetic signals may be exchanged between the components. It should be understood that other means of connecting the various components of the system not specifically described herein are included without departing from the scope of the present disclosure.

[0024] Referring now to the drawings, FIG. 1 A depicts an elevator system 1 that includes an elevator assembly schematic that illustrates various components for a first aspect of an example elevator assembly 10. In this aspect, the example elevator assembly 10 may include an elevator car 12, a plurality of elevator hoisting members 14 illustrated for schematic reasons as a single suspension member and herein referred to as hoisting members, a hoistway 16 or elevator shaft, a plurality of sheaves 18, an example frame 20, and a plurality of weights 24 that act as a counterweight to the elevator car 12. The plurality of weights 24 move within the example frame 20 in the system vertical direction (i.e., in the + / - Z direction). The example frame 20 may be an elevator frame, a counterweight elevator frame, and / or the like, as discussed in greater detail herein. The plurality of elevator hoisting members 14 include a distal end 26a and a proximate end 26b. As used herein, the elevator car 12 may be referred to as an elevator cab.

[0025] Further, in this aspect, as illustrated and without limitation, the example frame 20 includes two sheaves of the plurality of sheaves 18. For example, one sheave is fixedly mounted to an upper portion of the example frame 20 positioned in an upper portion of the hoistway 16 above the elevator car 12 in a vertical direction (i.e., in the + / - Z direction) and another sheave moves with the weights 24 as the elevator car 12 moves between various landings. This is nonlimiting, and any number of the plurality of sheaves 18 may be mounted anywhere within the hoistway 16 and there may be more than or less than the two sheaves illustrated as being in the example frame 20.

[0026] At least one of the plurality of sheaves 18 within the hoistway 16 may include a motor such that the sheave is a traction sheave capable of driving the plurality of elevator hoisting members 14 through a plurality of lengths between the elevator car 12 and the traction sheave. Further, the plurality of sheaves 18 may further include a plurality of idler sheaves that may also be mounted at various positions in the hoistway 16, and, in this aspect, are also coupled to the elevator car 12. Idler sheaves are passive (they do not drive the elevator hoisting members 14, but rather guide or route the plurality of elevator hoisting members 14) and form a contact point, or engagement point, with the elevator car 12. The plurality of elevator hoisting members 14 and the plurality of sheaves 18 move the elevator car 12 between a plurality of positions within the hoistway 16 including to a plurality of landings. The plurality of sheaves 18 may include any combination of traction type sheaves and idler type sheaves.

[0027] The elevator car 12 may include at least one elevator door 36 that is configured to open and close at particular or predetermined landings. Further, in some embodiments, the elevator car 12 may include one or more sensors 38 configured to sense, detect, and / or transmit data respective to the elevator car 12. For example, the one or more sensors 38 may transmit an elevator door position, a position of the elevator car 12 within the hoistway 16, a door trip, and the like, as discussed in greater detail herein.

[0028] A plurality of additional sensors 34 may be positioned within the hoistway 16 and configured to monitor the operating conditions of the elevator car 12 and other operating conditions of the elevator assembly 10. The elevator assembly 10 may also include other sensors that may detect operational parameters associated with the elevator car 12 and other components of the elevator assembly 10. In some examples, sensors may detect errors in operation of the elevator car 12, temperature of the hoistway 16, errors in the traction sheaves 18, and the like.

[0029] As illustrated in FIG. 1A, the elevator assembly 10 is an underslung elevator system, with the idler sheaves positioned on a bottom surface of the elevator car 12. Each of the plurality of elevator hoisting members 14 may be movably coupled to the traction sheave and a portion of the plurality of elevator hoisting members 14 may be coupled to the bottom surface of the elevator car 12 to suspend the elevator car 12 via the idler sheaves. As such, the elevator hoisting members 14 pass under the elevator car 12 on a bottom of the elevator car 12 via the idler sheaves, and are coupled at the top of the hoistway 16 under tension to various structures, such as to the example frame 20, a plurality of rail caps 22 (e.g., terminating structures), and / or the like. For example, the proximate end 26b of the plurality of elevator hoisting members 14 may be fixedly coupled to the rail caps 22 and the movably coupled portion of the plurality of elevator hoisting members 14 are under tension to move the elevator car 12 between various landings. The example frame 20 may include a dead end hitch, at least one of the plurality of rail caps 22, machine beams, or other structural components.

[0030] As illustrated in FIG. 1A, the elevator assembly 10 may include a controller 40 and network interface hardware 50. The controller 40 may receive data from the elevator car 12, other elevator components (e.g., each of the plurality of sheaves 18, and the like) and may control operation of the elevator assembly 10. For example, the controller 40 may receive data (e.g., fromthe plurality of additional sensors 34, the one or more sensors 38, and the like) regarding opening and closing of the at least one elevator door 36 of the elevator car 12, speeds of the opening and closing of the at least one elevator door 36 of the elevator car 12, data regarding movement of the elevator car 12 between floors, speeds of the elevator car 12 moving between different floors, and the like. The controller 40 may also receive data regarding errors detected by various sensors (e.g., from the plurality of additional sensors 34, the one or more sensors 38, and the like) of the elevator assembly 10. The controller 40 may also receive data associated with elevator calls (e.g., when an elevator passenger pushes an elevator button to call the elevator car 12 to a particular floor). In other examples, the controller 40 may receive other data from the elevator car 12 and / or other components of the elevator assembly 10. The controller 40 may also control operation and movement of the elevator car 12. The controller 40 may also receive control signals or data from components remote to the elevator assembly 10.

[0031] The network interface hardware 50 may be communicatively coupled to the controller 40. Accordingly, the network interface hardware 50 can include a communication transceiver for sending and / or receiving any wired or wireless communication. For example, the network interface hardware 50 may include an antenna, a modem, LAN port, Wi-Fi card, WiMax card, mobile communications hardware, near-field communication hardware, satellite communication hardware and / or any wired or wireless hardware for communicating with other networks and / or devices. The network interface hardware 50 may receive data about the elevator assembly 10 captured by the controller 40. The network interface hardware 50 may also be communicatively coupled to a remote computing device 300 (FIG. 3), as discussed in further detail below.

[0032] Referring now to FIG. IB, a schematic illustrating various components for a second elevator system 1’ that includes a second aspect of an example elevator assembly 10’ is depicted. It should be appreciated that in the discussion herein, the elevator assembly 10, and components thereof, may refer to either elevator assembly 10, 10’. In this aspect, the elevator assembly 10’ may include an elevator car 12’, a plurality of elevator hoisting members 14’ illustrated for schematic reasons as a single suspension member, a hoistway 16’ or elevator shaft, a plurality of sheaves 18’, such as traction sheaves and / or idler sheaves, an example grounded frame 20’, and a plurality of weights 24’ that move within the example frame 20’ in the system vertical direction(i.e., in the + / - Z direction). In this aspect, the plurality of elevator hoisting members 14’ extend a length between the weights 24’ and the elevator car 12’. Further, in this aspect, at least one of the plurality of sheaves 18’ is a traction sheave, which, for example, may be mounted to a lower surface of the hoistway 16’. This is non-limiting, and the traction sheave of the plurality of sheaves 18’ may be mounted anywhere within the hoistway 16’ and the plurality of sheaves 18’ may include a plurality of idler sheaves and at least one traction sheave. It should be appreciated that the traction sheave may include a motor such that at least one of the plurality of sheaves 18’ is a device to drive the plurality of elevator hoisting members 14’ through a plurality of lengths with respect to the length between the traction sheave and the contact point of the elevator car 12’. The idler sheaves may also be mounted at various positions in the hoistway 16’ including within the example frame 20’. The idler sheaves are passive (they do not drive the plurality of elevator hoisting members 14’ but rather guide or route the plurality of elevator hoisting members 14’). The plurality of elevator hoisting members 14’ are coupled to the elevator car 12’ to form the contact point. At least one temperature sensor 34’ may be positioned within the hoistway 16’. The at least one temperature sensor 34’ may output data indicative to a temperature within the hoistway 16’. The elevator assembly 10’ may also include the controller 40 and the network interface hardware 50.

[0033] It should be appreciated that the illustrated schematics of FIGS. 1 A-1B are merely examples and that the plurality of elevator hoisting members 14 routing may vary significantly or slightly from these illustrated schematics. For example, there may be several idler sheaves positioned in the hoistway 16 between the traction sheave and the contact point with the elevator car 12.

[0034] Referring now to FIG. 2 A, an upper portion of the at least one elevator door 36 is shown. The example of FIG. 2A schematically depicts the elevator car 12 having center opening doors. However, in other examples, the elevator car 12 may have a single speed slide opening door design or style, a two-speed side opening door design or style, a three-speed side opening door design or style, a two-speed center opening door design or style, a three-speed center opening door design or style, among others. In some examples, elevator doors may be on multiple sides of the elevator car 12.

[0035] In the example of FIG. 2A, a first elevator door panel 36A and a second elevator door panel 36B are shown. It should be appreciated that the first elevator door panel 36A and the second elevator door panel 36B may each be panels of the elevator door 36. The first and second elevator door panels 36A and 36B may be controlled by the controller 40 to open and close so that passengers can enter and exit the elevator car 12.

[0036] In the example of FIG. 2A, a first accelerometer 200 is affixed to the first elevator door panel 36A and a second accelerometer 202 is affixed to the second elevator door panel 36B. The first and second accelerometers 200, 202 may measure vibration data associated with the first and second elevator door panels 36 A, 36B. That is, as the elevator door panels 36 A, 36B open and close, the first and second accelerometers 200, 202 measure time series acceleration values of the elevator door panels 36 A, 36B. This may include vibration data that may be used to determine whether certain components of the elevator assembly 10 (FIG. 1A) are damaged, as disclosed in further detail below. In some examples, the accelerometers 200, 202 may also be mounted to or embedded in door headers, such as door header 218 of the elevator car 12. Further, in additional non-limiting examples, one accelerometers 200, 202 may be mounted to or embedded in door headers, such as door header 218 of the elevator car 12, while the other one accelerometers 200, 202 may be affixed to the door panel (e.g., the first elevator door panel 36A and / or the second elevator door panel 36B).

[0037] The example elevator door panels 36 of FIG. 2A also includes sprocket 208, pulley 210 and linkage arms 212, 214. The sprocket 208 and the pulley 210 may rotate to cause the linkage arms 212, 214 to open and close the elevator door panels 36A, 36B, as appreciated by those having skill in the art.

[0038] In the illustrated example, the first and second accelerometers 200, 202 each measure acceleration values along three axes. In other examples, the accelerometers 200, 202 may only measure acceleration values along a single axis (i.e., the axis along which the elevator door panels 36A, 36B open and close). In some examples, a single accelerometer may be affixed to the one or more elevator door panels 36 rather than one accelerometer being affixed to each of the first and second elevator door panels 36A, 36B.

[0039] Referring still to FIG. 2A, a time-of-flight (TOF) sensor 204 may be affixed to the elevator door panel 36A such as to an upper portion of the first elevator door panel 36A, and a target 206 is affixed to the second elevator door panel 36B such as an upper portion of the second elevator door panel 36B. The TOF sensor 204 may emit a laser beam, or other light, towards the target 206. The emitted laser beam, or light, may reflect off the target 206 back towards the TOF sensor 204 and may be detected by the TOF sensor 204. The TOF sensor 204 may detect the time it takes for the laser beam to be emitted from the TOF sensor 204, reflected off the target 206, and the return detected by the TOF sensor 204. This detected time of flight may indicate a distance between the first and second elevator door panels 36A, 36B. In an example elevator car having side opening doors, the TOF sensor 204 may include an emitter affixed to a door panel (e.g., the elevator door panel 36A and / or the elevator door panel 36B) and a target affixed to the elevator car 12 such as the door header 218 or other fixed structural component (e.g., cab header), or an emitter may be affixed to the elevator car 12, such as the door header 218 or other fixed structural component (e.g., cab header), and the target may be affixed to the door panel (e.g., the first elevator door panel 36A and / or the second elevator door panel 36B).

[0040] In some examples, the TOF sensor 204 may continually emit the laser beam, or light, and measure its time of flight. As such, when the elevator door panels 36 begin to open and move apart from each other, the detected time of flight will increase. This may allow the elevator assembly 10 to precisely measure the times when the elevator door panels 36 begin to open, finish opening, begin closing, and finish closing. These times may be used to trim the data collected by the accelerometers 200, 202 as disclosed in further detail below.

[0041] Referring now to FIG. 2B, another example portion of the elevator door panels 36 is shown. FIG. 2B is similar to FIG. 2A except that the TOF sensor 204 is mounted to one end of a door hanger track 215, and the door hanger track 215 is affixed to the elevator door panel 36A. As such, in this example, the elevator door panel 36A may travel along the door hanger track 215 when moved by the actuator 216.

[0042] Referring now to FIG. 3, a remote computing device 300 is schematically depicted. The remote computing device 300 may be communicatively coupled to the elevator assembly 10 of FIG. 1 A or the elevator assembly 10’ of FIG. IB. In particular, the remote computing device300 may be communicatively coupled to the network interface hardware 50 of FIG. 1A. In the illustrated example, the remote computing device 300 includes a cloud computing server. However, in other examples, the remote computing device 300 may be any other type of computing device. In the illustrated example, the remote computing device 300 is located remotely from the elevator assembly 10. However, in other examples, the remote computing device 300 may be located in the same location as the elevator assembly 10 (e.g., in the same building as the elevator assembly 10). In embodiments, the remote computing device 300 may be communicatively coupled to a plurality of elevator assemblies and / or to a plurality of elevator systems that include various elevator assemblies, and as such, may receive data from multiple elevator assemblies and / or elevator systems, as disclosed herein.

[0043] In the example of FIG. 3, the remote computing device 300 includes one or more processors 302, one or more memory modules 304, network interface hardware 306, and a communication path 308. The one or more processors 302 may be a controller, an integrated circuit, a microchip, a computer, a central processing unit (CPU), or any other computing device. The one or more memory modules 304 may include RAM, ROM, flash memories, hard drives, or any device capable of storing machine readable and executable instructions such that the machine readable and executable instructions can be accessed by the one or more processors 302.

[0044] The network interface hardware 306 can be communicatively coupled to the communication path 308 and can be any device capable of transmitting and / or receiving data via a network. Accordingly, the network interface hardware 306 can include a communication transceiver for sending and / or receiving any wired or wireless communication. For example, the network interface hardware 306 may include an antenna, a modem, LAN port, Wi-Fi card, WiMax card, mobile communications hardware, near-field communication hardware, satellite communication hardware and / or any wired or wireless hardware for communicating with other networks and / or devices. The network interface hardware 306 of the remote computing device 300 may transmit data to and receive data from the elevator assembly 10. For example, the network interface hardware 306 of the remote computing device 300 may be communicatively coupled to the network interface hardware 50 of the elevator assembly 10.

[0045] The one or more memory modules 304 include a vibration data reception module 310, a TOF data reception module 312, an elevator data reception module 314, a moving phase determination module 316, a vibration data trimming module 318, a feature extraction module 320, a neural network training module 322, an elevator damage determination module 324, and a warning transmission module 326. Each of the vibration data reception module 310, the TOF data reception module 312, the elevator data reception module 314, the moving phase determination module 316, the vibration data trimming module 318, the feature extraction module 320, the neural network training module 322, the elevator damage determination module 324, and the warning transmission module 326 may be or the combination be a program module in the form of operating systems, application program modules, and other program modules stored in the one or more memory modules 304 (e.g., each of which may be embodied as a computer program, firmware, or hardware, as an example). In some embodiments, the program modules may be stored in the elevator assembly 10 rather than the remote computing device 300. Such a program module may include, but is not limited to, routines, subroutines, programs, objects, components, data structures and the like for performing specific tasks or executing specific data types as will be described below.

[0046] The vibration data reception module 310 may receive vibration data from the accelerometers 200, 202 affixed to the first and second elevator door panels 36 A, 36B. As discussed above, the accelerometers 200, 202 may be affixed to the elevator door panels 36A, 36B such that the accelerometers 200, 202 move when the elevator door panels 36 move. As such, the accelerometers 200, 202 may collect vibration data associated with the elevator door panels 36. The network interface hardware 50 may transmit the vibration data collected by the accelerometers 200, 202 to the remote computing device 300. The accelerometer data may be collected by the vibration data reception module 310.

[0047] In the illustrated example, an accelerometer may be embedded in or affixed to each elevator door panel 36 A, 36B. However, in other examples, an accelerometer may only be embedded in or affixed to one of the elevator door panels 36A or 36B. In the illustrated example, the accelerometers 200, 202 collect acceleration data over three axes. However, in other examples, the accelerometers 200, 202 may collect acceleration data over a single axis (e.g., the axis alongwith the elevator door panels 36 open and close). As used herein, acceleration data may be referred to as vibration data.

[0048] FIG. 4A shows an example plot of data that may be received by the vibration data reception module 310 from one of the accelerometers 200, 202. The plot of FIG. 4A includes time series acceleration data associated with one of the elevator door panels 36. As shown in FIG. 4A, each time that one of the elevator door panels 36 opens or closes, vibration data is recorded. For example, vibration data is recorded around 10 seconds in the example of FIG. 4A. However, after the elevator door finishes opening or closing, no additional vibration data is recorded until the elevator door begins to move again. In the example of FIG. 4A, no vibration data is recorded between about 15 seconds and 19 seconds. As used herein, a time period when an elevator door is moving may be referred to as a moving phase, and a time period when an elevator door is not moving may be referred to as a nonmoving phase.

[0049] As discussed above, vibration data associated with an elevator door (e.g., the elevator door panel 36 A, 36B) may be processed and input to a trained neural network, which may output a prediction of which components, if any, of the elevator door are damaged. In particular, the neural network may be trained to classify damage from features extracted from vibration data into various categories indicating different types of damaged components associated with the vibration data. This is possible due to the fact that different types of damage may cause minute variations in the vibration data associated with an elevator door, which a neural network may be trained to detect.

[0050] While a neural network may be trained to classify damage from features extracted from vibration data, such as the vibration data of FIG. 4A, to indicate damage to components, only data collected during a moving phase is useful to the neural network, as data collected during a nonmoving phase does not contain any useful vibration data. Accordingly, data received by the vibration data reception module 310 may be trimmed to eliminate data collected during nonmoving phases, as discussed in further detail below.

[0051] Referring back to FIG. 3, the TOF data reception module 312 may receive data from the TOF sensor 204. In particular, the TOF data reception module 312 may receive data indicating a time-of-flight for a laser beam to be emitted by the TOF sensor 204, reflected off thetarget 206, and detected by the TOF sensor 204. This time-of-flight data may be continually received by the TOF data reception module 312 from the TOF sensor 204. The measured time- of-flight may indicate a distance between the elevator door panels 36A and 36B. That is, when the elevator door panels 36 A, 36B are closed, the time-of-flight will have a first value. As the elevator door panels 36 A, 36B begin to open, the time-of-flight will begin to increase as the distance between the elevator door panels increases. Finally, when the elevator door panels 36A, 36B are fully open, the time-of-flight will reach its maximum value.

[0052] Accordingly, the data received by the TOF data reception module 312 may be used to determine when the elevator door panels 36A, 36B begin to open, and when they are fully open. Similarly, the data received by the TOF data reception module 312 may be used to determine when the elevator door panels 36 A, 36B begin to close, and when they are fully closed. As such, this data may be used to determine moving phases and nonmoving phases of the elevator door panels 36 A, 36B, as discussed in further detail below.

[0053] Referring still to FIG. 3, the elevator data reception module 314 may receive data from the controller 40 (FIG. 1 A). In particular, the elevator data reception module 314 may receive data from the controller 40 (FIG. 1A) indicating that the elevator doors (e.g., the elevator door panels 36A, 36B) are being opened or closed. In some examples, the TOF data reception module 312 only receives data after the controller 40 (FIG. 1A) indicates that the elevator door panels 36A, 36B are about to begin opening or closing. Similarly, in some examples, the TOF sensor 204 may only operate after the controller 40 (FIG. 1 A) indicates that the elevator door panels 36 A, 36B are about to begin opening or closing. This may reduce the amount of time that the TOF sensor 204 and / or the TOF data reception module 312, which may reduce power consumption and / or data transmission. In some examples, data received by the elevator data reception module 314 may be used to determine moving phases and nonmoving phases of the elevator door panels 36A, 36B, as discussed in further detail below.

[0054] Referring still to FIG. 3, the moving phase determination module 316 may determine moving phases as nonmoving phases of the elevator door panels 36A, 36B (FIG. 2A), as disclosed herein. As discussed above, a neural network trained to detect damaged elevator components based on vibration data from the elevator door panels 36A, 36B (FIG. 2A) mayoperate more effectively if only vibration data collected during moving phases of the elevator doors is used. As such, the moving phase determination module 316 may determine when moving phases occur, so that vibration data can be modified accordingly.

[0055] In one example, the moving phase determination module 316 may determine when moving phases of the elevator door panels 36 A, 36B (FIG. 2A) occur based on data received by the TOF data reception module 312. In particular, when time-of-flight data received by the TOF data reception module 312 is constant, indicating that the elevator door panels 36A, 36B (FIG. 2A) are not moving, the moving phase determination module 316 may determine that the elevator doors are in a nonmoving phase. However, when the time-of-flight data received by the TOF data reception module 312 is increasing or decreasing, indicating that the elevator door panels 36A, 36B (FIG. 2A) are opening or closing, the moving phase determination module 316 may determine that the elevator doors are in a moving phase. The moving phase determination module 316 may record time stamps indicating starting and ending times of moving phases such that they may be correlated to vibration data received by the vibration data reception module 310, as disclosed in further detail below.

[0056] In some examples, the moving phase determination module 316 may determine when moving phases of the elevator door panels 36 A, 36B (FIG. 2A) occur based on data received by the elevator data reception module 314. In particular, when the elevator data reception module 314 receives a signal from the controller 40 (FIG. 1 A) that the elevator doors are to be opened or closed, the moving phase determination module 316 may record a time stamp indicating that a moving phase is beginning. Similarly, when the elevator data reception module 314 receives a signal from the controller 40 (FIG. 1 A) that the elevator doors (e.g., the elevator door panels 36A, 36B depicted in FIG. 2A) are to stop opening or closing, the moving phase determination module 316 may record a time stamp indicating that a moving phase is ending. However, this data may not be as accurate as the time-of-flight data in determining moving and nonmoving phases, as there may be a lag between when a signal is sent to begin opening or closing the elevator door panels 36A, 36B, (FIG. 2A) and when the elevator doors actually begin opening or closing.

[0057] In another example, the moving phase determination module 316 may determine when moving phases of the elevator door panels 36 A, 36B (FIG. 2A) occur based on data receivedby the vibration data reception module 310. In particular, the moving phase determination module 316 may determine that a moving phase begins when the vibration data received from the accelerometers 200 and / or 202 begins to change. However, this data may not be as accurate as the data as time-of-flight data to determine moving and nonmoving phases as data from the accelerometers 200 and / or 202 may drift over time and may have integration error over longer times.

[0058] In other examples, the moving phase determination module 316 may determine when moving and nonmoving phases of the elevator door panels 36A, 36B (FIG. 2A) occur based on other types of data. For example, the moving phase determination module 316 may determine when moving and nonmoving phases occur based on data received from a Hall effect sensor, or other types of sensors embedded in or associated with the elevator door panels 36 A, 36B (FIG. 2A).

[0059] Referring still to FIG. 3, the vibration data trimming module 318 may trim the vibration data received by the vibration data reception module 310 based on when moving and nonmoving phases occur, as determined by the moving phase determination module 316. In particular, the vibration data trimming module 318 may trim or remove vibration data collected by the accelerometers 200, 202 during nonmoving phases of the elevator door panels 36A, 36B (FIG. 2A).

[0060] As discussed above, FIG. 4A illustrates example vibration data that may be received by the vibration data reception module 310. As shown in FIG. 4A, the vibration data includes several moving phases and nonmoving phases, which may be identified by the moving phase determination module 316 using the techniques discussed above. After identifying the moving and nonmoving phases, the vibration data trimming module 318 may remove the vibration data from the nonmoving phases, thereby generating modified vibration data containing only the vibration data from the moving phases, as shown in FIG. 4B. The modified data may then be used to train a neural network or to be input into a trained neural network, as discussed in further detail below.

[0061] Referring back to FIG. 3, the feature extraction module 320 may extract features from the modified vibration data generated by the vibration data trimming module 318. Theextracted features may be input to a neural network. In the illustrated example, the feature extraction module 320 performs a Fourier Transform of the modified vibration data generated by the vibration data trimming module 318. FIG. 4C shows an example spectrogram of vibration data generated by performing Short Time Fourier Transform with a 256 sample Hanning window and 75% sample overlap, with overlapping windows frames of 31 ms duration each. However, it should be understood that the feature extraction module 320 may perform any method to determine a Fourier Transform of the modified vibration data.

[0062] In the illustrated example, after the Fourier Transform is determined, the feature extraction module 320 determines at least one of a plurality of Cepstral coefficients of the modified vibration data based on the Fourier Transform. As used herein, the cepstrum C(T) is defined as the Fourier Transform of the logarithmic spectrum (the natural logarithm of the magnitude of the Fourier spectrum) of the time signal. The Cepstral features are expressed by the following equation:C(T) = ^{ln[|y{ / (t)}|]}Where T denotes the Fourier Transform, t is time, f(t) is the time signal and the independent variable r represents the quefrency (this quantity has the dimension of time and is defined as the reciprocal of the frequency spacing in Hz in the original frequency spectrum). An example Cepstrogram is shown in FIG. 4D with 10% overlap between segments.

[0063] In the illustrated example, the feature extraction module 320 uses the first 64 real coefficients of the Cepstrum as features to be input to a neural network. However, in other examples, any other number of coefficients of the Cepstrum may be used. Furthermore, in still other examples, the feature extraction module 320 may generate features from the modified vibration data in any other matter.

[0064] Referring back to FIG. 3, the neural network training module 322 may train a neural network to predict elevator damage based on vibration data. In particular, the vibration data reception module 310 may receive vibration data from a plurality of elevators and the TOF data reception module 312 may receive time-of-flight data from each of the elevators. All of this vibration data may be used as training data to train a neural network. In particular, the movingphase determination module 316 may determine moving phases for each set of training data, the vibration data trimming module 318 may generate modified vibration data for each set of training data, and the feature extraction module 320 may generate features for each set of training data.

[0065] The remote computing device 300 may also receive ground truth data associated with each set of training data indicating what, if any, elevator damage is associated with each set of training data. The neural network training module 322 may then utilize the training data and ground truth values to train a neural network to predict elevator damage based on input vibration data, using supervised learning techniques. After the neural network is trained, the trained neural network may be used to make real-time prediction as to what components of an elevator are damaged, as discussed in further detail below.

[0066] Referring still to FIG. 3, the elevator damage determination module 324 may determine what components of the elevator assembly 10 are damaged, as disclosed herein. In particular, after the feature extraction module 320 extracts features from the modified vibration data generated by the vibration data trimming module 318, the elevator damage determination module 324 may input the extract features into a trained neural network (e.g., a neural network trained by the neural network training module 322). The trained neural network may output a prediction as to what components of the elevator assembly 10 may be damaged based on the input features. As such, the elevator damage determination module 324 may determine which components of the elevator assembly 10 are damaged based on the output of the trained neural network.

[0067] Referring still to FIG. 3, the warning transmission module 326 may transmit a warning based on the determination made by the elevator damage determination module 324. In particular, if the elevator damage determination module 324 determines that any components of the elevator assembly 10 are damaged, the warning transmission module 326 may transmit a warning, an alert, and / or the like, to an owner, operator, or technician associated with the elevator assembly 10 (FIG. 1A) indicating which components are damaged. The party receiving the warning may then cause corrective actions to occur in order to repair or replace the damaged components. In some embodiments, the warning or alert may include specific data related to which components are damaged, how badly the damage (e.g., degrees of damage or specific indicatorson component life remaining), or otherwise highlight or make obvious the type of component failure and, in some embodiments, the corrective action to make the repair.

[0068] Turning now to FIG. 5, a flow chart is depicted of an example method that may be performed by the remote computing device 300 to train a neural network to predict which components of an elevator are damaged. At step 500, the remote computing device 300 receives training data. In particular, the vibration data reception module 310 may receive vibration data from a plurality of training data sets and the TOF data reception module 312 may receive time-of- flight data from a plurality of training data sets. Each training data set may be associated with an opening or closing operation of elevator doors. Each training data set may also include ground truth data indicating which, if any, components of the elevator are damaged during opening or closing of the elevator doors.

[0069] At step 502, the moving phase determination module 316 determines moving phases for each set of training data. In particular, for each set of training data, the moving phase determination module 316 may determine when the time-of-flight data received by the TOF data reception module 312 begins to increase or decrease and stops increasing or decreasing, and may identify each such period of time as a moving phase.

[0070] At step 504, the vibration data trimming module 318 generates modified vibration data. In particular, for each set of training data, the vibration data trimming module 318 may remove vibration data associated with nonmoving phases.

[0071] At step 506, the feature extraction module 320 extract features from the modified vibration data. In particular, for each set of training data, the feature extraction module 320 may determine a plurality of Cepstral coefficients of the modified vibration data. In other examples, the feature extraction module 320 may extract other types of features from the modified vibration data.

[0072] At step 508, the neural network training module 322 trains a neural network to predict which components of an elevator are damaged based on input vibration data. In particular, the neural network training module 322 may utilize supervised learning techniques to train a neuralnetwork based on the ground truth values and the modified vibration data associated with the training data.

[0073] Turning now to FIG. 6, a flow chart is depicted of an example method that may be performed by the remote computing device 300 to identify damaged components of the elevator assembly 10 after a neural network has been trained. At step 600, the vibration data reception module 310 receives vibration data. In particular, the vibration data reception module 310 may receive vibration data from the accelerometers 200 and / 202 embedded in the elevator door panels 36A and / or 36B.

[0074] At step 602, the TOF data reception module 312 receives time-of-flight data. In particular, the TOF data reception module 312 may receive time-of-flight data from the TOF sensor 204. As discussed above, the time-of-flight data received by the TOF data reception module 312 may indicate a time-of-flight for a laser beam to travel from the TOF sensor 204 to the target 206 and back to the TOF sensor 204.

[0075] At step 604, the moving phase determination module 316 determines moving phases of the elevator door panels 36A, 36B based on the time-of-flight data received by the TOF data reception module 312. In particular, when the time-of-flight data begins to increase or decrease, the moving phase determination module 316 may identify the beginning of a moving phase, and when the time-of-flight data stops increasing or decreasing, the moving phase determination module 316 may identify the end of the moving phase.

[0076] At step 606, the vibration data trimming module 318 generates modified vibration data. In particular, the vibration data trimming module 318 removes vibration data associated with nonmoving phases determined by the moving phase determination module 316.

[0077] At step 608, the feature extraction module 320 extracts features from the modified vibration data generated by the vibration data trimming module 318. In the illustrated example, the feature extraction module 320 may perform Fourier Transforms of the modified vibration data and determine Cepstral coefficients associated with the modified vibration data. In other examples, the feature extraction module 320 may extract features using other techniques.

[0078] At step 610, the elevator damage determination module 324 identifies damaged components of the elevator assembly 10. In particular, the elevator damage determination module 324 may input the features extracted by the feature extraction module 320 into a trained neural network. As discussed above, the neural network may be trained to classify input vibration data to specify components of the elevator associated with the vibration data that may be damaged. The elevator damage determination module 324 may then identify damaged components of the elevator assembly 10 based on the output of the trained neural network.

[0079] It should now be understood that embodiments disclosed herein provide a method and system to improve pattern recognition for classification of damage in elevator components. In particular, using time-of-flight data may allow for accurate identifications of moving phases and nonmoving phases of elevator doors. The identification of moving phases and nonmoving phases may be used to generate modified vibration data that only includes vibration data collected during moving phases. The modified vibration data may then be input into a trained neural network to identify any components of the elevator associated with the vibration data that may be damaged. By using the modified vibration data that does not include any vibration data associated with nonmoving phases, the neural network may operate more accurately and efficiently.

[0080] While particular embodiments have been illustrated and described herein, it should be understood that various other changes and modifications may be made without departing from the spirit and scope of the claimed subject matter. Moreover, although various aspects of the claimed subject matter have been described herein, such aspects need not be utilized in combination. It is therefore intended that the appended claims cover all such changes and modifications that are within the scope of the claimed subject matter.

Claims

CLAIMS1. A computing device comprising one or more processors configured to: receive vibration data associated with an elevator assembly; identify one or more moving phases when at least one elevator door of the elevator assembly is moving, and one or more nonmoving phases when the at least one elevator door of the elevator assembly is not moving; generate modified vibration data comprising one or more portions of the received vibration data collected during the one or more moving phases; generate features based on the modified vibration data; input the features into a trained neural network; and identify one or more damaged components of the elevator assembly based on an output of the trained neural network.

2. The computing device of claim 1, wherein the one or more processors are further configured to transmit a warning to one or more individuals associated with the elevator assembly indicating the one or more damaged components.

3. The computing device of claim 1, wherein the one or more processors are further configured to receive the vibration data from one or more accelerometers embedded in the at least one elevator door or a door header.

4. The computing device of claim 1, wherein the one or more processors are further configured to: receive time-of-flight data from one or more sensors embedded in the at least one elevator door or a door header; and identify the one or more moving phases and the one or more nonmoving phases based on the time-of-flight data.

5. The computing device of claim 4, wherein the one or more processors are further configured to:identify a start of a moving phase when a time-of-flight of the time-of-flight data begins to increase or decrease; and identify an end of the moving phase when the time-of-flight of the time-of-flight data stops increasing or decreasing.

6. The computing device of claim 1, wherein the one or more processors are further configured to: receive first data from an elevator controller associated with the elevator assembly indicating when the elevator controller causes the at least one elevator door to open or close; and identify the one or more moving phases and the one or more nonmoving phases based on the first data.

7. The computing device of claim 1, wherein the one or more processors are further configured to: receive first data from one or more Hall effect sensors embedded in the at least one elevator door or a door header; and identify the one or more moving phases and the one or more nonmoving phases based on the first data.

8. The computing device of claim 1, wherein the one or more processors are further configured to: determine a plurality of Cepstral coefficients associated with the modified vibration data; and generate the features based on the plurality of Cepstral coefficients.

9. A method comprising: receiving vibration data associated with an elevator assembly; identifying one or more moving phases when at least one elevator door of the elevator assembly is moving, and one or more nonmoving phases when the at least one elevator door of the elevator assembly is not moving;generating modified vibration data comprising one or more portions of the received vibration data collected during the identified one or more moving phases; generating features based on the modified vibration data; inputting the generated features into a trained neural network; and identifying one or more damaged components of the elevator assembly based on an output of the trained neural network.

10. The method of claim 9, further comprising transmitting a warning to one or more individuals associated with the elevator assembly indicating the one or more damaged components.

11. The method of claim 9, further comprising receiving the vibration data from one or more accelerometers embedded in the at least one elevator door or a door header.

12. The method of claim 9, further comprising: receiving time-of-flight data from one or more sensors embedded in the at least one elevator door or door header; and identifying the one or more moving phases and the one or more nonmoving phases based on the time-of-flight data.

13. The method of claim 12, further comprising: identifying a start of a moving phase when a time-of-flight of the time-of-flight data begins to increase or decrease; and identifying an end of the moving phase when the time-of-flight of the time-of-flight data stops increasing or decreasing.

14. The method of claim 9, further comprising: receiving first data from an elevator controller associated with the elevator assembly indicating when the elevator controller causes the at least one elevator door to open or close; and identifying the one or more moving phases and the one or more nonmoving phases based on the first data.

15. The method of claim 9, further comprising: receiving first data from one or more Hall effect sensors embedded in the at least one elevator door or a door header; and identifying the one or more moving phases and the one or more nonmoving phases based on the first data.

16. The method claim 9, further comprising: determining a plurality of Cepstral coefficients associated with the modified vibration data; and generating the features based on the plurality of Cepstral coefficients.

17. A system comprising: an elevator assembly comprising an elevator car and at least one elevator door; and a computing device comprising one or more processors configured to: identify one or more moving phases when the at least one elevator door of the elevator assembly is moving, and one or more nonmoving phases when the at least one elevator door of the elevator assembly is not moving; generate modified vibration data comprising one or more portions of the received vibration data collected during the one or more moving phases; generate features based on the modified vibration data; input the features into a trained neural network; and identify one or more damaged components of the elevator assembly based on an output of the trained neural network.

18. The system of claim 17, wherein the one or more processors are further configured to: receive time-of-flight data from one or more sensors embedded in the at least one elevator door or a door header; and identify the one or more moving phases and the one or more nonmoving phases based on the time-of-flight data.

19. The system of claim 18, wherein the one or more processors are further configured to: identify a start of a moving phase when the a time-of-flight of the time-of-flight data begins to increase or decrease; and identify an end of the moving phase when the time-of-flight of the time-of-flight data stops increasing or decreasing.

20. The system of claim 17, wherein the one or more processors are further configured to: determine a plurality of Cepstral coefficients associated with the modified vibration data; and generate the features based on the plurality of Cepstral coefficients.

Images