Systems and methods for monitoring moving elements

Abstract

A system for monitoring moving elements in a monitored system, comprising: an illumination device configured to periodically provide momentary light to a longitudinally moving element; an image sensor configured to capture at least one image of at least a portion of the moving element; and one or more processors configured to acquire one or more images of the moving element under illumination emitted by the illumination device, analyze the images to determine whether one or more failure modes are present in the captured portion of the moving element, and register the images with one or more other images to acquire further information regarding the at least one failure mode.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of Provisional Patent Application No. 63 / 394,150, filed August 1, 2022, entitled "Systems and Methods for Monitoring Cables Integrity and Detecting Potential Faults Thereof," and Provisional Patent Application No. 63 / 521,140, ​​filed June 15, 2023, entitled "System and Method for Monitoring Longitudinal Moving Elements," which are hereby incorporated by reference in their entireties for all purposes without creating any negligence.

[0002] FIELD OF THE DISCLOSURE The present disclosure relates generally to automated systems and methods for monitoring moving elements, and more particularly, for monitoring such elements with relative motion. [Background technology]

[0003] Machinery maintenance is an important part of the operation of any plant or facility that uses machinery or other systems and is necessary to reduce the risk of accidents and injuries, minimize downtime of the system or its components, and adhere to schedules.

[0004] Machine maintenance includes regularly scheduled service visits, routine inspections, and planned or emergency repairs. Part of maintenance may include replacing, repairing, or reconditioning parts that are worn, damaged, out of place, etc., or that are expected to become so between scheduled visits.

[0005] Currently, most maintenance work is performed at predetermined intervals, which on the one hand can be wasteful as fully functional units may be replaced simply because protocol dictates it, but on the other hand other problems may go unnoticed and progress or cause damage before the next scheduled maintenance visit.

[0006] In the field of monitoring systems and their elements, a particularly difficult challenge relates to analyzing the health of moving objects, especially detecting anomalies that do not affect the object's motion. Object movement makes it difficult for humans to fully inspect the object, especially if the object is complex, has multiple components, and moves at high speed. Furthermore, movement makes it difficult to capture a clear image of the object that can be analyzed offline by a human user or a computerized system. Meanwhile, monitoring an element only when it is stationary may make it impossible to detect certain problems, may interfere with the normal operation of devices to which the element is connected, and may not ensure that all areas and segments of the element are monitored as needed.

[0007] Also, in some implementations, the moving object is located in a hard-to-reach location.

[0008] A particular type of such object is a longitudinally moving element that is generally assumed to move relative to its long axis, e.g., a cable, strap, rope, etc., with one or more ends connected to another object. Some examples include elevators, conveyor belts, escalators, helicopter cable systems, (flight) control cables, load-handling winches, etc. Such elements may move horizontally, vertically, or diagonally.

[0009] Another particular type of such element is an element that moves in a radial movement around a center, such as a fan.

[0010] Another particular type of such element is a cable wound onto a drum, the winding mechanism of which may also require monitoring and maintenance.

[0011] Some of the problems or failures detected in such elements, such as tears or stretches, are common to multiple element types, while others are specific to one or more element types. Summary of the Invention

[0012] One exemplary embodiment of the disclosed subject matter is a system for monitoring moving elements in a monitored system, the system comprising: an illumination device configured to periodically provide momentary light to a longitudinally moving element; an image sensor configured to capture one or more images of at least a portion of the moving element and comprising a rolling shutter; and at least one processor configured to acquire one or more images of the moving element in motion under illumination emitted by the illumination device configured to begin illuminating the moving object after all rows or columns of the image sensor are open and to end illumination before any of the rows or columns are closed; analyze the images; and determine whether one or more failure modes are present in the portion of the captured moving element.

[0013] Another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving element in a monitored system, the system comprising: an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture one or more images of at least a portion of the moving element; and at least one processor configured to acquire one or more images of the moving element in motion under the illumination emitted by the illumination device, analyze one or more of the images to determine whether at least one failure mode is present in the captured portion of the moving element, and register the analyzed image with one or more other images to obtain further information regarding the failure mode. In the system, the acquisition and analysis are optionally performed iteratively. In the system, the acquisition and analysis are optionally repeated until images of all portions of the moving element have been acquired and analyzed within a predefined time frame. The system may further comprise determining a next portion of the moving element to be captured. In the system, the registration optionally comprises registering a first image with a second image showing a second area adjacent to the area captured in the first image. In this system, the registration optionally comprises registering an image with a predetermined image to determine a location shown in the image. The system may further comprise analyzing at least one trend of a failure mode at the location over time. The system may further comprise providing an output indicating whether a failure mode is identified in the moving element. In this system, the output is optionally provided by updating a database, sending a message, displaying a message on a display device, or issuing an audio alert. The system may further comprise obtaining an indication of whether the moving element is moving relative to the monitored system, and performing the obtaining, analyzing, and registering conditionally if the moving element is moving. In the system, the image sensor operates with a rolling shutter.In the system, the lighting device is optionally configured to start illuminating the moving object after sensors in all rows or columns of the image sensor have opened, and to end illumination before any of the rows or columns have closed. In the system, optionally, the moving element is a longitudinally moving element. In the system, the moving element is optionally a rotationally moving element. In the system, the moving element is optionally a cable. In the system, the monitored system is optionally a system selected from the group consisting of an elevator, a helicopter, a crane, a conveyor belt, a turbofan, an aerial tram, a cable car, and a cable winder.

[0014] Yet another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving element in a monitored system, the system comprising: a motion sensor for determining whether the moving element is moving; an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture one or more images of at least a portion of the moving element; and one or more processors configured to obtain an indication of whether the moving element is moving relative to the monitored system, operate the illumination device to emit light at predetermined times if the moving element is moving, obtain one or more images of the moving element under the illumination emitted by the illumination device, and analyze the images to determine whether one or more failure modes are present in the moving element. In the system, the illumination device is optionally timed to activate and deactivate in up to 10 milliseconds.

[0015] Yet another exemplary embodiment of the disclosed subject matter is a system for monitoring a rescue hoist cable descending from a helicopter, the system comprising: an illumination device configured to periodically provide momentary light to a drum on which the rescue hoist cable is wound; an image sensor configured to capture one or more images of the drum; and one or more processors configured to operate the illumination device to emit light at predetermined times, obtain one or more images of at least a portion of the drum when the illumination device is operating, and analyze the images of the drum to determine if one or more failure modes exist in the moving element. In the system, the failure mode is optionally a failure of the cable. In the system, the failure mode is optionally a failure of a winding mechanism configured to wind the cable onto the drum. In the system, analyzing the images optionally comprises determining that a failure mode exists if the windings are of different widths. In the system, analyzing the images optionally comprises determining that a failure mode exists if the lines separating the widths are not parallel. In the system, analyzing the image optionally comprises determining that a failure mode exists provided that the lines separating the cable strands are not parallel.

[0016] Yet another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving element in a monitored system, the system comprising: an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture one or more images of at least a portion of the moving element; and one or more processors configured to obtain images of the moving element moving under the illumination emitted by the illumination device and analyze the images to determine whether one or more failure modes are present in the moving element, wherein the image sensor is configured to capture the same position within the moving element throughout an exposure time used to capture one image. In the system, the image sensor is optionally moved in accordance with the moving object throughout the exposure time. The system may further comprise a mirror configured to be moved in accordance with the moving object throughout the image capture time to reflect the same position within the moving element throughout the capture time to the image sensor. In the system, the image sensor optionally captures different images of different portions of the moving element.

[0017]

[0010] Yet another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving element in a monitored system, the system comprising: an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture one or more images of the moving element; and one or more processors configured to obtain an indication of whether the moving element is moving relative to the monitored system, to obtain one or more images of the moving element under illumination emitted by the illumination device if the moving element is moving, and to analyze the images to determine whether one or more of a set of predetermined failure modes are present in the moving element. In the system, the moving element is optionally a longitudinally moving element. In the system according to any of the preceding claims, the processor(s) are optionally further configured to inspect the images to determine whether any of them comprise a change relative to a previously captured image, to perform said analysis if a change is detected, and to inhibit any action related to the change if it determines that none of the set of predetermined failure modes is present in the longitudinally moving element. In the system, the processor(s) are optionally further configured to add images and corresponding labels to a training set of the prediction engine. In the system, on condition that the longitudinally moving element is moving, the processor(s) are optionally configured to upload images to a remote storage device, and analysis of the images is optionally performed by a second computing platform having access to the images uploaded to the remote storage device. In the system, the moving element is optionally a cable. In the system, the set of predetermined failure modes optionally comprises corrosion of the moving element. In the system, the set of predetermined failure modes optionally comprises localized damage to the cable. In the system, the set of predetermined failure modes optionally comprises global damage to the cable. In the system, the set of predetermined failure modes optionally comprises lubrication decay of the cable.In the system, the moving element is optionally a cable having one end connected to the elevator chamber and the other end connected to the counterweight. The system may further comprise a communication device for receiving information regarding the position of the moving element from a controller of the device comprising the moving element. In the system, the processor(s) are optionally further configured to determine the position of the moving element by identifying and monitoring over time at least one mark on the moving element. In the system, the processor is optionally further configured to activate and operate the lighting device and the static image sensor when the longitudinal moving element is in a required position. The system is optionally installed in an engine compartment of the device comprising the longitudinal moving element. The system is optionally installed statically with respect to the monitoring system. The system is optionally installed in a shaft along which the longitudinal moving element is located. In the system, analyzing the image optionally comprises segmenting the image to detect at least a portion of the moving element. In the system, determining whether one of the predetermined failure modes exists is optionally performed using a machine learning engine. In the system, the processor(s) are optionally further configured to apply a model to analyze the severity of at least one of a set of predetermined failure modes. In the system, the processor(s) are optionally further configured to apply a model to determine trends associated with at least one of a set of predetermined failure modes. In the system, the processor(s) are optionally further configured to analyze the plurality of images over time to detect trends in the failure modes. In the system, the processor(s) are optionally further configured to take an action, the action comprising one or more items selected from the group consisting of sending a report, sending a message to a person in charge, shutting down the monitored system, and scheduling a technician visit.In the system, the report optionally comprises one or more items selected from the group consisting of: the detected condition, an indication of the position of the longitudinal moving element, the fault that escalated into a failure, the severity of the detected fault, the severity of the detected failure mode, a timestamp, a recommendation related to maintenance of the monitored system, and a recommendation to schedule a technician visit. In the system, the processor(s) are optionally further configured to operate the image sensor to capture a plurality of images indicative of the entire length of the longitudinal moving element. In the system, the processor(s) are optionally further configured to operate the image sensor to capture a plurality of images indicative of the entire circumference of the moving element. In the system, the processor(s) are optionally further configured to operate the image sensor to capture a plurality of images indicative of the entire circumference at all positions along the entire length of the moving element.

[0018] Yet another exemplary embodiment of the disclosed subject matter is a method for monitoring a moving element in a monitored system, the method comprising: obtaining an indication of whether a moving element in the monitored system is moving relative to the monitored system; if the moving element is moving, obtaining one or more images of the moving element under illumination emitted by an illumination device; and analyzing the images to determine whether one or more of a set of predetermined failure modes are present in the moving element.

[0019] Yet another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving cable in a monitored system, the system comprising: an image sensor configured to capture one or more images of the cable; and one or more processors configured to operate the capture device to capture one or more images of the cable in synchronization with a device for evaluating which portions of the cable to monitor, and to analyze the images to determine whether the cable has one or more of a predetermined set of failure modes. In the system, the image sensor is optionally positioned statically relative to the monitored system. In the system, the image sensor is optionally positioned statically relative to the cable. In the system, the image sensor optionally comprises multiple image sensors positioned on a ring surrounding the cable, thereby capturing the entire circumference of the cable. In the system, the cable is optionally fixed at one end to an element that is static with respect to movement of the cable. In the system, the monitored system is optionally a helicopter.

[0020] Another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving longitudinally moving element in a monitored system, the system comprising: an image sensor configured to capture one or more images of the longitudinally moving element, one end of which is fixed to another element that is stationary with respect to the movement of the longitudinally moving element; and one or more processors configured to obtain an indication of whether the longitudinally moving element is moving relative to the monitored system, operate the capture device to capture one or more images of the longitudinally moving element if the longitudinally moving element is moving, and analyze the images to determine whether at least one of a set of predetermined failure modes is present in the longitudinally moving element, in synchronization with the device for evaluating which portions of the longitudinally moving element to monitor. In the system, the longitudinally moving element is optionally monitored by a capture device that is stationary with respect to the monitored system. In the system, the longitudinally moving element is optionally monitored by a capture device that moves in a plane perpendicular to the direction of travel of the longitudinally moving element. In the system, the longitudinally moving element is optionally monitored by a capture device that captures different windings of the cable.

[0021] Another exemplary embodiment of the disclosed subject matter is a system for monitoring a moving moving element in a monitored system, the system comprising: one or more image sensors configured to capture one or more images of the moving element; and one or more processors configured to: obtain an indication of whether the moving element is moving relative to the monitored system; operate the image sensor to capture two or more images of the moving element if the moving element is moving; identify a segment of the moving element in one of the two or more images; upon detecting the segment in a second image, designate an area of ​​the moving element adjacent to the segment as being monitored; analyze the images to determine whether one or more of a set of predetermined failure modes are present in the longitudinally moving element; and repeat the above steps until all outer portions of the moving element have been captured and analyzed. In the system, the images optionally overlap along a direction parallel to the direction of movement of the moving element. In the system, the images optionally overlap along a direction perpendicular to the moving element. In the system, detecting the segment in the second video frame is optionally performed by registering at least two video frames. In the system, the detection of the segment in the second video frame is optionally used to detect a rotation of the moving element. In the system, the processor(s) are optionally further configured to determine a fault or failure throughout the moving element.

[0022] The subject matter of the present disclosure will be more fully understood and appreciated from the following detailed description taken in conjunction with the drawings in which like numerals or characters indicate corresponding or similar components. Unless otherwise indicated, the drawings provide exemplary embodiments or aspects of the present disclosure and are not intended to limit the scope of the present disclosure. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a diagram of an apparatus in an exemplary environment, according to some exemplary embodiments of the present disclosure. [Figure 2] 1 is a schematic block diagram of a computing platform associated with an apparatus, according to some exemplary embodiments of the present disclosure. [Figure 3A] 1 is a flowchart of steps in a method for detecting an obstruction in a longitudinally moving element, according to some exemplary embodiments of the present disclosure. [Figure 3B] 1 is a schematic diagram of an environment in which it is necessary to calculate the blind spots of an image sensor, according to some exemplary embodiments of the present disclosure. [Figure 3C] 10A-10C are schematic diagrams of time limitations of pulsed light when a capture device with a rolling shutter is used, according to some exemplary embodiments of the present disclosure. [Figure 3D] 1 shows a schematic diagram of a first embodiment of a capture device that moves on a pulley belt, according to some exemplary embodiments of the present disclosure. [Figure 3E] FIG. 10 shows a schematic diagram of a second embodiment of a capture device that moves on a pulley belt, according to some exemplary embodiments of the present disclosure. [Figure 3F] FIG. 10 shows a schematic diagram of a third embodiment of a capture device that moves on a pulley belt, according to some exemplary embodiments of the present disclosure. [Figure 3G] 1 illustrates a flowchart of steps in an exemplary method for analyzing images for faults or failures, according to some exemplary embodiments of the present disclosure. [Figure 4] 1 is a diagram of a cable with rust stains analyzed in accordance with some exemplary embodiments of the present invention. [Figure 5] 5 is a view of the cable of FIG. 4 taken at a different time. [Figure 6] 1 is a diagram of the general structure of a cable. [Figure 7] 1 is a diagram of a cable having a structural fault, an open circuit, analyzed in accordance with some exemplary embodiments of the present invention. [Figure 8] 10A-10C are diagrams of cables having different types of structural faults that may be analyzed in accordance with certain exemplary embodiments of the present invention. [Figure 9]1A-1C are diagrams of a frame and its segments before and after processing, according to some exemplary embodiments of the present disclosure. [Figure 10] FIG. 10 is a frame diagram illustrating the cable or strap diameter reduction or slippage failure mode. [Figure 11] FIG. 1 is a schematic diagram of a side view of an elevator, according to some exemplary embodiments of the present disclosure. [Figure 12] 1 is an exemplary diagram of a system in which one end of a cable is stationary relative to a monitored system, according to some exemplary embodiments of the present disclosure. [Figure 13] FIG. 1 is a schematic diagram of a side view of a first exemplary embodiment of a system for monitoring a cable being released from a drum, in accordance with some exemplary embodiments of the present disclosure. [Figure 14] 14A and 14B are schematic illustrations of a side view and a top view, respectively, of a second exemplary embodiment of a system for monitoring a released cable, according to some exemplary embodiments of the present disclosure. [Figure 15] FIG. 10 is a schematic diagram of a top view of a third exemplary embodiment of a system for monitoring a released cable, according to some embodiments of the present disclosure. [Figure 16] 1A-1C are schematic diagrams of a drum wound with a cable having an irregularity and a method for detecting the same, according to some exemplary embodiments of the present disclosure. [Figure 17] 1 shows a schematic diagram of a monitored turbo motor, according to some exemplary embodiments of the present disclosure. [Figure 18] FIG. 1 illustrates a schematic diagram of a monitored cardan bearing, according to some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] In some embodiments of the present disclosure, the term "moving element" is broadly interpreted to refer to any element that is intended to move, such as, but not limited to, a cable, strap, rope, belt, chain, elastic bearing, or Cardan bearing. The movement can also relate to a device for holding the elements together, such as a drum for winding a cable. The movement can be generally parallel to the long axis of the element, in a closed loop, or in any other manner.

[0025] In some use cases, the element may move linearly (horizontally, vertically, diagonally, etc.) or piecewise linearly along its length and is fixed at at least one end to an element that is stationary with respect to the element's movement, such as, but not limited to, a crane, helicopter, etc.

[0026] In other use cases, the element may be attached at one end to a moving part of the monitored system, such as, but not limited to, an elevator or flight control cable.

[0027] In other use cases, elements may move in various patterns depending on the configuration and design of the system, such as circular or elliptical loops, such as pulley belts. "Pulley belt" may refer to a broad category of belts used in various mechanical systems. Some examples of belts that fall under the general term "pulley belt" include: V-belt: A belt with a trapezoidal cross section designed to fit over a V-shaped pulley. V-belts can be used in a variety of applications, including automobile engines, industrial machinery, and heating, ventilation, and air conditioning (HVAC) systems. Timing belt: A belt with internal teeth that match the teeth on a pulley. Timing belts can be used in applications that require precise synchronization of shafts or where high torque transmission is required, such as in automobile engines and robotics. · Flat belt: A belt with a rectangular cross section used in applications where a large, flat surface is required for power transmission, such as conveyor systems, printing machines, and textile machines. Serpentine belt: A flat belt with multiple grooves on one side. Serpentine belts are used in automobile engines to drive various accessories, such as alternators, power steering pumps, and air conditioning compressors. Round Belt: A belt with a circular cross section that can be used in applications requiring high flexibility and minimal vibration. Round belts are often used in conveying systems, power transmission in small appliances, and some types of delicate machinery.

[0028] In some use cases, the element is a longitudinal element. For simplicity, unless otherwise stated, according to some embodiments, the term "cable" is used interchangeably with the term "longitudinal movement element" and is not necessarily limited to a cable, but may refer to any other longitudinal movement element.

[0029] In some use cases, elements move in a radial motion around a center. In these cases, according to some embodiments, the motion can be captured and analyzed to convert the radial motion into longitudinal motion or a radial element into a longitudinal element, and such examples are included in the terms "longitudinal moving element" or "longitudinal element."

[0030] In some embodiments of the present disclosure, the term "fault" is interpreted broadly to cover any undesirable effect or process in a part of a machine that may or may not result in a failure, but requires follow-up to analyze whether repair or replacement is required or whether the monitored object exhibits an abnormal appearance or function.

[0031] In some embodiments of the present disclosure, the term "failure" is intended to be broadly interpreted to cover any problem that may occur in a part of a monitored device that renders the part unusable and / or puts the monitored device, people or objects in the vicinity of the device, etc., at risk.

[0032] In some embodiments of the present disclosure, the term "failure mode" is intended to be broadly interpreted to cover any manner in which a fault or failure may occur, such as rusting, breaking, cracking, rotating, etc. It is understood that a component may be subject to multiple failure modes related to its different features or functions. For example, a cable may rust or tear.

[0033] In some embodiments of the present disclosure, the terms "trend" or "failure mode trend" are intended to be broadly interpreted to cover any behavior of a fault or failure mode over time when or under what circumstances a fault turns into a failure. Trends are optionally associated with additional circumstances, such as environmental conditions, device usage characteristics, characteristics of a user of the device, etc.

[0034] In some embodiments of the present disclosure, the term "condition" may refer to any abnormal situation, including but not limited to faults, failures, failure modes, and trends.

[0035] In some embodiments of the present disclosure, the terms "predictive model," "prediction engine," "AI engine," or similar terms are intended to be broadly interpreted to cover any artificial intelligence (AI) engine designed to receive input comprising an image or another representation of a monitored device and provide output comprising a probability of a fault, failure, failure mode, or condition occurring given other input parameters. Prediction engines may be implemented using a variety of techniques, such as any type of artificial neural network (ANN), including, for example, deep NNs, convolutional NNs, etc. Prediction engines may also be implemented using any other machine learning techniques.

[0036] In some embodiments, the term "engine" may relate to the "AI engine" described above, and may also relate to other engines, such as an image analysis engine, a video analysis engine, a statistical calculation engine, etc.

[0037] One technical problem addressed by the disclosed subject matter relates to the need to check the health of moving elements in a monitored system, regardless of whether the moving elements move at high speeds, whether the elements are longitudinally moving elements, radially moving elements, etc. For example, cables, in particular, are one of the most critical components of an elevator system, and their health is crucial to elevator safety. Therefore, problems with elevator or escalator cables need to be detected and monitored as early as possible so that the cables can be properly repaired or replaced before any damage occurs. In another example, the health of a helicopter rescue cable and associated retraction mechanism is also a critical component, and it is important to detect failures as early as possible. Meanwhile, recognizing a dangerous system condition can lead to overly frequent scheduling of technician visits and part replacements, resulting in the inconvenience of elevator downtime and unnecessary repair or replacement costs. Therefore, there is a need for automatic and continuous monitoring of the health of critical components, such as longitudinally moving elements. The term "continuous" according to some embodiments may refer to continuous monitoring, periodic monitoring sessions scheduled over time, ad hoc monitoring sessions over time, and the like.

[0038] Another technical problem with some embodiments of the present disclosure relates to the need to monitor the entire length of the moving elements to avoid situations where failure modes manifest or occur in areas that are difficult to reach or are not regularly monitored. In some situations, longitudinally moving elements also need to be monitored from multiple directions because only a portion of their perimeter is visible from any one viewpoint.

[0039] Another technical problem of the present disclosure relates to the need to inspect moving elements in motion. This need arises because some failure modes can only be detected when the components, particularly cables or belts, are moving and cannot be detected in a stationary position. Furthermore, if the system needs to be immobilized for inspection, which may result in system shutdown, this may be inconvenient for the system user. Therefore, monitoring may need to be performed during normal system operation, not limited to times when the system is stationary. However, in some cases, inspection should or can be performed when the longitudinally moving element is stationary, such as an elevator in an office building at night, when the system is normally idle. Particular difficulties arise when monitoring elements that move at high speeds, whether linear, radial, or other, because high speeds cause the element to move a significant distance during the exposure time, making it difficult to obtain a clear image that can be analyzed.

[0040] Another technical problem of the present disclosure relates to the need to detect various types of failure modes depending on the particular element and type of element being monitored. Some failure modes, such as tears, rotations, or local or global deformations, may be common to various types of elements, while others may be specific to one or more types of elements. For example, linearly moving elements may suffer from corrosion if made of aluminum, rust and lubrication issues if made of steel, patina may develop on metals such as copper and brass, straps may suffer from tears or creases, belts used in loops may suffer from stretching, etc. Furthermore, while some failure modes are known, others may be recognized later and added to the list of detected failure modes, for example, by learning from other systems.

[0041] Yet another technical problem of the present disclosure relates to detecting failure modes caused by interrelationships between monitored elements, such as deviations from required distances between longitudinally moving elements, uneven extension, rotation, and the like.

[0042] Yet another technical problem of the present disclosure relates to detecting trends in failure modes, for example, a small stain of rust or corrosion on a cable may not jeopardize the system, but the spread of rust or corrosion that may reach unacceptable levels when considered over the entire length and / or circumference of the cable may pose such a hazard and should be notified. In other examples, the penetration depth of corrosion may also be considered in addition to the spread.

[0043] One technical solution comprises a method and apparatus for automatically monitoring moving cables and other longitudinally moving elements. The apparatus may comprise at least an illumination device and an image capture device, also referred to as an image sensor, and the illumination device may be configured to illuminate at least a portion of the field of view of the capture device.

[0044] A solution may comprise sensing whether a monitored system including moving elements is in motion. This may be done in a variety of ways, such as, but not limited to, receiving an indication from a controller, receiving an output from a sensor such as a motion sensor, a vibration sensor, or a magnetic sensor, or receiving one or more images, e.g., a series of video frames, of any moving parts of the system from an image capture device and determining whether the system is in motion by observing blurring of the images.

[0045] Once the system is determined to be moving, the lighting device may be activated in synchronization with an image capture device, such as, but not limited to, a video camera capturing one or more sections of the cable. Images are thus captured in sufficient light, are high resolution, and can be analyzed. The capture device may be near or within the field of view of the longitudinal movement element or a portion thereof. In further embodiments, the capture device may be fixed to or relative to the longitudinal movement element. The lighting device may operate in a strobe manner according to the exposure period of the capture device.

[0046] It will be appreciated that in order to be operable for illumination during the exposure time of a frame, the capturing of the illumination device may be configured to be activated and deactivated in a short period of time, such as less than 1 millisecond, less than 10 milliseconds, less than 20 milliseconds, etc.

[0047] In some embodiments, the capture device may operate with a rolling shutter, and in such embodiments, the strobe light may be configured to operate when all rows of image sensors in the capture device are open, i.e., after the last row opens and before the first row closes, so that the entire image is captured under illumination.

[0048] In some embodiments, the capture device may continuously capture moving elements with or without synchronized illumination. However, if the monitored system is not moving or the illumination is not activated, the video may be discarded after a predetermined period of time; only if the illumination device is activated may the video be stored in non-volatile memory and analyzed. In some embodiments, the apparatus may include one or more processors for analyzing the video images. Optionally, frames captured under illumination may be uploaded or otherwise transmitted to another, optionally remote, computing platform for analysis.

[0049] The images may be analyzed to determine one or more failure modes of the moving element. In some embodiments, each failure mode may be determined individually or by a different engine, e.g., an image analysis engine, a classifier, a change detection engine, an AI engine, etc., so that additional engines can be added as new failure modes are recognized. In some embodiments, the engines may operate in parallel, serially, etc.

[0050] In some embodiments, any type of cable may suffer from a structural failure mode, such as breakage of one or more wires, strands, or core. Additional failure modes may be related to changes in cable pattern, stretching, etc. Additionally or alternatively, certain types of cables may suffer from particular failure modes. For example, metal cables may suffer from rust, metallic or other lubricated cables may suffer from lubrication failure modes, other materials may suffer from corrosion, straps or strap systems may suffer from slippage, etc. Moving elements other than cables may suffer from other failure modes, as exemplified below.

[0051] In some embodiments, cable reeling mechanisms, such as helicopter rescue hoist cables, suffer from problems that cause the cable to reel in or release irregularly, which harms its ability to be released and retrieved as needed, and may even cause damage to the cable itself.

[0052] In some embodiments, particularly when monitoring belts, particularly closed-loop belts such as pulley belts, conveyor belts, etc., the capture device may capture a portion of the belt and may be adjusted to capture the same portion of the belt throughout the exposure time. In some examples, the capture device may be positioned to move with the belt so that zero or minimal relative motion between the capture device and the belt provides a clear image. In other embodiments, the capture device may be stationary but continue to capture the same portion of the belt, for example, by a mirror that moves with the belt and reflects the same area of ​​the belt to the static capture device.

[0053] In some embodiments, the device may be in communication with a controller of the monitored system and may receive information, for example, the information may include an indication of whether the system is moving, the location of the system or part thereof, such as the floor or location within the floor where the elevator is currently located.

[0054] In other embodiments, the device may include one or more sensors, such as a magnetic sensor, a motion sensor, etc., to identify whether the monitored system is moving. In another example, the device may include a second capture device to capture an image without illumination and identify whether the image is blurred, where a blurred image may indicate movement. The determination of whether the monitored system is moving may also be made by other sensors, such as a motion sensor.

[0055] While the first option of receiving an indication from a controller may be advantageous as it may also report where the monitored system is located, the second option of a sensor, such as a capture device, motion sensor, magnetic sensor, encoder, or any other sensor, provides a standalone device that is easy to deploy and does not require integration with the monitored system.

[0056] The device may keep track of monitored locations and distribute capture and analysis so that all portions of the monitored element, e.g., the entire length of a cable, are captured and analyzed. In some embodiments, capture may follow a predetermined schedule, e.g., "capture cable when elevator is on floor 1," "capture cable when elevator is on floor 2," etc.

[0057] In some embodiments, to ensure that the entire cable is monitored, the device may identify multiple points or marks along the cable, the points or marks being placed along the cable in a manner that allows for ensuring that the cable is ultimately completely captured and analyzed. In further embodiments, complete cable capture may be ensured by image registration, as described in more detail below.

[0058] Detected faults or failures and their locations may be stored along with their respective dates and times, and optionally related information received from the controller, so that maximum information is available to analyze the situation.

[0059] The device may be installed in one or more locations near the longitudinally moving element, for example in an engine room, in an elevator shaft, along the path of a conveyor belt, or attached to a stationary part through which the moving element passes.

[0060] In some embodiments, two or more devices or at least two or more capture devices may be installed. Because a single capture device may not be able to capture the longitudinal movement element from all directions, one or more additional capture devices, or capture devices that rotate around the longitudinal movement element, may be used to capture parts hidden from the first capture device. Images from all capture devices may be analyzed to detect faults in corresponding parts of the longitudinal movement element. In some embodiments, findings from nearby locations acquired by different capture devices may be integrated. For example, different parts of the same rust stain may be shown in overlapping frames, which may indicate a more severe level of rust than if they were shown individually in each frame. Therefore, the rusted area may be monitored more frequently and a warning may be issued to the user at an early stage. In further embodiments, two or more cameras may be housed in a housing, which may be designed, for example, as an open ring, to surround the longitudinal movement element and capture its entire circumference.

[0061] In some embodiments, for example, in a particularly large elevator, there may be multiple systems of longitudinal moving elements. In such situations, multiple systems according to the present disclosure may operate. The systems may be synchronized, information related to different longitudinal moving element systems may be shared, etc.

[0062] In some embodiments, the capture device(s) or a housing comprising the same may be positioned on a movement platform such that it moves substantially parallel to the longitudinal axis of the longitudinal movement element and can capture different areas of the longitudinal movement element, for example its entire length.

[0063] In some embodiments, the capture device or a housing comprising the same may be disposed on a device that rotates around the longitudinal movement element to capture the entire circumference of the longitudinal movement element, hi some embodiments, the housing may comprise multiple capture devices that simultaneously capture the entire circumference of the longitudinal movement element.

[0064] In either option, i.e., where multiple capture devices are provided or where the capture device rotates around the element, the capture device(s) may be fixed relative to the monitored device so that they capture different parts of it as the longitudinally moving element moves, or move along the longitudinally moving element.

[0065] In embodiments in which images are captured while the moving element is stationary, an illumination mechanism may not be necessary, as the capture device or housing may move at any desired speed, and therefore may move at a speed slow enough to ensure capture of a clear image of the longitudinally moving element.

[0066] One potential technical effect of embodiments of the present disclosure is automatic, continuous monitoring of a monitored system comprising one or more moving elements. Continuous monitoring may save unnecessary technician visits, unnecessary part replacements, or other maintenance work. On the other hand, continuous or periodic monitoring, which may be more frequent than scheduled technician visits, may provide early detection of a fault or a fault in progress, so that a technician visit may be scheduled early enough before the fault endangers the monitored system or its users. The moving elements may be longitudinally moving elements, radially moving elements, closed-loop moving elements such as belts, any combination thereof, etc.

[0067] Another potential technical effect of embodiments of the present disclosure relates to detecting obstacles more accurately than a human could achieve due to the clear images made possible by the use of high resolution cameras and, if necessary, lighting.

[0068] High quality fault detection can predict failures before they occur, making them easy to replace or repair, and can also provide preventative maintenance or health monitoring of devices before further damage occurs.

[0069] On the other hand, the accurate detection made possible by identifying the required failure mode also increases the reliability of the solution by reducing the number of false alarms.

[0070] Another potential technical effect of embodiments of the present disclosure is avoiding unnecessary downtime by enabling inspection of a monitored system while it is in motion. Additionally, the methods and apparatus provide for monitoring faults that can only be detected when the monitored system is in motion, which is not always possible for a human, e.g., a technician, due to the high speed of movement of the longitudinally moving element or the inaccessible location of the element being detected during operation.

[0071] Another potential technical effect of embodiments of the present disclosure is to ensure that the entire length and circumference of a longitudinally moving element is inspected for faults, such that no section may have an undetected fault. Furthermore, faults may be considered in a more holistic manner across the entire length and circumference of the longitudinal movement. For example, multiple corrosion areas may collectively cover a significant percentage of the element, even if the size of each such area is itself below a predetermined threshold.

[0072] Another potential technical advantage of embodiments of the present disclosure is that they may be applicable to monitoring elements that move in a radial motion, such as a fan, or in a repetitive motion, such as a closed-loop belt.

[0073] Another potential technical effect of embodiments of the present disclosure is that they may be applicable to systems with a combination of radial and longitudinal motion, such as helicopter rescue hoist cables and winding mechanisms for winding the cable onto a drum, where the released cable may be monitored for faults or failures and the windings may be monitored for misalignment.

[0074] Another potential technical effect of embodiments of the present disclosure is that they are applicable to any type of moving element, such as a cable, strap, belt, fan, etc., whether moving linearly, in a closed loop, horizontally, vertically, diagonally, radially around a center, or any combination thereof, and whether the entire longitudinal moving element moves or whether one end of the longitudinal moving element is attached to a fixed point and the longitudinal moving element is wound.

[0075] Another potential technical effect of embodiments of the present disclosure is that the system may monitor for any one or more visually detectable failure modes, without being limited to a specific set of failure modes known at the time of deployment. The set of faults or failures may change over time, and different sets of failure modes may be sought in different types of monitored elements.

[0076] Another potential technical effect of embodiments of the present disclosure relates to detecting faults associated with the interrelationship between two or more longitudinal movement elements or between different portions of the same longitudinal movement element, for example, changes in distance between the longitudinal movement elements, overlaps, etc., which may result in or be caused by further faults in any of the cables or their interrelationships. Additionally, faults associated with the interrelationship between a longitudinal movement element and another element may be detected.

[0077] Another potential technical problem overcome by embodiments of the present disclosure relates to detecting trends in failures so that ongoing failures can be identified that do not require immediate attention, but that need to be monitored over time to track their progress and predict when appropriate action is needed.

[0078] For example, with reference to a particular failure mode, the failure mode recognized in one or more images may be quantified. A first threshold of the failure mode may define a failure, and a second threshold of the failure mode, which may be higher than the first threshold, may define a more severe failure or failure. The rate of progression of the failure may indicate a trend and predict when the failure may become a failure so that preventative or corrective action can be taken before a failure forms.

[0079] Referring now to FIG. 1, there is shown a diagram of the main components in a typical environment in which the present disclosure may be implemented, according to some exemplary embodiments of the present disclosure.

[0080] Longitudinal moving elements, such as cables, are used in a variety of devices, and FIG. 1 illustrates a typical elevator environment 100 in which cables are monitored. The cables are shown passing between ceiling 124 and floor 128 of a typical floor in the elevator shaft. The cables may include one or more cables, such as cable 104 and cable 108. One end of the cable may be coupled to the elevator chamber and the other to a counterweight. When the elevator goes from one floor to another, the counterweight travels the same distance in the opposite direction, and this reciprocal motion is enabled by the cables.

[0081] The device may include a sensor 118 for sensing cable movement. In some embodiments, the sensor 118 may be an optical sensor, such as a camera or video camera. The sensor 118 may include a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) sensor (or active pixel sensor), a photodetector (e.g., an IR sensor, a visible light sensor, a UV sensor), a distance measurement sensor, such as a lidar sensor, or any combination thereof. When the sensor 118 is implemented as an optical sensor, it may be intended and capable of capturing a frame stream or images of one or more sections of cables, such as the cable 108. The frames captured by the camera may be stored in a buffer in a temporary memory device. The frames may be deleted after a predetermined time, such as 10 seconds, 30 seconds, 1 minute, 5 minutes, etc.

[0082] In other embodiments, the sensor 118 may be a magnetic sensor that can sense changes in a magnetic field, a vibration sensor that can sense that the cable is moving, or any other sensor that can provide a motion indication.

[0083] The apparatus may further include an illumination device 116, such as a light-emitting diode (LED), or any other light source emitting light in the visible, infrared (IR), near-IR, UV, or any other range. The illumination device 116 may be positioned and configured to illuminate all or a portion of the field of view of the capture device 120. The illumination device 116 may be adapted to emit short strobe pulses of light so that frames captured by the capture device 120 under this illumination are sharp. In some embodiments, the capture device 120 may include one or more of a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) sensor (or active pixel sensor), a point sensor, a distributed sensor, an external sensor, an internal sensor, a through-beam sensor, a diffuse reflective sensor, a retro-reflective sensor, or any combination thereof. In some embodiments, the capture device 120 may include one or more lenses and / or fiber optic sensors. While FIG. 1 shows the illumination device 116 separately from the capture device 120, it is understood that in some embodiments, the illumination device 116 and the capture device 120 may be implemented within the same housing.

[0084] It will be understood that capture device 120 may be configured to sense light, including light in the range emitted by illumination device 116, regardless of whether the emitted light is visible to the human eye. For example, some sparks that need to be sensed by capture device 120 may be in the UV range. In some embodiments, one or more lenses of capture device 120 may include an appropriate coating for receiving the emitted light.

[0085] It will also be appreciated that if the capture device 118 includes an image sensor, it may be configured to sense relevant wavelengths that may be required to sense movement.

[0086] In some embodiments, when the sensor 118 is implemented as a capture device, one or more frames captured by the sensor 118 may be analyzed for clarity. If the frame is blurry rather than sharp, this may indicate that the monitored system, for example, an elevator, is moving and that a cable is moving.

[0087] In some embodiments, the computing platform 114 may be in communication with a controller of the monitored system, such as a controller for an elevator. The controller may provide details about the status and operation of the monitored system to the computing platform. For example, in the case of an elevator, the controller may provide information about whether the elevator is moving.

[0088] If the cable is determined to be moving, by any implementation of the sensor 118 or by communication with the controller, the lighting device 116 may be activated in synchronization with the capture device 120 to capture frames showing the segment of cable that is moving. In other embodiments, the lighting may emit a light strobe in synchronization with the capture device 120 regardless of movement of the monitored system, but if the system is not moving, the images may be discarded. If the system is moving, the frames may be analyzed and copied to a non-volatile location and analyzed for one or more fault conditions.

[0089] In some embodiments, the apparatus may include one or more computing platforms 114 located on-premise or remotely and in wired or wireless communication with the lighting devices 116 and the capturing devices 120. For example, the computing platform 114 may be connected to the capturing devices 120 and the lighting devices 116 via wires that pass through the structure 112.

[0090] In some embodiments, the apparatus may include a controller, which may or may not be part of the computing platform 114, that synchronizes the emittance of illumination pulses by the illumination device 116 with the frame rate and / or exposure time of the illumination device 116 so that light is emitted when the capture device 120 captures an image.

[0091] It is understood that the capture device 120 may operate in a global shutter or rolling shutter manner, in which the light pulses emitted by the illumination device 116 may be synchronized with a time frame in which a maximum or similar number of sensor rows are exposed.

[0092] In some embodiments, the controller of the monitored system, e.g., the elevator controller, may also provide additional information about the state of the system, e.g., in the case of an elevator, information about which floor the elevator is on, which floor the elevator is going to, whether the doors are open or closed, the speed of the elevator, etc. Information about the elevator location can ensure that entire sections of the cable length are captured and analyzed within a specific time frame, e.g., weekly, biweekly, monthly, etc.

[0093] In some embodiments, the additional information may be used as additional input to one or more of the obstacle detection engines, e.g., AI engines, adapted to detect one or more obstacles in the captured image.

[0094] In some embodiments, the location of the monitored device, such as an elevator, can be used to sample the cable and provide statistical significance to the monitoring results.

[0095] In some embodiments, the computing platform 114 may determine the shutter speed of the capture device 120 or the width and frequency of the illumination pulses of the illumination device 116 according to the speed of movement of the cable 108, as predetermined, as received from the controller, as analyzed from previously captured images, etc.

[0096] Referring now to FIG. 2, there is shown a schematic block diagram of a computing platform associated with an apparatus, according to some exemplary embodiments of the present disclosure.

[0097] The device may include a motion sensor 118 and an illumination device 116. In some embodiments, the device may also include a capture device 120, as described above.

[0098] The apparatus may include an on-premise computing platform 114. The computing platform 114 may be implemented as one or more co-located or distributed computing platforms that may be in communication with each other. In some embodiments, the output of the motion sensor 118, e.g., a movement indication, may be provided to the computing platform 114 or a controller such that the lighting device 116 and / or the capture device 120 may be operated.

[0099] The apparatus may include additional sensors 204, such as pressure sensors, temperature sensors, infrared sensors, audio sensors, etc., to provide additional input regarding the health of the monitored system that can be combined with the analyzed detection results from the capture device to provide complete health monitoring of the machine.

[0100] The computing platform 114 may include one or more processors 224, which may be one or more central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), microprocessors, electronic circuits, integrated circuits (ICs), etc. The processor 224 may be configured to provide necessary functionality, for example, by loading into memory and activating modules stored on a storage device 236, described in more detail below.

[0101] The computing platform 114 may also include input / output (I / O) devices 228, such as a display, a pointing device, a keyboard, a touch screen, general-purpose input / output (GPIO), etc. The I / O devices 228 may be used to receive input from a user, such as a set of device parameters, provide output to a user, receive indications that the device is operating, receive indications for detected faults, etc.

[0102] The computing platform 114 may include a communication device 232 for communicating with other devices or computing platforms, such as a controller of a monitored system, a remote computing platform, etc., via any communication channel, such as a cellular network, a wide area network, a local area network, an intranet, the Internet, etc.

[0103] The computing platform 114 may also include storage devices 236, such as hard disk drives, flash disks, random access memory (RAM), memory chips, etc. The storage devices 236 may be distributed across two or more platforms, stored in cloud storage, etc.

[0104] In some exemplary embodiments, storage device 236 may retain program code operable to cause processor(s) 224 to perform actions associated with any of the modules listed below or steps of the methods of Figure 3A or 3G below. The program code may comprise one or more executable units, such as, for example, modules, functions, libraries, standalone programs, etc., adapted to execute instructions as described in more detail below.

[0105] The storage device 236 may store a blur analysis module 240, which, if the sensor 118 is a camera, may analyze a captured image to determine whether it is clear or blurry.

[0106] The storage device 236 may store an illumination synchronization module 244 for optionally determining and synchronizing the pulse rate, pulse width, frequency, and wavelength of the illumination device 116 with the capture device 120. However, in some embodiments, these parameters may be determined during deployment such that the illumination synchronization module 244 may be omitted.

[0107] The storage device 236 may store an analysis module 248 for analyzing frames captured by the capture device 120. However, in some embodiments, analysis of these frames may be performed by another computing platform, for example, by an analysis module 276 of the remote computing platform 256, as described in more detail below. The analysis module 248 may be operable in determining which portion or portions of the longitudinal movement element are being or should be monitored. In some embodiments, the analysis module 248 may be in communication with or otherwise receive input from a controller of the monitored system. The analysis module 248 may further be operable in determining whether the entire length of the determined portion has been captured and analyzed. The analysis module 248, if present, may comprise multiple engines, each adapted to detect the occurrence of a particular failure mode of the longitudinal movement element.

[0108] The storage device 236 may include a temporary memory 252 for storing frames acquired by the capture device 120. If frames are acquired under lighting and / or when the monitored system or camera is moving, they may or may not be analyzed while in the temporary memory 252 and / or may or may not be moved or copied to another computing platform for further analysis.

[0109] Computing platform 114 may be in communication with a remote computing platform 256. Providing captured images or other information to, and / or performing analysis on, the remote computing platform may be advantageous in that data from multiple systems associated with one or more users may be collected and used for training or other learning to improve the performance of the multiple systems.

[0110] The remote computing platform 256 may include one or more processors 260, I / O devices 264, communication devices 268, and storage devices 272 similar to the processors 224, I / O devices 228, communication devices 232, and storage devices 236 of the computing platform 114.

[0111] Storage device 272 may store analysis module 276. Analysis module 276 may perform some of the analysis detailed in connection with analysis module 248. Analysis module 276 may include one or more engines adapted to detect the occurrence of a particular type of fault within one or more frames.

[0112] Analysis module 276 and / or analysis module 248 may be operable in determining what portion of the monitored longitudinally moving element has been captured and analyzed.

[0113] In some embodiments, determining the portion may comprise first determining a global position that needs to be captured or is being captured, e.g., a portion of a longitudinal motion element that is on the opposite side of the system when the elevator is at a particular floor or between two particular consecutive floors. Such information may be received, for example, from a controller of the elevator.

[0114] Then, at a more local level, frames captured when the elevator is at that position can be analyzed to ensure that the entire length of that portion of the longitudinal moving element has been monitored. Determining the portion of the longitudinal moving element may be done by recognizing specific marks on the longitudinal moving element and identifying the marks in additional frames, the marks being spaced so as to determine which portion of the longitudinal moving element has been captured. In a non-limiting example, for a given speed and frame rate of the longitudinal moving element, if the marks are spaced so that each frame contains at least one mark and / or two marks are visible in one frame, it can be determined whether a portion of the longitudinal element has been fully captured. The marks may be color marks, special patterns of wire, additional wires, or any other visual or structural marks. It is understood that marks may also be provided around the periphery of the element to ensure that the entire circumference of the element is captured and analyzed. It is understood that marks may also be provided and analyzed to monitor for specific anomalies. In some embodiments, the marks may be used to determine that the entire length of the cable has been monitored without receiving specific information, such as from the elevator controller.

[0115] In some embodiments, a registration process between consecutive images, either directly or indirectly, can be applied to determine whether the entire length of the moving element has been captured and to track issues, such as rust development, in a particular segment. In some embodiments, the longitudinally moving element may be unmarked, and coverage can be determined by identifying segments of the longitudinally moving element. Registration can include analyzing frames and identifying one or more small segments of the element, such as a 6x6 or 12x12 segment. In some embodiments, a hash value can be calculated for each such segment and associated with the segment. The segment can then be searched for in additional frames by comparing hash values. In some embodiments, registration can be facilitated by known elevator position, direction of movement, longitudinally moving element speed, and frame rate. Thus, registration and section tracking indicate the section of the longitudinally moving element being inspected.

[0116] In some embodiments, registration may refer to identifying an object or location in an image based on a comparison to a predefined object or location, rather than a comparison to a previous or subsequent image. Identifying a location or object in an image by comparison to a predefined object or location may allow verification that the object and optionally its surroundings can be captured and analyzed.

[0117] By collecting information regarding inspected sections of the longitudinal movement element, it is possible to keep track of which portions of the longitudinal movement element have been inspected, thereby ensuring that, for example, the entire length of the longitudinal movement element has been inspected within a given period of time. Additionally or alternatively, by keeping track of specific segments, it is possible to monitor the segments over time for specific faults or potential faults, predict their condition, and ensure that they are addressed when the time comes.

[0118] The storage device 272 may store an action / report generation module 280 for generating reports related to one or more frames, frames analyzed over a period of time, etc. If one or more faults are detected or suspected, the action / report generation module 280 may take action, such as sending a message or report to personnel, shutting down the monitored system if severe, automatically scheduling a technician visit, etc. The report may relate to the detected fault, to a fault trend indicating the rate at which the fault is progressing if maintenance is suggested, to indicate the location of the fault, to indicate which longitudinal movement elements need to be replaced, etc. The report may include one or more images illustrating the fault or failure mode. The report may comprise a failure trend, i.e., a prediction of when the fault will escalate into a failure, based on the fault or failure mode, previous analysis or inputs, and optionally environmental parameters.

[0119] The storage device 272 may include non-volatile memory 284 for storing frames received from the computing platform 114 and analyzed for fault detection.

[0120] Referring now to FIG. 3A, there is shown a general flowchart of steps in a method for detecting faults in longitudinal elements, according to some exemplary embodiments of the present disclosure.

[0121] In step 304, the processor may determine which section of the longitudinal element, e.g., cable, to capture and monitor. The portion to be captured may be determined randomly and may wait until the monitored system is in the required position to allow capture. In other embodiments, once the monitored system reaches a location, it may be determined whether this location needs to be monitored. In further embodiments, the monitored locations may be determined according to a schedule intended to cover all portions of the longitudinally moving element within a predetermined duration. It is understood that there may be additional methods for determining the section to be monitored, as well as combinations of such methods. In further embodiments, monitoring may be continuous, time-based, e.g., only during business hours in an office building, etc.

[0122] In some embodiments, the entire length of a moving element needs to be monitored within a predetermined time frame. In a non-limiting example, technician visits to an elevator may be scheduled monthly, and inspecting each section of the elevator cable twice between any two such visits may be sufficient. In another non-limiting example, technician visits are scheduled only upon call, so that it may be advantageous to call a technician only if a fault or failure is detected. Thus, for example, if a particular section is determined to be inspected, a second inspection of that section may be skipped if it has been inspected twice since the last technician visit. At a predetermined time, e.g., more than halfway to the next technician visit or a predetermined time before the next technician visit, it may be determined which portions of the elevator cable have not been adequately monitored, and monitoring of those sections may be intentionally targeted. For example, the next time the monitored system is in a suitable position to monitor such segments, monitoring may be activated. Thus, it may be ensured that all portions of the moving element have been captured and analyzed within a specific time frame.

[0123] In some embodiments, a collection of sections requiring inspection may be maintained, and if the monitored system is in a position where such a section can be monitored, the section may be captured, analyzed, and removed from the collection.

[0124] In step 308, it may be determined whether the monitored system is moving or stationary. In some embodiments, the indication may be received from a controller of the monitored system, such as, for example, an elevator controller. Optionally, the controller may provide an indication of the location of another part of the monitored system, such as, for example, an elevator chamber, from which the section of cable to be captured may be derived. In some embodiments, the indication may comprise a section of cable that is within the field of view of a stationary camera.

[0125] In some embodiments, for example when the device is standalone and does not receive information from a controller or another source, the determination of whether the monitored system is moving may be made by a motion sensor. An example of a motion sensor may be the use of a camera, such as:

[0126] In step 312, an image may be taken of the portion of the longitudinally moving element that is within the field of view of the capture device. The capture device may be a still camera, a video camera, etc. Optionally, the camera is the same as the capture device used for motion detection.

[0127] In step 316, the captured image may be analyzed to determine whether the image is blurred. In some embodiments, blur may be assessed by the “sharpness” of the image. In some embodiments, blur may be assessed by a trained engine, such as a classifier. The classifier may be trained based on a plurality of blurry images and a plurality of clear images, each associated with a corresponding label. When the image is provided to the engine, the engine may output whether the image is blurry or clear. In some embodiments, the engine may receive a label comprising a blurriness degree and may output a prediction of such degree, for example, between 0 and 100. If the image is blurry, for example, if the blurriness degree exceeds a predetermined threshold, it may be inferred that the surveillance system is moving, and vice versa.

[0128] If the surveillance system is determined to be stationary and not moving, the image may be deleted in step 318 and capture may continue in step 302 and acquisition of motion indications may continue in step 308 .

[0129] In some embodiments, in addition to the image captured under illumination by the capture device, a clear image captured by a second camera may be analyzed, as described below in connection with step 336.

[0130] However, if the system is running, then in step 320 the illumination pulse duration may be determined if not set during system deployment or as needed if a change occurs.

[0131] In some embodiments, the illumination pulse duration may be predetermined, for example, at the time of deployment, based on the position of the capture device relative to the longitudinal movement element, the exposure time of the capture device's shutter, the velocity of the longitudinal movement element if constant over a relatively long period of time, etc. The illumination pulse duration may be predetermined to be as long as possible and shorter than the exposure time of the capture device's shutter.

[0132] In another embodiment, the illumination pulse duration may be determined as follows.

[0133] In step 324, the velocity of the monitored system, in particular the velocity of the longitudinally moving element during movement, may be obtained, for example, from a controller. In a further embodiment, the velocity may be obtained from the difference in the position of the identified section, mark, object, or artifact appearing in two or more consecutive frames.

[0134] In step 328, the velocity may be used to determine the illumination pulse duration and / or exposure time of the capture device shutter, as described in more detail below in connection with FIG. 3C.

[0135] Once the capture parameters have been determined, in step 332 the illumination device and / or capture device may be configured accordingly and synchronized with each other.

[0136] It will be appreciated that in some embodiments, step 302 occurs continuously, with the video camera continuing to capture frames whether or not the lighting device is active.

[0137] Thus, when the lighting device is active and a clearer image is captured, an image may be taken during the lighting pulse in step 336. Capturing may include receiving one or more signals from the capture device. According to some embodiments, the one or more signals may include one or more images. According to some embodiments, the one or more signals may include one or more portions of an image. According to some embodiments, the one or more signals may include a set of images, such as, for example, a packet of images. According to some embodiments, the one or more signals may include one or more videos.

[0138] In some embodiments, one or more images, or portions thereof, may be transmitted to a non-volatile memory device, which may be part of or associated with the computing platform, or may be remote, e.g., an on-premise server, a remote server, cloud storage, etc.

[0139] In step 340, the image may be analyzed for faults of one or more failure modes, e.g., by an on-premise computing platform, a remote computing platform, etc. The analysis may include checking for the presence of multiple faults, as described in more detail below.

[0140] Analysis of the images may also comprise determining that no portion of the monitored section remains unanalyzed. In some embodiments, the system may detect marks on the longitudinal movement element to ensure that the entire length of the longitudinal movement element has been inspected by ensuring that all marks have been captured and each image analyzed.

[0141] In other embodiments, one or more segments of the longitudinally moving element may be identified and searched for in successive frames to ensure continuity of inspection. The search may comprise registration, for example, by assigning a hash value to one or more segments in an image and searching for other areas with the same hash value in one or more successive frames.

[0142] Registration with images showing overlapping areas can provide a more comprehensive view of the failure mode, for example, determining how far the failure mode extends beyond the current image. Additionally or alternatively, registration can determine the exact location of the capture area for comparison with past or future images to recognize trends in the failure mode.

[0143] In a further embodiment, a statistical model may be used to ensure that each point on the cable is monitored at least once in a predefined period. The relationship between cable speed, pulsed light frequency, and the number of monitoring iterations that must be performed to ensure monitoring of the entire length of the cable may be presented as follows: L f =2 * D * tg(FOV / 2) ·D indicates the distance from the cable to the acquisition sensor. FOV indicates the camera's field of view. L f denotes the length of the cable captured in a single frame.

[0144] For example, assume that the pulsed light is of very short duration and simulates a fixed cable (i.e., the cable does not move during illumination) with a camera having a 70° FOV located 100 mm from the cable. The equation is: L f =2 * 100 * tg(35°)=140mm which means that a 140 mm section of the cable is captured during one frame. To calculate the distance the cable travels during the duration of the pulse of light, the following formula can be used: P=1 / F * V B a =PL f During the ceremony, ·V indicates the cable speed. ·F denotes the pulsed light frequency, which is equal to the frames per second (FPS) rate of the optical sensor. ·P indicates the distance the cable travels during one frame cycle time. ·B a denotes the blind spot, i.e. the length of cable that is not captured between two successive light pulses.

[0145] Faults, failures, failure modes, and trends may be output to a user, for example, by updating a database, sending a message, displaying a message on a display device, issuing an audio alert, etc. An output may also be provided indicating that the monitored element is operational and no problems have been detected.

[0146] Referring now to FIG. 3B, there is shown a schematic diagram of an environment in which it is necessary to calculate the blind spot of a sensor in order to determine the time required to ensure that all portions of a cable are captured and analyzed.

[0147] The environment includes an optical sensor 350 located at a distance D from a cable 351. The cable travels a distance P in a single cycle of the sensor 350 and has a length L f A portion of the cable having is captured in a single frame. According to some embodiments of the present disclosure, the FOV, calculated FOV, and blind spots (BI) are shown.

[0148] For example, a 30FPS camera with a 140mm L f , and for a cable speed of 10 m / s, B a =1 / 30 * 10,000-140=193mm is.

[0149] It will be appreciated that the smaller the blind spot between frames, the fewer iterations that need to be made to ensure that the entire length of the cable has been captured with at least a given level of confidence.

[0150] To reduce the length of the blind spot (thereby reducing the number of repetitions), the pulsed light frequency or pulse width needs to be increased. It is understood that the pulse width can be increased within the limits imposed by the cable speed, as excessively wide pulses will produce blurry images. Additionally or alternatively, blind spots can be reduced by increasing the area captured in the frame, which can be achieved by increasing the FOV or increasing the camera's distance from the cable. However, it is understood that this can have a negative impact on resolution, so a trade-off must be considered.

[0151] To calculate the pulse width, the following factors are considered or combined with some other factors: cable speed, sensor resolution, and FOV.

[0152] Contrary to the simple assumption above, when using illumination pulses, the object to be captured moves during the illumination period. To overcome the motion blur effect, it is necessary to determine how much movement the object is allowed to have. Simply put, the object may move within a single pixel during the illumination period.

[0153] To do this, we need to calculate the actual pixel size. For example, the FOV of a sensor with 200x200 resolution is such that it allows capturing an area of ​​1mx1m size, and since the movement is generally along one direction, it is considered to be 1m. Therefore, the minimum distance an object can move is 1m / 200 = 0.005m. If the cable moves at a speed of 30m / s, the pulse time can be calculated as 0.005m / 30m / s = 166 microseconds.

[0154] The above calculations can be combined into a single formula. Pulse length(s) = (FOV(m) / Resolution(pix) / Cable speed(m)

[0155] However, this formula is applicable to single-pixel moving objects, such as defects up to one pixel in size. In machine vision, to ensure accurate and reliable detection of objects of interest, such as defects, the sensor resolution may be designed so that the object size is at least three pixels. Thus, the object has a distance movement of two pixels during the illumination period. The formula may therefore be adjusted as follows: Pulse length(s) = (FOV(m) / resolution(pix) / 2)) / cable speed(m)

[0156] In the current example, 1(m) / (200 / 2)=0.01 m, and 0.01 m / 30(m / s)=333 microseconds. Thus, if the system is as described above, a pulse width of 333 microseconds will ensure that an object, e.g., a defect, is contained within 3 pixels, so that motion blur effects are overcome and the defect can be analyzed.

[0157] Therefore, if the system parameters including cable speed are known, the FPS cost of the optical sensor, lighting power, and quality and duration of monitoring can be optimized.

[0158] The pulse frequency may be matched to the frame rate of the capture device, and the illumination time may be of the longest possible duration that ensures a sharp image and is less than or equal to the shutter exposure time of the capture.

[0159] It will be appreciated that the above calculations may be performed every frame, every predetermined number of frames, or periodically, for example daily, weekly, or monthly. In further embodiments, the calculations may be performed once during deployment of the system or after a change in the configuration of the device or monitored system.

[0160] In some embodiments, the optical capture device may operate in a global shutter manner, where all pixels of the sensor are opened and closed simultaneously. However, capture devices that operate in a rolling shutter manner may be preferred in various applications because they are more readily available and significantly cheaper. Because a rolling shutter can induce distortions that can result from pixels being exposed to light for different times, the processor of the capture device may execute code configured to correct for distortions in the captured image.

[0161] Referring now to FIG. 3C, a time limit for pulsed light is shown when a capture device with a rolling shutter is used, according to some exemplary embodiments of the present disclosure.

[0162] In rolling shutter technology, the first row (or column) of pixels of the sensor, e.g., the top row, is opened at time T1, then the second row (or column) of pixels is opened, and so on until the last row of pixels is opened at time T2, after which the first row of pixels is closed at time T3, then the second row of pixels is closed, and so on until the last row of pixels is closed at time T4.

[0163] Therefore, if all pixels are to be opened, the pulsed light should begin at or after T2 and end at or before T3. This restriction is combined with the above calculation of the maximum pulse width that allows for the capture of a clear image. Therefore, the selected pulse length must comply with two conditions:

[0164] In some embodiments, to obtain a clear image, the optical capture device may move to correspond with the cable movement, minimizing relative motion between them and ensuring a clear image and capturing the same area during the exposure time. Depending on the particular configuration and available space, mechanical complexes may be implemented in various ways to enable capture of fast-moving elements. It is understood that having the capture device capture the same portion of the cable throughout the frame exposure time provides a clear image and may allow for the elimination of the use of illumination, such as a strobe light. Furthermore, it is understood that a capture device moving or otherwise capturing the same portion of the cable may be associated with one or more images, provided that the common movement of the capture device or another element with the cable is intermittent, and additional images captured by the capture device are of other portions of the cable.

[0165] Referring now to FIG. 3D, shown is a schematic diagram of a first embodiment of a capture device that moves with a monitored element, such as a pulley belt, according to some exemplary embodiments of the present disclosure.

[0166] In the example of FIG. 3D , a mechanical system is positioned adjacent to belt 352 and is configured to move optical sensor 355 parallel to belt 352 at a speed essentially equal to the cable speed during at least a portion of the belt's 352 movement. The system includes two pulleys 353A / 353B (collectively referred to herein as pulleys 353) connected via cable 354 to which optical sensor 355 is attached. By way of example, the optical sensor 355 has an FPS rate of 30 and a shutter exposure time of 1 / (30×2)=1 / 60. The cable speed is known to be 5 m / s, for example, as provided by receiving data from a controller or by analyzing images received from optical sensor 355 and calculating the cable speed using image processing techniques. In this case, the calculation of how long the cable passes during one frame is 5×(1 / 60)×1000=83.3 mm. Therefore, the belt 352 passes 83.3 mm in one frame, which means the FPS rate of the optical sensor 355 is too slow to capture a cable moving at a speed of 5 m / s.

[0167] To overcome this problem, pulley 353 to which optical sensor 355 is attached moves the optical sensor parallel to belt 352 along segment 356 of belt 352 at a speed of 5 m / s, which is equal to the cable speed, thereby enabling optical sensor 355 to capture a clear image of the same segment 356 of belt 352 as it moves.

[0168] Belt 354 continues to move and optical sensor 355 stops moving parallel to belt 352, moves around pulley 353 as shown at positions 355A, 355B, and 355C, and rejoins belt 352 to capture another segment 356 thereof.

[0169] In some embodiments, when images of the belt 352 and particularly its segments 356 are captured by the moving optical sensor 355, the segments 356 may be illuminated by an illumination source, such as, for example, a strobe illumination source.

[0170] Referring now to FIG. 3E, shown is a schematic diagram of a second embodiment of a capture device that moves with a monitored element, such as a pulley belt, according to some exemplary embodiments of the present disclosure.

[0171] 3E, pulley 353A is coupled to belt 352 and rotates at the same speed as belt 352. Pulley 353A, via belt 357, rotates pulley 353B, to which capture device 355 is coupled.

[0172] Thus, as belt 352 moves, capture device 355 rotates. Capture device 355 may operate when it is at an angle to capture at least a portion of segment 359 corresponding to section 358. Alternatively, capture device 355 may operate continuously, but ignore photographs captured in other directions.

[0173] In some embodiments, when capturing an image of belt 352, particularly when capture device 355 is pointing at section 358, the corresponding segment 359 may be illuminated by an illumination source, such as, for example, a strobe illumination source.

[0174] In a further embodiment, the camera may capture the same area over the exposure period without movement, but using an optical element such as a mirror that reflects the same area.

[0175] Referring now to FIG. 3F, shown is a schematic diagram of a third embodiment of a capture device that moves with a monitored element, such as a cable, according to some exemplary embodiments of the present disclosure.

[0176] In the system shown in Figure 3F, pulley 353A is coupled to belt 352 and rotates at the same speed as belt 352. Pulley 353A rotates pulley 353B, to which mirror 360 is coupled, via cable 357.

[0177] Thus, as belt 352 moves, mirror 360 rotates. During part of its circular path, mirror 360 assumes an angle that reflects a portion of belt 352 to capture device 355. Capture device 355 may operate when mirror 360 is in this angular range. Alternatively, capture device 355 may operate continuously, but ignore photos captured when mirror 360 is in other directions.

[0178] In some embodiments, when capturing an image of belt 352, when capture device 355 captures at least a portion of belt 352, the corresponding segment of belt 352 may be illuminated by an illumination source, such as, for example, a strobe illumination source.

[0179] Referring now to FIG. 3G, a flowchart of an exemplary method for analyzing images for faults or failures is shown, according to some exemplary embodiments of the present disclosure.

[0180] Once images are received, for example, from a capture device as described above, the images may undergo a pre-processing step 362. According to some embodiments, the pre-processing step 362 may include generating one or more images, one or more sets of images, and / or one or more videos. According to some embodiments, the pre-processing step 362 may include dividing the one or more images, one or more portions of one or more images, one or more sets of images, and / or one or more videos into tiles. According to some embodiments, the pre-processing step 362 may include applying one or more filters, such as, but not limited to, a noise reduction filter, to the one or more images, one or more portions of one or more images, one or more sets of images, one or more videos, and / or the tiles.

[0181] At least one change may be identified in the received signal and / or the at least one identified segment in step 366. According to some embodiments, the method may include applying a change detection algorithm to the received image.

[0182] According to some embodiments, identifying at least one change in the image includes identifying a change in the rate of change in the image. For example, the algorithm may be configured to identify changes that occur periodically in the analyzed image of the same location, after which the analyzed signal may "return" to a previous state (e.g., before the change in the analyzed signal). According to some embodiments, the algorithm may be configured to identify a change in the rate of occurrence of the identified change.

[0183] According to some embodiments, identifying at least one change in the analyzed signal may include analyzing raw data of the received signal, an image or part thereof, or a video obtained therefrom.

[0184] According to some embodiments, change detection 366 may include one or more of binary change detection, quantitative change detection, and qualitative change detection.

[0185] According to some embodiments, binary change detection may include an algorithm configured to classify an analyzed signal or image as having or not having a change relative to a previously captured signal or image. According to some embodiments, binary change detection may include an algorithm configured to compare two or more analyzed signals. According to some embodiments, if the comparison indicates that the compared analyzed signals are the same or essentially the same, the classifier labels the analyzed signal as having no detected (or identified) change. According to some embodiments, if the comparison indicates that the compared analyzed signals are different, the classifier labels the analyzed signal as having a detected (or identified) change. According to some embodiments, two or more different analyzed signals may have at least one different pixel. According to some embodiments, two or more identical analyzed signals may have identical features and / or pixels. According to some embodiments, the algorithm is configured to set a threshold number of different pixels above which two analyzed signals may be considered different.

[0186] Advantageously, change detection 366 allows for fast detection of changes in the analyzed signal or image and can be very sensitive to even the slightest changes therein. Furthermore, detection and alerting for binary change detection can occur within a single signal, e.g., within a few milliseconds depending on the signal output rate of the optical sensor, or, in the case of an optical sensor comprising a camera, within a single image frame, e.g., within a few milliseconds depending on the frame rate of the camera.

[0187] According to some embodiments, a binary change detection algorithm may, for example, analyze the signal or image and determine if non-black pixels change to black over time, indicating a possible change in the position of the longitudinal movement element, possibly due to deformation or a change in the position of other components associated with the longitudinal movement element. According to some embodiments, the binary change detection algorithm may detect a change in the signal, and a warning signal (or alarm) may be generated to alert equipment or technicians that maintenance may be required.

[0188] According to some embodiments, binary change detection may be configured to determine the cause of an identified change using one or more machine learning models, such as model 370. For example, for a black pixel that may change to a color other than black over time (or through successive analyzed signals), machine learning model 368 may output that the change indicates a change in the material of the longitudinal movement element, for example due to overheating.

[0189] According to some embodiments, the method may include identifying at least one change in the signal by analyzing dynamic motion of the element. According to some embodiments, the dynamic motion may include any one or more of linear motion, rotational motion, cyclic (repetitive) motion, etc. According to some embodiments, the change may be damage, defects, cut size / length, cut growth rate, cut propagation, breakage, structural damage, defect diameter, cut, warping, expansion, deformation, abrasion, wear, corrosion, oxidation, sparks, smoke, fluid flow rate, droplet size, fluid volume, liquid accumulation rate, texture change, color / shade change, size of formed bubbles, droplets, puddle formation, puddle propagation, dimensional change, position change, color change, texture change, size change, appearance change, or any combination thereof.

[0190] According to some embodiments, change detection may include quantitative change detection. According to some embodiments, quantitative change detection may include determining whether a change in magnitude above a certain threshold has occurred in the signal or image being analyzed. According to some embodiments, a change in magnitude above a certain threshold may include a cumulative change in magnitude independent of time and / or a rate (or rates) of change in magnitude. For example, a value reflecting a change in magnitude may represent a number of pixels that have changed, a percentage of pixels that have changed, a total difference in the numerical values ​​of one or more pixels within the field of view (or signal being analyzed), a combination thereof, etc. According to some embodiments, the quantitative change detection algorithm may output quantitative data related to the change in the signal being analyzed.

[0191] According to some embodiments, the change detection may comprise qualitative change detection. According to some embodiments, the qualitative change detection algorithm may comprise an algorithm configured to classify the analyzed signals as indicative of a change in the longitudinal movement element. According to some embodiments, the qualitative change detection algorithm may comprise a machine learning model configured to receive the analyzed signals and classify them into at least a category that includes a change in behavior of the longitudinal movement element and a category that does not include a change in behavior of the longitudinal movement element.

[0192] According to some embodiments, the change detection algorithm may be configured to analyze other, more complex changes in the analyzed signal generated by the optical sensor, for example, by one or more machine learning models 370. According to some embodiments, the machine learning models may be trained to recognize complex and diverse changes. According to some embodiments, the machine learning models may be able to identify complex changes in the signal generated by the optical sensor, for example, where the signal may begin to exhibit some periodic instability, such that the signal may appear normal for a time and then abnormally for a time before appearing normal again. Thereafter, the signal may exhibit some anomaly similar to before, but different, and the change detection algorithm may be configured to analyze the changes and train itself over time to detect the likely cause of the instability. According to some embodiments, the change detection algorithm may be configured to generate a warning or information signal, if necessary, to notify the user of the change in the longitudinal movement element.

[0193] In step 364, the failure mode in which a change was detected may be identified. Step 366 may include obtaining data related to the characteristics of at least one failure mode of the longitudinal movement element. The data may be obtained from a computerized device, a human, or a combination thereof. According to some embodiments, the data related to the characteristics of the at least one failure mode may include a location or range of locations of the failure mode on the element and / or a particular type of failure mode.

[0194] According to some embodiments, failure modes may include any one or more of: changes in dimension, location, color, texture, size, or appearance; breakage; structural damage; laceration; laceration size; severe laceration size; laceration location; laceration propagation; specific pressure on an element; changes in the movement of one component relative to another; defect diameter; cuts; warping; expansion; deformation; abrasion; wear; corrosion; oxidation; sparks; smoke; color / texture change; or any combination thereof. It is understood that some parameters, such as dimensions, may vary according to specific circumstances. For example, dimensions may be relative to capture distance. As an example, the diameter of a helicopter cable may change due to abrasion, but also due to a birdcage.

[0195] According to some embodiments, the severity of the failure may also be determined based on the analysis.

[0196] According to some embodiments, obtaining data relating to the characteristics of at least one failure mode includes receiving input data from a user via a user interface module.

[0197] According to some embodiments, the method may include obtaining data related to characteristics of at least one failure mode of the element by identifying a previously unknown failure mode. According to some embodiments, identifying the previously unknown failure mode may include applying the received signals and / or identified segments to a machine learning model 368 configured to determine a failure mode of the longitudinal movement element. According to some embodiments, the machine learning model 360 may be trained to identify potential failure modes of the element. According to some embodiments, a single machine learning model 368 may be used to identify multiple failure modes. In other embodiments, the machine learning model 368 may include multiple models, each associated with a single failure mode.

[0198] According to some embodiments, an identified failure mode may be associated with an indicator that indicates whether the element exhibits a fault, a failure, or nothing, based on, for example, the range of change or the recognized failure mode. For example, 5%-20% rust may be indicated as a fault, while more than 20% rust may be classified as a failure.

[0199] In some embodiments, a failure trend may be assessed based on existing models and, optionally, previous indications related to the failure mode. The trend may be used to predict when an element or system will require maintenance or when a failure may occur.

[0200] In step 372, certain detected conditions, such as detected changes that are not consistent with any of the failure modes, may be suppressed from further evaluation, not considered in calculating faults and trends, and to reduce or eliminate the number of false alarms and suppress any associated action. For example, dust or flies on the longitudinal elements, blurring of some areas of the image, etc. may appear as anomalies, but analysis of the image does not provide an indication of any known failure mode, so no further examination or action is taken.

[0201] In some embodiments, the suppression step 372 may classify the detected fault as likely to escalate into a failure, as indicated by a failure mode nodal point 380. In some embodiments, the suppression step 372 may use one or more machine learning models 376. In some embodiments, the suppression may indicate the fault as benign.

[0202] If the signal or image does not indicate a fault or failure of a particular failure mode ("NO" at branch point 380), the image or a portion thereof may be added to a training set used to enhance one of the machine learning models, such as machine learning model used to identify failure modes 360, machine learning model used to identify changes 368, or machine learning model used to suppress potential faults 376, via a training step 384. However, other instances indicated as faults or failures may also be labeled accordingly and added to the model's training set to improve the model's accuracy.

[0203] If the signal or image indicates a fault or failure (“YES” at branch point 380), execution may return to step 340 of FIG. 3A.

[0204] Referring again to FIG. 3A, once the images have been analyzed and optionally registered to identify an object or portion thereof that appears in two or more images, execution may return to step 304 and a location for monitoring may be determined.

[0205] Additionally or alternatively, determining faults and monitoring them over time in step 344 may be performed to recognize trends in failures. In an exemplary embodiment, with reference to a particular failure mode, a first threshold for the failure mode may define a fault, and a second threshold for the failure mode may define a more severe fault or failure. The rate of progression of the fault may indicate a trend and predict when the fault may become a failure so that preventative or corrective action can be taken before the fault forms. The trend may be calculated according to a known model that may indicate how severe the fault will be and / or predict when maintenance is required. The prediction may use detected faults, historical data, environmental conditions such as humidity and temperature, etc. The trend may be calculated upon initial detection of a fault and updated with further detection or received data. Thus, the trend may bring the time to the next maintenance or inspection earlier than normally scheduled.

[0206] In some non-limiting examples, rust stains may be monitored by comparing images taken over time showing the rust stain to see if it grows, the size or number of structural failures may be compared, etc. For example, with reference to the particular failure mode of rust, a cumulative area of ​​5% or more of the element's surface area may be defined as a failure, and a cumulative area of ​​20% or more of the element's surface area may be defined as a failure. Based on two or more images and optionally knowledge of rust behavior and specific conditions, the time it will take for rust to spread from 5% to 20% may be predicted, and a trend for this failure mode may be determined.

[0207] In step 348, action may be taken, particularly if a fault is detected or if it is determined that a fault trend is ongoing, e.g., rust stains or corrosion are spreading to a wider area or deeper towards the cable core, tears are getting longer, etc.

[0208] According to some embodiments, the action may include generating a signal, such as an information signal or a warning signal, as appropriate. According to some embodiments, the warning signal may be a one-time signal or a continuous signal and may require some form of action, for example to reset the warning signal.

[0209] The action may be, in non-limiting examples, sending or storing a report, sending a message to a responsible person including the fault or failure, its severity and a timestamp, storing the report on a storage device, etc. In some embodiments, depending on the type of fault, it may be recommended to the user, for example, to lubricate or replace cables, check the braking system, the engine, etc.

[0210] In some cases, for example, if a fault indicated as critical is detected with a severity and / or certainty exceeding a respective threshold, the monitored system may be shut down to ensure the safety of the monitored system and its users.

[0211] As mentioned above, a failure mode may be specific to a particular type of longitudinal element or may be common to multiple types. The types of failure modes may include, but are not limited to, structural damage, cuts, defects, cuts of a predetermined size or length, cut growth rate, cut propagation, breakage, defect diameter, warping, expansion, deformation, abrasion, wear, rust, corrosion, oxidation, sparks, birdcage, texture change, color / shade change, size of bubbles formed, change in dimension of at least some of the segments, change in position of at least some of the segments, change in color of at least some of the segments, change in texture of at least some of the segments, change in size of at least some of the segments, change in appearance of at least some of the segments, linear motion of at least some of the segments, rotational motion of at least some of the segments, cyclic (repetitive) motion of at least some of the segments, change in rate of motion of at least some of the segments, lack of lubrication, excess lubrication, diameter change, signs of wear, signs of wear, improper alignment, groove issues (missing cable / rope in groove), kinks in the cable or rope, worn strands, cuts as predictive signs of wear, or any combination thereof.

[0212] Below is a more detailed reference to some types of failure modes and their detection.

[0213] FIG. 4 shows a diagram of a cable 400 having rust stains 404 and 408.

[0214] In some embodiments, rust may be detected by first identifying pixels with values ​​outside the acceptable range of values ​​for the cable. In some embodiments, a minimum size of contiguous pixels or pixels separated by at most a predetermined distance may be required to be considered rust. In some embodiments, a polygon may be generated that encompasses each set of such adjacent pixels. Texture and color-related features may then be extracted for each such polygon, including, for example, the difference or uniformity between pixel values, the derivative of pixel values ​​along a pixel sequence, the density of changes in said values, etc.

[0215] These feature values ​​may be provided to a trained engine. The engine may be trained based on features extracted from polygons identified in previously captured images, each associated with a corresponding rust / no-rust label. During training, the engine may determine linear and non-linear relationships between different features and between features and labels. Then, at prediction time, the extracted features from the captured frame(s) are input to the trained engine, which provides a prediction of whether the cable has a rust stain at a particular capture location.

[0216] In some embodiments, the depth of rust into the cable may be estimated. In some embodiments, the engine may provide a certainty of the polygon indicating rust rather than a rust / no-rust prediction. In some embodiments, the certainty may be related to the number of pixels associated with the rust stain. In such implementations, a threshold may be set so that certainty above the threshold is considered rust and certainty below the threshold is considered not rust. In some embodiments, the depth of the rust stain, i.e., how deeply it has penetrated into the cable, may also be estimated, for example, by an engine trained based on images showing different penetration levels and corresponding labels.

[0217] In some embodiments, certain ranges of confidence values ​​or other conditions, such as a combination of color features indicative of rust and texture features not indicative of rust, may be interpreted as a "red" condition indicative of a mixture of oil and dust, which may indicate a fault or failure related to wear and tear on the cable and should be reported as well.

[0218] Another type of fault may be related to cable rotation.

[0219] Referring now to FIG. 5, a cable section 400 is shown, taken at a different time than shown in FIG. 4, containing rust stains 404' and 408'.

[0220] In some embodiments, each artifact, such as rust stains 404 and 408, may be associated with an identifier, calculated, for example, as a hash value. The hash values ​​of rust appearing in different frames may be compared. Thus, rust 404' and 408' may be identified as the same as rust 404 and 408. However, because their horizontal positions in the images are different, it may be inferred that cable 400 is rotating, given that video camera 118 is stationary, which is an indicator of undesirable behavior and may be reported as a possible fault.

[0221] In some embodiments, rotation can be detected by other means, for example, color stripes can be painted on the cable. As the cable rotates, the stripes will not stay straight but will rotate outwards, which can be tracked by the corresponding engine.

[0222] Referring now to Figure 6, a diagram of a typical cable is shown.

[0223] The cable, generally designated 600, may include a core 604 made of, for example, steel or fiber. The core 604 may have multiple, e.g., 3-6, strands 608 wrapped around it in a helical pattern. Each strand 608 may be made of multiple wires, for example, about 5 to about 10 wires, one of which is a central wire 616 and the other wires 612 are wrapped around the central wire. The wires may be made of, for example, steel or other metal.

[0224] Cables can suffer from multiple failure modes caused by, for example, friction between internal elements, heating, movement under load or stress, etc. Some types of faults can be related to structural failure of the cable.

[0225] Referring now to FIG. 7, there is shown a cable having an open circuit, a type of structural fault, as may be analyzed in accordance with some embodiments of the present invention.

[0226] 7 shows a cable 700 with a wire 702 having a break 704. To identify this type of fault, one or more frames showing such a break, referred to as a "window," may be acquired, such as frame 708. These windows may be relatively small, such as 6x6 and 12x12 pixels, and may show the break in the local context of the broken wire.

[0227] For frames acquired under illumination, each of these smaller windows, such as window 708, may be convolved with the frame capturing the cable to identify whether the frame contains a fragment that may be identified as one of the forms of break shown in the smaller window, in which case the cable itself may contain a break. It will be appreciated that the more windows available, the greater the likelihood of detecting a break in the frame, as more instances or forms of break can be inspected and detected.

[0228] It will be appreciated that strand breaks can be detected in a similar manner by taking windows showing different instances of broken strands and convolving the windows with a frame showing the cable.

[0229] Referring now to FIG. 8, there is shown another type of structural failure in a cable that may be analyzed in accordance with some embodiments of the present invention: core damage or cable fray.

[0230] FIG. 8 shows a cable 800 with wires 802 and 804 that belong to the same strand and are supposed to be adjacent to each other, but at some point one of the wires stretches so that in some locations, for example in area 810, the distance between wires 802 and 804 increases, and in other locations, for example in area 812, one of the wires rests on top of the other, for example wire 804 overlaps wire 806.

[0231] Such faults may be identified using a small size window showing various examples of such faults, for example, one wire being overstretched, increasing the distance to adjacent wires, or resting on top of other wires.

[0232] The small size window can be convolved with the cable captured in the frame, and thus it can be identified whether the image of the cable contains a section that may exhibit one of the fault modes shown in the small size window. It will be appreciated that the more windows that are available that exhibit this type of fault, the greater the chance of detecting a structural fault, as more fault instances can be inspected and detected.

[0233] Some types of faults may be associated with other structural deformations of the cable, which may have many forms and therefore be difficult to predict. Thus, in some embodiments, multiple small-sized windows representing normal wire or strand segments may be obtained. When an image of a monitored cable is analyzed, the image may be segmented, and each segment representing a portion of the cable may be examined and compared, for example, to the windows representing normal structure. If a segment fits into at least one of the windows representing normal patterns, it may be declared normal, but if a segment does not fit into any of the normal windows, it may be declared or suspected to be abnormal.

[0234] Structural faults such as those shown in Figures 7 and 8 above may be identified using an artificial analysis engine trained on multiple images showing any of the failure modes, and given another image of the cable, the engine may predict to some extent the probability that the cable will contain each type of failure mode, which can be related to the location of the fault determined during acquisition.

[0235] In some embodiments, an image showing cables may be segmented to distinguish the cables from the background, followed by processing of each such segment individually showing a single cable, e.g., as described above.

[0236] In some embodiments, segmentation of the cable may be performed by taking an image, e.g., a grayscale image, passing it through a low-pass filter to detect minima that indicate recesses between the strands, and subjecting it to morphological filtering to obtain a strand mask.

[0237] Referring now to FIG. 9, a frame illustrating a longitudinal transfer element and its segments before and after processing is shown, according to some exemplary embodiments of the present disclosure.

[0238] 9 shows a frame 900 depicting a monitored cable. Segments 908 and 912 each fit within a pre-acquired window representing a normal wire structure, while segments 916 and 920 do not. It is understood that segments 908, 912, 916, and 920 are merely exemplary, and all segments of the frame, including overlapping segments, are examined against the pre-acquired window.

[0239] After segmentation and matching, areas within the processed frame 910 associated with windows identified as anomalous may be unified and corresponding bounding polygons, such as polygons 924 and 928, may be defined.

[0240] A further failure mode may be related to lubrication issues in the cable. Lubricants may be applied to the wires during manufacturing and may penetrate down to the core. Lubricating the cable has two main benefits: it reduces friction as the individual wires and strands may move over one another, and it provides protection against corrosion of the interior and exterior surfaces of the cable.

[0241] Lubricant is substantially transparent and difficult to detect visually. However, the refractive index of lubricant is different from that of air. Therefore, values ​​of features related to light refraction can be obtained from segments of captured frames and compared to predetermined feature values. The predetermined features can be learned in a laboratory by capturing cables known to have lubrication faults (or where a lack of lubrication has been intentionally created) and cables known not to have such faults. In this way, segments exhibiting values ​​related to the faults can be identified and a bounding polygon can be defined.

[0242] In further embodiments, lubrication faults may be identified by a neural network that accounts for differences between cables, taking into account differences in capture angle, lighting conditions, etc. In some embodiments, a multi-backbone neural network, such as a convolutional neural network (CNN), may be used to identify areas indicative of lubrication faults. The networks may assess the condition of each cable individually, after which a fusion network combines the information obtained by the individual networks to assess the condition of the monitored cable system.

[0243] Some types of failure may be related to slippage or thinning, i.e., a reduction in the diameter of at least a portion of one or more cables or straps.

[0244] Referring now to FIG. 10, a typical example of a reduction in the diameter of a longitudinal transfer element is shown.

[0245] Frame 1000 is shown with multiple cables, including cable 1008, which are assumed to be equally spaced and of equal width.

[0246] In some embodiments, frame 1000 may undergo preprocessing to remove camera / sensor / lens-based distortions or to otherwise enhance the image. In some embodiments, preprocessing is also useful when a camera may need maintenance, or when determining that an image is unusable due to camera issues, such as a dirty lens, focus issues, etc. In some embodiments, preprocessing may include dividing at least frame 1000 or a portion thereof into multiple tiles. In some embodiments, preprocessing may include applying one or more filters, such as, but not limited to, a noise reduction filter, to one or more images.

[0247] In some embodiments, a background / foreground GMM filter may be applied to the image to perform background subtraction and better separate the longitudinal moving element from the background. Further morphological filters and edge detection may be applied to the image to obtain further separation. The resulting image clearly shows the morphological defects of the longitudinal moving element 1008, as shown in frame 1004, for example.

[0248] However, in other situations, in the image 1016 obtained with the filter, the longitudinally moving element 1008' appears thinner than the others, indicating a failure mode that may explain the thinning, such as excessive tension.

[0249] In a further example, as shown in frame 1020, longitudinal movement elements 1008″ and 1024 are of the expected width, but the distance between them is less than it should be and is also less than the distance between the other longitudinal movement elements, indicating a slippage failure mode of longitudinal movement element 1024.

[0250] A further problem that can arise relates to overstretched cables. One such situation can occur in the cables connecting the car and counterweight in an elevator.

[0251] 11, there is shown a schematic diagram of a side view of an elevator. The elevator comprises a car 1104, a counterweight 1108, and a cable 1112 connecting them via, for example, pulleys 1105 and 1106, or otherwise allowing relative motion between the car 1104 and the counterweight 1108 so that when one goes up the other goes down and vice versa, thereby requiring the operating engine to be used only to overcome the weight difference and not the full weight of the car with people, cargo, or other objects inside.

[0252] When the car is at its highest point, the counterweight is at its lowest point. Typically, it is necessary to damp the counterweight's descent upon reaching its lowest point and damp its ascent upon reaching its highest point so that the car gently stops upon reaching its destination. To this end, the counterweight may be coupled, for example, via a rod 1116, to a piston 1120 that enters a cylinder 1124 located on the elevator pit floor 1128. A given length of the cable 1112 determines the distance from the cylinder 1124 to the counterweight 1108 when the car 1104 is at its highest point. Stretching the cable 1112 may cause this distance to shorten, and a warning should be issued. Therefore, in some embodiments of the present disclosure, the area between the top of the cylinder 1124 and the counterweight's acceptable repositioning may be monitored by a capturing device 1132, such as a camera, video camera, or the like. This area may be captured multiple times. For example, this area may be captured at time intervals such that the lowest point can be captured within a predetermined period of time with at least a predetermined confidence level. In another embodiment, this area may be captured when the elevator controller indicates that the elevator is at the top floor, which implies that the counterweight is at its lowest position. If the distance falls below a predetermined threshold, an alarm may be issued.

[0253] In some embodiments, the diameter of the cable 1112 may be measured and compared to a predetermined value. If the diameter is smaller than expected, e.g., by at least a predetermined amount, an alert may be issued, as this may indicate wear and tear on the cable.

[0254] Various embodiments may be implemented to identify the distance between the cylinder 1108 and the counterweight 1108 or piston 1120. For example, identification may use a trained AI engine, template matching, recognition of engraved, printed, or affixed text, codes, QR codes, etc. It is understood that while the cylinder 1124 does not move, the position of the capture device 1132 may change due to changing light conditions or movement of nearby systems. Therefore, to determine the distance between the cylinder 1124 and the counterweight 1108, it may be necessary to also identify the cylinder 1124 rather than assume its position.

[0255] Another issue concerns distance calculations. The pixel / mm ratio can vary across image space and is a function of camera-specific parameters, such as focus, lens, and sensor characteristics, as well as external parameters, such as the distance to all objects in the FOV and their real-world sizes, which are typically not provided. Thus, the ratio may be different for the counterweight 1108 / piston 1120 and the cylinder 1124. The ratio can be obtained for each, for example, by measuring the actual segment for each and calculating the number of pixels in the corresponding part of the image. Given the two ratios, it is possible to interpolate the number of pixels in the image to a real-world distance and compare that distance to a threshold.

[0256] The above description relates primarily to cables, but is not limited to cables and is also applicable to other longitudinally moving elements, e.g., straps, ropes, etc. Some of the faults described above are also applicable to straps, but straps may also exhibit other faults, e.g., too much or too little tension, tears, slippage, breaks, etc., that can be detected in a similar manner by applying a different engine to analyze a frame capturing one or more straps.

[0257] While the above description focuses on elevator cables or other linearly moving elements, it will be understood that the present disclosure is not limited to such environments, but is equally applicable to longitudinal elements that move in closed loops, such as conveyor belts, aerial trams, cable cars, cable winders, etc., and elements that are connected at one end to systems that move on horizontal, vertical, inclined surfaces, etc., such as helicopters and cranes. The apparatus and methods may operate to detect faults related to the configuration and type of element.

[0258] It will be appreciated that the present disclosure is also applicable to longitudinally moving elements that are fixed at one end to another element that is stationary relative to the movement of the longitudinally moving element, and that the other end is free to retract. One example of such a situation is a rescue hoist cable used to lower from a helicopter and lift people or objects into the helicopter.

[0259] Referring to FIG. 12 , a system for releasing a cable from a helicopter, which may be complex, according to some embodiments of the present disclosure is shown. The system, generally referenced 1200, includes a cylinder 1202, also referred to as a drum, to which one end of a cable 1204 is attached, around which the cable 1204 is wound, and from which the cable 1204 descends. Solutions such as those described above can be used to detect movement in the cable from a controller that provides a cable movement indication by image analysis or any other motion sensor capable of sensing movement as described above, or in any other manner. Once movement is detected, one or more images of the cable can be captured, as described below in FIGS. 13 , 14A , 14B , and 15 . The embodiments shown in FIGS. 13 , 14A , 14B , and 15 can be used, for example, to monitor a rescue hoist cable descending from a helicopter, used to lift a person or object into the helicopter. Similar systems can also be used for crane cables, etc.

[0260] Referring now to FIG. 13, there is shown a schematic diagram of a side view of a first exemplary embodiment of a system for monitoring a cable being released or retrieved, according to some embodiments of the present disclosure.

[0261] The system includes a ring 1308 that is centered around the cable 1204. The ring 1308 can be coupled below the cable driver 1208.

[0262] The ring 1308 may include one, two, or more image capture devices, such as, for example, cameras 1312 or 1316. Each such capture device may capture a segment of the cable 1204, the segment being determined by a corresponding viewing angle 1324, 1328, respectively.

[0263] The position of the ring 1308 and the position of the capture devices 1312, 1316 that capture the cable as it is released or retrieved results in capture of the moving cable 1204 so that an image of the entire length of the cable (excluding the top where it connects to the drum) can be accumulated and analyzed for faults or failures.

[0264] In some embodiments, ring 1308 may include or be operably coupled to a motor adapted to rotate ring 1308 about cable 1204. If ring 1308 is adapted to rotate, fewer capture devices, for example, one or two capture devices, may be used, but it may take a longer time to ensure that all segments of cable 1204 have been captured from all directions. A larger number of cameras may provide more rapid coverage because they can capture cable 1204 from multiple directions simultaneously.

[0265] If the ring 1308 is not adapted to rotate, the ring 1308 may include a greater number of capture devices, e.g., three devices, four devices, etc., to ensure that all segments of the cable 1204 are captured from all directions. The number of capture devices may also depend on the distance between the capture device and the cable, the field of view of the capture device, and the cable diameter. Additionally or alternatively, assuming the cable 1204 rotates about its longitudinal axis, a longer capture time may provide for capturing all segments of the cable from all directions. In this embodiment, capture primarily operates when the cable is released or retrieved; otherwise, only certain segments of the cable are captured, or not at all.

[0266] Thus, when ring 1308 is stationary, the cable is monitored by a capture device that is stationary relative to the monitored system, whereas when ring 1308 is rotating, the cable is monitored by a capture device that moves in a plane perpendicular to the cable's travel.

[0267] 14A and 14B, there are shown schematic side and top view illustrations, respectively, of a second exemplary embodiment of a system for monitoring cables, in accordance with some embodiments of the present disclosure.

[0268] 14A and 14B, the system may include a cable release component 1412 that advances along a bracket 1408 located below the drum. The cable release component 1412 may advance along a straight line 1424 such that at each position, the cable release component 1412 releases a corresponding winding of the cable. For example, in the position shown, winding 1406 is released, and when the cable release component 1412 reaches the right end of the bracket 1308, windings 1404 and 1405 are released.

[0269] The cable release component 1412 may extend facing the drum 1202, with a member 1416 having a capture device 1420 attached thereto. The capture device 1420 may thus capture the portion of the cable being released or retrieved from the winding facing it. As the cable release component 1412 advances along the bracket 1408, so too does the member 1416 and capture device 1420, and as additional segments of the cable are released or retrieved, they are captured by the capture device 1420. Thus, this embodiment is also primarily useful when the cable is being released or retrieved; otherwise, the cable release component 1412 remains stationary and captures only a small portion of the cable.

[0270] During the movement of the cable release component 1412, the capture device 1420 may capture the windings of the cable and thus identify failure modes of the cable as well as failure modes of the windings of the cable, such as whether the windings are overlapping or if the windings are too far apart.

[0271] Thus, in this embodiment, the cable is monitored by capture devices that capture different windings of the cable.

[0272] Referring now to FIG. 15, there is shown a schematic illustration of a top view of a third exemplary embodiment of a system for monitoring cables, in accordance with some exemplary embodiments of the present disclosure.

[0273] 15, one or more still cameras 1516 may be mounted on a bracket 1508 attached to the cover of the drum 1500. The capture devices 1516 may be mounted inside the cover to face the cable windings. Each capture device 1516 may capture a segment or portion of one or more windings, and the captured segment changes as the cable is released from the respective winding.

[0274] The embodiment of FIG. 15 may provide identification of faults in the cable as well as the windings of the cable, such as overlapping windings or windings that are spaced too far apart.

[0275] Thus, in this embodiment, the cable is monitored by one or more static capture devices.

[0276] In some embodiments, it may be ensured that all portions of the cable are monitored within a time frame using any of the methods described above, e.g., random, scheduled, etc. Also, in some embodiments, a registry may be maintained of the portions of the cable that have been monitored and their respective dates and / or times to ensure that all portions of the cable are monitored.

[0277] Referring now to FIG. 16, an illustration of a drum with irregularly wrapped cables and a method for detecting the same is shown, according to some exemplary embodiments of the present disclosure.

[0278] As the cable is released (or retracted), the cylinder rotates, and the cable releases one winding, then the adjacent winding, and so on. When the last winding in the row is released, the direction is reversed and the windings are released in the other direction. Thus, if the release or winding proceeds as expected, the windings to the left of the currently released winding will form a uniform array, and so will the windings to the right.

[0279] In some embodiments, the windings may be recognized and their widths measured, optionally after correcting for any fisheye effect that may be caused by a capturing device in close proximity to the cable.

[0280] If the widths differ significantly, for example if the difference between the maximum and minimum widths exceeds a threshold, this may indicate a problem with the cable or winding mechanism. For example, the width 1605 of one winding is nearly similar to the width of the other windings except for width 1603, which is significantly smaller, which may indicate a problem.

[0281] In another example, the bottom of each winding appears as part of an ellipse or parabola. Therefore, it can be determined if the bottoms of the windings are all in a straight line. If not, there is a problem with the cable or winding mechanism.

[0282] For example, line 1612 actually connects to the bottom of the winding to the left of the currently released winding 1610. However, line 1616 does not include point 1620, which is the lowest point of one winding, and therefore presents a problem with winding or releasing. It is understood that some deviation from a straight line may be tolerated, for example, up to a predetermined distance from a line connecting as many lowest points as possible. Deviation beyond a threshold may indicate a problem.

[0283] It is understood that the lines connecting the lowest points of the windings to the left or right of the currently released winding are not necessarily assumed to be aligned with each other, as one is likely to represent one more winding than the other.

[0284] In yet another example, the structure of the cable may be analyzed to determine, for example, whether lines separating windings, such as lines 1600 and 1604, are substantially parallel, whether lines separating strands within the same winding, such as lines 1608 and 1612, are substantially parallel, or whether lines separating wires within the same strand or winding are substantially parallel.

[0285] If any of these groups of lines are not parallel, for example if their slope difference exceeds a predetermined threshold, this may be an indication of a problem.

[0286] In some embodiments, the system may be coupled to a device for calculating the position of the cable being monitored at a given time, hi some embodiments, an odometer may be used for such an assessment.

[0287] In some embodiments, the capture location of the cable may be determined based on marks on the cable, identifying small segments of the cable as described above, or the like.

[0288] In some embodiments, the winding speed of the cable may be slow enough relative to the shutter speed of the capture device that the cable appears stationary in each frame and illumination may not be necessary.

[0289] It is understood that cables may be monitored for some of the failure modes detailed above in connection with other systems, such as, for example, rust, lubrication issues, breaks, or other structural failures. However, cables may be subject to unique failure modes, such as, for example, alignment as they are being reeled in, which may be detected using tools, including the image analysis tools, AI engines, etc. described above, for analyzing frames capturing segments of the cable. As noted above, the tools for analyzing the captured frames may operate simultaneously, sequentially, etc.

[0290] It will be appreciated that the present disclosure is also applicable to other rapidly moving systems or elements. In combination with a strobe light source that operates in accordance with the capture device to enable capture of a clear image, and image registration can ensure at least a predetermined level of confidence and analyze for failure modes. The elements may be linearly moving elements, rotating elements, or elements that move in any other manner, such as with linear, angular, or compound velocities, in two or three dimensions. Thus, the present disclosure is particularly applicable to rapidly moving elements.

[0291] In one example, a turbomotor may be analyzed as shown in FIG. 17. A turbo (including a turbocharger or supercharger) increases the power output of an internal combustion engine by compressing a larger volume of air and forcing it into the cylinders. A turbomotor typically consists of a turbine 1700 and a compressor. The turbine is driven by exhaust gases, an external (usually electric) motor, or is powered directly by the engine. A belt pulley drives gears to rotate the compressor fan, while the compressor draws in and compresses the outside air and delivers it to the cylinders. Turbochargers typically rotate at high speeds, ranging from tens of thousands to over 100,000 RPM, with smaller turbochargers operating at higher speeds. The rotational speed varies based on factors such as the size of the turbocharger, engine requirements, and the specific application.

[0292] By capturing an area of ​​the turbine, the images can be analyzed for one or more failure modes, such as rust, cracks, structural distortions, etc. In addition to analyzing each image individually, the images can be registered to analyze objects or locations that appear in two or more images. By analyzing images of an area taken over a period of time, trends, such as evolving failure modes, can be discovered. By analyzing images that capture different views but cover a common area, it can be determined whether all areas of the monitored system have been analyzed, which areas require further analysis, etc.

[0293] In another example, as shown in FIG. 18, the present disclosure may be applied to imaging and analysis of cardan bearings (also referred to as cardan joints), such as cardan bearing 1800.

[0294] The present disclosure is not limited to the illustrated examples, but rather may be applied to any other rotating or moving element, including high speed rotating or moving elements.

[0295] The subject matter of the present disclosure may be a system, a method, and / or a computer program product, which may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to perform aspects of the disclosed subject matter.

[0296] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded thereon, and any suitable combination of the above. Computer-readable storage medium, as used herein, is not to be construed as being a transitory signal per se, such as, for example, radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.

[0297] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may comprise copper wire, fiber optics, wireless networks, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium in the respective computing / processing device.

[0298] Computer-readable program instructions for carrying out operations of the disclosed subject matter may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as, for example, Smalltalk, C++, and conventional procedural programming languages ​​such as, for example, the "C" programming language and similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuit to perform aspects of the disclosed subject matter.

[0299] Aspects of the disclosed subject matter are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosed subject matter. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0300] Computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to create a machine whose instructions, executing on the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts set forth in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored on a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other device to function in a particular way, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions that implement aspects of the functions / acts set forth in one or more blocks of the flowcharts and / or block diagrams.

[0301] The computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device to generate a computer-implemented process that causes a series of operational steps to be executed on the computer, other programmable apparatus, or other device, such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / acts set forth in one or more blocks of the flowcharts and / or block diagrams.

[0302] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the disclosed subject matter. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing the described logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by dedicated hardware-based systems that perform the specified functions or acts, or execute a combination of dedicated hardware and computer instructions.

[0303] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the disclosed subject matter. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is further understood that as used herein, the terms "comprises" and / or "comprising" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0304] Corresponding structure, materials, acts, and equivalents of all means or step-plus-function elements in the following claims are intended to include any structure, material, or acts for performing a function in combination with other claimed elements as specifically claimed. The description of the disclosed subject matter has been presented for purposes of illustration and description only and is not intended to be exhaustive or to limit the invention to the precise form disclosed. Numerous modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the disclosed subject matter. The embodiments were chosen and described to best explain the principles and practical applications of the disclosed subject matter and to enable those skilled in the art to understand the disclosed subject matter in various embodiments with various modifications suited to the particular applications intended.

Claims

1. 1. A system for monitoring moving elements in a monitored system, comprising: an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture one or more images of at least a portion of the moving element; acquiring at least one image of the moving element in motion under illumination emitted by the illumination device; analyzing the at least one image to determine if at least one failure mode exists in the captured portion of the moving element; registering the at least one image with at least one other image to obtain further information regarding the at least one failure mode; at least one processor configured to A system comprising:

2. The system of claim 1 , wherein the acquiring and analyzing is performed iteratively.

3. The system of claim 2 , wherein the acquiring and analyzing is repeated until images of all portions of the moving element are acquired and analyzed within a predefined time frame.

4. The system of claim 1 , further comprising determining a portion of the moving element to be captured next.

5. The system of claim 1 , wherein the registration comprises registering the at least one image with an image showing a second area adjacent to an area captured in the at least one image.

6. The system of claim 1 , wherein the registration comprises registering the at least one image with a predetermined image to determine locations shown in the at least one image.

7. The system of claim 1 , further comprising analyzing trends of the at least one failure mode at the location over time.

8. The system of claim 1 , further comprising providing an output indicating whether the at least one failure mode is identified in the moving element.

9. The system of claim 8 , wherein the output is provided by updating a database, sending a message, displaying a message on a display device, or issuing an audio alert.

10. obtaining an indication of whether the mobile element is moving relative to the monitored system; performing said acquiring, said analyzing, and said registering under the condition that said mobile element is in motion; The system of claim 1 further comprising:

11. The system of claim 1 , wherein the image sensor operates with a rolling shutter.

12. 12. The system of claim 11, wherein the lighting device is configured to begin illuminating the moving object after sensors in all rows or columns of the image sensor are open and to end illumination before any of the rows or columns are closed.

13. The system of claim 1 , wherein the moving element is a longitudinal moving element.

14. The system of claim 1 , wherein the moving element is a rotational moving element.

15. The system of claim 1 , wherein the moving element is a cable.

16. The system of claim 1 , wherein the monitored system is selected from the group consisting of an elevator, a helicopter, a crane, a conveyor belt, a turbofan, an aerial tram, a cable car, and a cable winding machine.

17. 1. A system for monitoring moving elements in a monitored system, comprising: an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor with a rolling shutter configured to capture one or more images of at least a portion of the moving element; acquiring at least one image of the moving element in motion under illumination emitted by the illumination device configured to start illuminating the moving object after all rows or columns of the image sensor are open and to end illumination before any of the rows or columns are closed; analyzing the at least one image to determine if at least one failure mode is present in the portion of the captured moving element; at least one processor configured to A system comprising:

18. 1. A system for monitoring moving elements in a monitored system, comprising: a motion sensor for determining whether the moving element is moving; an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture at least one image of at least a portion of the moving element; obtaining an indication of whether the mobile element is moving relative to the monitored system; Provided that the moving element is moving, operating the lighting device to emit light at a predetermined time; acquiring at least one image of the moving element under illumination emitted by the illumination device; analyzing the at least one image to determine if at least one failure mode exists in the moving element; at least one processor configured to A system comprising:

19. 20. The system of claim 18, wherein the lighting devices are coordinated to be activated and deactivated within a maximum of 10 milliseconds.

20. 1. A system for monitoring a rescue hoist cable being lowered from a helicopter, comprising: a lighting device configured to periodically provide a momentary light onto a drum on which the rescue hoist cable is wound; an image sensor configured to capture at least one image of the drum; operating the lighting device to emit light at a predetermined time; acquiring at least one image of at least a portion of the drum when the illumination device is operating; analyzing the at least one image to determine if at least one failure mode exists in the moving element; at least one processor configured to A system comprising:

21. 21. The system of claim 20, wherein the failure mode is a cable failure.

22. 21. The system of claim 20, wherein the failure mode is a failure of a take-up mechanism configured to take up the cable onto the drum.

23. 23. The system of claim 22, wherein analyzing the at least one image comprises determining that the at least one failure mode is present if the windings are of different widths.

24. 23. The system of claim 22, wherein analyzing the at least one image comprises determining that the at least one failure mode is present if dividing lines are of different widths.

25. 23. The system of claim 22, wherein analyzing the at least one image comprises determining that the at least one failure mode is present if lines separating windings are not parallel.

26. 23. The system of claim 22, wherein analyzing the at least one image comprises determining that the at least one failure mode is present if lines separating cable strands are not parallel.

27. 1. A system for monitoring moving elements in a monitored system, comprising: an illumination device configured to periodically provide momentary light to the longitudinally moving element; an image sensor configured to capture at least one image of at least a portion of the moving element; acquiring at least one image of the moving element under illumination emitted by the illumination device; analyzing the at least one image to determine if at least one failure mode exists in the moving element; at least one processor configured to Equipped with The system is configured such that the image sensor captures the same position within the moving element throughout an exposure time used to capture the at least one image.

28. 28. The system of claim 27, wherein the image sensor is moved according to the moving element throughout the exposure time.

29. 28. The system of claim 27, further comprising a mirror, wherein the system is configured such that the mirror is moved in accordance with the moving object throughout the capture time of the at least one image, thereby reflecting the same position within the moving element to the image sensor throughout the capture time.

30. 28. The system of claim 27, wherein the image sensor captures different images of different portions of the moving element.

Images