Escalator operation abnormity identification monitoring method and system

Abstract

The invention relates to the field of image data processing, in particular to a method and system for recognizing and monitoring abnormal operation of an escalator, and the method comprises the steps: carrying out the escalator movement mode matching analysis on the movement feature data of parts in the escalator, determining the mechanical loss change trend of the parts, and predicting the load and movement deviation associated information of the escalator; bearing measured data and a load state image of the escalator are jointly analyzed to obtain actual motion offset state information, and load dynamic image recognition is performed on the escalator to obtain actual dynamic characteristics of a load on the escalator, so that external action information of the escalator is determined, and whether an abnormal operation event occurs to the escalator or not is judged; and the escalator braking parameter information and the posture feature information of the load on the escalator are visually recognized, so that the escalator is adjusted to be switched to the motion state of the braking mode. According to the method, the abnormal running condition of the escalator can be accurately predicted through image recognition, smooth running transition switching is conducted on the escalator in time, and the running safety of the escalator is improved.

Description

[Technical field]

[0001] The present invention relates to the field of image data processing, and in particular to a method and system for identifying and monitoring abnormal operation of an escalator. [Background technology]

[0002] As an automatic transportation tool, escalators are widely used in crowded places such as shopping malls and subway stations. Escalators can transport a large number of people and goods vertically in a short period of time. They can not only quickly and automatically transfer a large number of people and goods from a small space to provide convenience and a good travel experience for people, but also effectively avoid safety accidents caused by excessive concentration of people.

[0003] The escalator includes a complex transmission mechanical structure. The components of the transmission mechanical structure will inevitably wear out during long-term operation, causing the escalator movement, especially the movement of all the steps under the escalator, to deviate from the original trajectory. Although escalators are special mechanical equipment and are regularly inspected and maintained, they face various complex transportation scenarios during daily operation, which increases the probability of abnormal events such as jamming during escalator operation. In order to timely control the operating status of the escalator when an abnormal event occurs, the escalator has been equipped with emergency brake buttons and other solutions to facilitate the active stopping of the escalator operation in an emergency. However, the above solution is only a remedy when an abnormal event has occurred in the escalator, and it is impossible to accurately predict the potential risks of the escalator during operation, let alone control the operation of the escalator in a stable manner. It can be seen that how to monitor the operation of the escalator from multiple dimensions, predict the abnormal operation of the escalator, and timely switch the escalator to a smooth operation transition is of great significance to improving the safety of escalator operation. [Summary of the invention]

[0004] In order to globally monitor the operation of the escalator from the internal and external aspects of the escalator, identify the unstable factors that affect the operation of the escalator, accurately predict the abnormal operation of the escalator, timely switch the escalator to smooth operation, and improve the safety of escalator operation, the present invention provides an abnormal operation identification and monitoring method for an electric escalator, the method comprising the following steps:

[0005] Acquire motion characteristic data of components inside the escalator, perform motion pattern matching analysis on the component motion characteristic data about the escalator, and determine the mechanical wear variation trend of components inside the escalator; based on the mechanical wear variation trend of components, predict the load and motion offset correlation information of the escalator;

[0006] Acquire measured load-bearing data and a load state image of the escalator, perform joint analysis on the measured load-bearing data and the load state image, and determine actual motion offset state information of the escalator; perform load dynamic image recognition on the escalator to obtain actual dynamic characteristics of the load on the escalator, thereby determining external action information on the escalator; and determine whether an abnormal operation event will occur on the escalator based on the actual motion offset state information and the external action information;

[0007] When an abnormal operation event occurs in the escalator, the escalator is subjected to step dynamic visual recognition and load posture visual recognition to obtain escalator stopping parameter information and posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and the posture characteristic information, the movement state of the escalator is adjusted to switch to the stopping mode.

[0008] Preferably, the acquiring of motion characteristic data of components inside the escalator, performing motion pattern matching analysis on the escalator on the motion characteristic data of the components, determining the mechanical loss variation trend of components inside the escalator; and predicting the load and motion offset correlation information of the escalator based on the mechanical loss variation trend of the components, specifically comprises:

[0009] Obtaining the motion friction and motion separation gap change characteristic data of each of the transmission components inside the escalator, and obtaining the component transmission operation path data of all the transmission components inside the escalator in the corresponding motion mode;

[0010] Based on the motion friction and motion separation gap variation characteristic data and the component transmission operation path data, modeling and analysis are performed on all transmission components inside the escalator to determine the mechanical loss variation trend of all transmission components inside the escalator under the corresponding motion mode; wherein the mechanical loss variation trend includes the physical structure wear degree variation rate of all transmission components under the corresponding motion mode and the power transmission loss variation rate between other adjacent transmission components;

[0011] Based on the mechanical loss variation trend of all transmission components in the corresponding motion mode, the motion trajectory offset information of the escalator itself under different load-bearing states is predicted, which is used as the load and motion offset correlation information of the escalator; wherein the motion trajectory offset information includes the offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

[0012] Preferably, the method comprises acquiring measured load-bearing data and a load state image of the escalator, jointly analyzing the measured load-bearing data and the load state image to determine the actual motion offset state information of the escalator; and performing load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator, specifically:

[0013] Acquire the measured global load-bearing data of the step surface of the escalator and the global load state image of the step surface of the escalator; perform object contour recognition on the global load state image to obtain the load distribution position information of the step surface of the escalator;

[0014] Based on the load distribution position information, the global load-bearing measured data of the step surface is processed by region identification to obtain the actual load bearing state corresponding to the escalator; the actual load bearing state is compared with the load and motion offset association information of the escalator to determine the actual motion offset state information of the escalator; wherein the actual offset state information of the escalator includes the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom;

[0015] The escalator is dynamically photographed with load to obtain a global dynamic image of the load on the step surface of the escalator; the load contour position time evolution analysis is performed on the global dynamic image to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; and based on the movement speed and the movement amplitude, the intensity information of the action on the escalator from the load along the specific direction is determined.

[0016] Preferably, judging whether an abnormal operation event will occur in the escalator based on the actual motion offset state information and the external action information is specifically:

[0017] Based on the actual motion offset state information and the action intensity information, the operation offset recovery state of all the steps under the escalator is identified to determine whether all the steps can be restored to a state without motion offset within a preset time range; if so, it is determined that the escalator will not have an abnormal operation event; if not, it is determined that the escalator will have an abnormal operation event, and the position information of all abnormal steps is determined.

[0018] Preferably, when an abnormal operation event occurs in the escalator, the escalator is subjected to step dynamic visual recognition and load posture visual recognition to obtain escalator stopping parameter information and posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and the posture characteristic information, the motion state of the escalator is adjusted to switch to the stopping mode, specifically:

[0019] When an abnormal operation event occurs in the escalator, all abnormal steps are tracked and photographed based on the position information of all abnormal steps to obtain dynamic motion images and load dynamic images of each abnormal step;

[0020] Performing visual recognition on the dynamic motion image to obtain the allowable stopping distance information of the escalator for stopping, which is used as the escalator stopping parameter information; performing visual recognition on the load center of gravity posture of the load dynamic image to obtain the center of gravity offset posture angle information of the load on the abnormal step, which is used as the posture feature information of the load on the escalator;

[0021] Based on the allowable braking distance information and the center of gravity offset attitude angle information, the braking acceleration when the escalator is switched to the braking mode is adjusted.

[0022] On the other hand, the present invention provides an escalator control operation system based on image recognition, the system comprising the following modules:

[0023] A component wear variation trend determination module is used to obtain component motion characteristic data inside the escalator, perform motion pattern matching analysis on the component motion characteristic data about the escalator, and determine the component mechanical wear variation trend inside the escalator;

[0024] A load and motion offset correlation prediction module, used to predict the load and motion offset correlation information of the escalator based on the mechanical wear variation trend of the components;

[0025] An escalator load-bearing and load identification module, used to obtain the measured load-bearing data and load status image of the escalator;

[0026] An actual motion offset analysis module, used for jointly analyzing the load-bearing measured data and the load state image to determine the actual motion offset state information of the escalator;

[0027] A load dynamic action recognition module is used to perform load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator;

[0028] An operation abnormality judgment module is used to judge whether an operation abnormality event will occur in the escalator based on the actual motion offset state information and the external action information;

[0029] An escalator visual recognition module is used to perform step dynamic visual recognition and load posture visual recognition on the escalator when an abnormal operation event occurs on the escalator, and obtain escalator stopping parameter information and posture feature information of the load on the escalator;

[0030] A braking motion adjustment module is used to adjust the motion state of the escalator when it switches to a braking mode based on the escalator braking parameter information and the posture feature information.

[0031] Preferably, the component wear variation trend determination module obtains component motion characteristic data inside the escalator, performs motion pattern matching analysis on the escalator on the component motion characteristic data, and determines the component mechanical wear variation trend inside the escalator, specifically:

[0032] Obtaining the motion friction and motion separation gap change characteristic data of each of the transmission components inside the escalator, and obtaining the component transmission operation path data of all the transmission components inside the escalator in the corresponding motion mode;

[0033] Based on the motion friction and motion separation gap variation characteristic data and the component transmission operation path data, modeling and analysis are performed on all transmission components inside the escalator to determine the mechanical loss variation trend of all transmission components inside the escalator under the corresponding motion mode; wherein the mechanical loss variation trend includes the physical structure wear degree variation rate of all transmission components under the corresponding motion mode and the power transmission loss variation rate between other adjacent transmission components;

[0034] The load and motion offset correlation prediction module predicts the load and motion offset correlation information of the escalator based on the mechanical loss change trend of the components, specifically:

[0035] Based on the mechanical loss variation trend of all transmission components in the corresponding motion mode, the motion trajectory offset information of the escalator itself under different load-bearing states is predicted, which is used as the load and motion offset correlation information of the escalator; wherein the motion trajectory offset information includes the offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

[0036] Preferably, the escalator load-bearing and load identification module obtains the measured load-bearing data and load status image of the escalator, specifically:

[0037] Acquiring measured global load-bearing data of the step surface of the escalator and a global load state image of the step surface of the escalator;

[0038] The actual motion offset analysis module performs a joint analysis on the load-bearing measured data and the load state image to determine the actual motion offset state information of the escalator, specifically:

[0039] Performing object contour recognition on the load global state image to obtain load distribution position information on the step surface of the escalator;

[0040] Based on the load distribution position information, the global load-bearing measured data of the step surface is processed by region identification to obtain the actual load bearing state corresponding to the escalator; the actual load bearing state is compared with the load and motion offset association information of the escalator to determine the actual motion offset state information of the escalator; wherein the actual offset state information of the escalator includes the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom;

[0041] The load dynamic action identification module performs load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator, specifically:

[0042] The escalator is dynamically photographed with load to obtain a global dynamic image of the load on the step surface of the escalator; the load contour position time evolution analysis is performed on the global dynamic image to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; and based on the movement speed and the movement amplitude, the intensity information of the action on the escalator from the load along the specific direction is determined.

[0043] Preferably, the operation abnormality judgment module judges whether an operation abnormality event will occur in the escalator based on the actual motion offset state information and the external action information, specifically:

[0044] Based on the actual motion offset state information and the action intensity information, the operation offset recovery state of all the steps under the escalator is identified to determine whether all the steps can be restored to a state without motion offset within a preset time range; if so, it is determined that the escalator will not have an abnormal operation event; if not, it is determined that the escalator will have an abnormal operation event, and the position information of all abnormal steps is determined.

[0045] Preferably, when an abnormal operation event occurs in the escalator, the escalator visual recognition module performs step dynamic visual recognition and load posture visual recognition on the escalator to obtain escalator stopping parameter information and posture feature information of the load on the escalator, specifically:

[0046] When an abnormal operation event occurs in the escalator, all abnormal steps are tracked and photographed based on the position information of all abnormal steps to obtain dynamic motion images and load dynamic images of each abnormal step;

[0047] Performing visual recognition on the dynamic motion image to obtain the allowable stopping distance information of the escalator for stopping, which is used as the escalator stopping parameter information; performing visual recognition on the load center of gravity posture of the load dynamic image to obtain the center of gravity offset posture angle information of the load on the abnormal step, which is used as the posture feature information of the load on the escalator;

[0048] The stopping motion adjustment module adjusts the motion state of the escalator switching to the stopping mode based on the escalator stopping parameter information and the posture feature information, specifically:

[0049] Based on the allowable braking distance information and the center of gravity offset attitude angle information, the braking acceleration when the escalator is switched to the braking mode is adjusted.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The motion characteristic data of the components inside the escalator are obtained, and the motion mode matching analysis of the component motion characteristic data is performed on the escalator to determine the change trend of the mechanical loss of the components inside the escalator; based on the change trend of the mechanical loss of the components, the load and motion offset correlation information of the escalator are predicted. The escalator has a large and complex transmission mechanical mechanism inside, and the transmission mechanical mechanism contains a large number of components such as gears and belts. These components will inevitably suffer physical mechanical loss during the long-term operation of the escalator, and the physical mechanical loss of these components will show different change trends due to the different motion modes of the escalator. For example, the degree of physical mechanical loss of the components when the escalator is in a high-speed motion mode is higher than that of the components when it is in a low-speed motion mode. It can be seen that the higher the escalator's motion speed, the more likely the components will suffer physical mechanical loss, that is, the type of motion mode the escalator is in directly affects the change trend of the mechanical loss of the components inside the escalator. In addition, the change trend of the mechanical loss of the components will affect the motion path of the escalator, especially the motion path of the subordinate steps of the escalator, causing the motion path of the escalator to deviate.

[0052] When a certain degree of mechanical loss occurs to the components inside the escalator, the friction and mechanical joint tightness between the corresponding components will also change accordingly, which is positively correlated with the change in the degree of mechanical loss of the components. To this end, the motion characteristic data of the components inside the escalator are first obtained, and the acquired motion characteristic data of the components are matched and analyzed with the current motion mode of the escalator to determine the loss change trend of the internal components of the escalator under the current motion mode. In this way, the loss change trend of the components of the escalator under different motion modes can be targetedly identified. In addition, based on the loss change trend of the escalator's components under different motion models, the escalator's own motion path deviation under different load states (i.e., the state of carrying people and / or objects of different weights) is differentiated and predicted, providing a reliable basis for the subsequent determination of the actual motion deviation state of the escalator.

[0053] Obtain the measured load-bearing data and load status image of the escalator, conduct joint analysis on the measured load-bearing data and load status image, and determine the actual motion offset state information of the escalator; perform load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator; based on the actual motion offset state information and external action information, determine whether the escalator will have an abnormal operation event. In actual operation, the escalator will carry people and / or objects of different weights at different steps, so that the actual load-bearing and load state of the escalator are not fixed. Considering the correlation between the actual load-bearing capacity and load of the escalator and the movement deviation of the escalator, in order to determine the actual movement deviation state of the escalator without the need for complex actual detection of the escalator, the measured load-bearing data and load state image of the escalator are first obtained, and the above measured load-bearing data and load state image are jointly analyzed, that is, the actual load of the escalator is obtained from the two levels of the actual load-bearing weight and load distribution image of the escalator, and the actual movement deviation state information of the escalator is matched and compared with the above predicted load and movement deviation association information to obtain the actual movement deviation state information of the escalator, and the actual movement trajectory deviation state of the escalator under the current load is quantitatively determined. In addition, the people carried on the escalator will walk or jump during the escalator ride, so as to accurately identify the external effects on the escalator during the action. When the people carried by the escalator initiate an action and apply external force to the escalator, the external force will aggravate the degree of movement deviation of the escalator itself, causing the escalator to further deviate on the basis of its original actual movement trajectory deviation, resulting in the escalator being unable to rely on itself to recover to its original normal movement state, and finally causing abnormal events such as jamming or tilting of the escalator. To this end, based on the actual motion offset state information and external action information, the escalator operation is predicted and analyzed, which can accurately predict whether an abnormal event will occur when the escalator operation gradually deteriorates but no actual operation abnormality occurs.

[0054] When an abnormal operation event occurs in the escalator, the escalator is subjected to dynamic visual recognition of the steps and visual recognition of the load posture to obtain the escalator stopping parameter information and the posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and posture characteristic information, the escalator is adjusted to switch to the stopping mode. When an abnormal operation event occurs in the escalator, it indicates that at least one step in the escalator has an abnormal problem of running stuck or tilting. In order to avoid the step with abnormal problems affecting other normal steps and causing danger to the people on the escalator, the escalator needs to be stopped in time. The existing escalator stopping operation is to reduce the escalator's movement speed to zero in a short time. Considering that most people on the escalator do not hold the handrails firmly and the escalator may also carry large objects such as suitcases, people and objects will fall under the action of inertia and cause safety accidents. In order to achieve a smooth transition of the escalator's braking and stopping operations, the escalator is subjected to dynamic visual recognition of the steps and load posture recognition to obtain the escalator's braking parameter information and the posture characteristic information of the load on the escalator. The allowable braking distance of the escalator is accurately determined, and the standing posture of the people on the escalator is calibrated. In this way, the deceleration state of the escalator during the braking and stopping operation is adjusted to ensure that the escalator can decelerate smoothly and avoid accidents caused by excessive deceleration.

Brief Description of the Drawings

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0056] Figure 1 The present invention provides a flow chart of an escalator operation abnormality identification and monitoring method.

[0057] Figure 2 It is a structural diagram of the escalator control operation system based on image recognition provided by the present invention. [Specific implementation method]

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention rather than all structures are shown in the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0059] The terms "including" and "having" and any variations thereof in the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] See also Figure 1 As shown, the present invention provides a method for identifying and monitoring abnormal operation of an escalator, the method comprising the following steps:

[0062] S100, obtaining motion characteristic data of components inside the escalator, performing motion pattern matching analysis on the component motion characteristic data, and determining the mechanical wear variation trend of components inside the escalator; based on the mechanical wear variation trend of components, predicting the correlation information between the load and motion offset of the escalator.

[0063] Furthermore, the motion characteristic data of the components inside the escalator are obtained, and the motion pattern matching analysis of the escalator is performed on the motion characteristic data of the components to determine the change trend of the mechanical loss of the components inside the escalator; based on the change trend of the mechanical loss of the components, the load and motion offset correlation information of the escalator is predicted, specifically:

[0064] Obtaining the motion friction and motion separation gap change characteristic data of all the transmission components inside the escalator, and obtaining the component transmission operation path data of all the transmission components inside the escalator in the corresponding motion mode;

[0065] Based on the characteristic data of motion friction and motion separation gap changes and the transmission operation path data of components, all transmission components inside the escalator are modeled and analyzed to determine the mechanical loss change trend of all transmission components inside the escalator under the corresponding motion mode; the mechanical loss change trend includes the change rate of the physical structure wear degree of all transmission components under the corresponding motion mode and the change rate of the power transmission loss between other adjacent transmission components;

[0066] Based on the mechanical loss change trend of all transmission components in the corresponding motion mode, the motion trajectory offset information of the escalator itself under different load-bearing states is predicted, which is used as the load and motion offset correlation information of the escalator; among which, the motion trajectory offset information includes the offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

[0067] As a special mechanical equipment, the escalator has a large and complex transmission mechanical mechanism, which includes a large number of parts such as gears and belts. During the long-term operation of the escalator, these parts will inevitably suffer physical mechanical losses. For example, the gear surface will wear out, resulting in the inability of two adjacent gears to closely contact for power transmission, and / or the belt will undergo elastic relaxation and cannot maintain the original elastic deformation performance, resulting in gaps between the belt and other parts such as gears. These physical mechanical losses will affect the normal operation of the escalator and will continue to increase as the escalator continues to operate. In particular, the physical mechanical losses of these parts will show different deterioration trends due to the different motion modes of the escalator. The degree of physical mechanical loss of the parts in the high-speed motion mode of the escalator is much higher than that in the low-speed motion mode of the escalator. That is, the higher the escalator's movement speed, the more likely the parts will suffer physical mechanical losses. The movement speed of the escalator's motion mode is positively correlated with the rate of change of mechanical loss of the parts inside the escalator. In addition, the changing trend of the mechanical wear of components will also affect the movement path of the escalator, especially the running path of the subordinate steps of the escalator. When the mechanical wear of a certain component exceeds the preset tolerable range, the steps that have a mechanical transmission relationship with the component will not be able to obtain sufficient driving force and the movement path will be deviated, which will eventually affect the entire escalator and cause its movement path to deviate.

[0068] Through the above analysis, it can be seen that when the parts inside the escalator have a corresponding degree of mechanical loss, the friction and mechanical joint tightness between the corresponding parts will also change accordingly, and these changes are positively correlated with the changes in the degree of mechanical loss of the parts. In order to quantitatively identify the trend of mechanical loss changes of the parts inside the escalator, all the transmission parts inside the escalator are first tested to obtain the characteristic data of the movement friction and movement separation gap changes of all the transmission parts (that is, the movement friction force change data of the surface of the transmission parts during the working process and the separation gap size change data between the transmission parts and other adjacent transmission parts during the working process), and obtain the transmission motion path data of all the transmission parts inside the escalator in the corresponding motion mode (that is, the motion trajectory path data of the transmission parts when the escalator is in motion modes with different motion speeds). Then, based on the characteristic data of motion friction and motion separation gap changes and the data of component transmission motion path, all transmission components inside the escalator are modeled and analyzed to determine the mechanical loss change trend of all transmission components inside the escalator under the corresponding motion mode, so as to accurately characterize the correlation between the change rate of the physical structure wear degree of all transmission components under the corresponding motion mode and the change rate of the power transmission loss between other adjacent transmission components, and realize the differentiated characterization of the mechanical loss change trend of transmission components under different motion modes. Among them, the above modeling and analysis belong to the conventional technical means in this field and will not be introduced in detail here.

[0069] In addition, when the weight of the people and / or objects carried by the escalator is different, the movement deviation state of the escalator is also different. Generally speaking, the heavier the weight of the people and / or objects carried by the escalator, the greater the movement deviation of the hairpin. Therefore, based on the trend of component wear changes of the escalator under different motion models, the deviation of the escalator's own motion path under different load states (i.e., the state of carrying people and / or objects of different weights) is differentiated and predicted, and the deviation information of the motion trajectory of all steps under the escalator in six degrees of freedom is accurately determined, providing a reliable basis for the subsequent determination of the actual motion deviation state of the escalator.

[0070] S200, obtaining the measured load-bearing data and load status image of the escalator, jointly analyzing the measured load-bearing data and the load status image to determine the actual motion offset state information of the escalator; performing load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator; based on the actual motion offset state information and the external action information, determining whether the escalator will experience an abnormal operation event.

[0071] Furthermore, the measured load-bearing data and load state image of the escalator are obtained, and the measured load-bearing data and the load state image are jointly analyzed to determine the actual motion offset state information of the escalator; the load dynamic image of the escalator is recognized to obtain the actual dynamic characteristics of the load on the escalator, so as to determine the external action information on the escalator, specifically:

[0072] Obtaining measured global load-bearing data of the escalator step surface and a global load state image of the escalator step surface; performing object contour recognition on the global load state image to obtain load distribution position information on the escalator step surface;

[0073] Based on the load distribution position information, the global load-bearing measured data of the step surface is processed by region identification to obtain the actual load-bearing state corresponding to the escalator; the actual load-bearing state is compared with the load and motion offset correlation information of the escalator to determine the actual motion offset state information of the escalator; wherein the actual offset state information of the escalator includes the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom;

[0074] The escalator is dynamically photographed for load to obtain a global dynamic image of the load on the step surface of the escalator; the load contour position time-evolution analysis is performed on the global dynamic image to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; based on the movement speed and movement amplitude, the intensity information of the load acting on the escalator along a specific direction is determined.

[0075] Furthermore, based on the actual motion offset state information and external action information, it is determined whether the escalator will have an abnormal operation event, specifically:

[0076] Based on the actual motion offset state information and action intensity information, the operation offset recovery state of all steps under the escalator is identified to determine whether all steps can be restored to a state without motion offset within a preset time range; if so, it is determined that the escalator will not have an abnormal operation event; if not, it is determined that the escalator will have an abnormal operation event and the location information of all abnormal steps is determined.

[0077] During the actual operation of the escalator, people and / or objects of different weights will be carried on different steps, so that the actual load-bearing and load state of the escalator are not fixed. Considering the correlation between the actual load-bearing and load of the escalator and the motion offset of the escalator, and the aforementioned predicted load and motion offset associated information of the escalator, at this time, it is only necessary to determine the actual load-bearing and load distribution state of the escalator, and compare it with the aforementioned and predicted load and motion offset associated information to determine the actual motion offset state of the escalator, which saves the operation of complex actual detection of the escalator. To this end, the actual load-bearing measured data and load state image of the escalator are first obtained, and the above-mentioned actual load-bearing measured data and load state image are jointly analyzed, that is, from the two levels of the actual load-bearing weight situation and load distribution image situation of the escalator, the actual load situation of the escalator is obtained, and this is matched and compared with the above-mentioned predicted load and motion offset associated information to obtain the actual motion offset state information of the escalator, and the actual motion trajectory offset state of the escalator under the current load situation is quantitatively determined, so as to accurately determine the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

[0078] In addition, the people carried on the escalator will walk or jump during the escalator ride, which will have a negative impact on the normal movement of the escalator. The greater the intensity of the movement, the more likely the escalator, especially the steps directly corresponding to the movement, will experience abnormal events such as freezing. In order to accurately predict the potential abnormal events of the escalator, the load dynamic image of the escalator is first taken to obtain the global dynamic image of the load on the step surface of the escalator; the load global dynamic image is analyzed for the load contour position to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; then based on the movement speed and movement amplitude, the intensity information of the action from the load along a specific direction on the escalator is determined, so as to accurately determine the external force on the escalator, and provide a reliable basis for the subsequent judgment of whether the escalator has an abnormal operation event. In addition, when the people carried by the escalator initiate an action and apply external force to the escalator, the external force will aggravate the degree of movement deviation of the escalator itself, causing the escalator to further deviate on the basis of its original actual movement trajectory deviation, resulting in the escalator being unable to rely on itself to recover to its original normal movement state, and finally causing abnormal events such as freezing or tilting of the escalator. To this end, based on the actual motion offset state information and external action information, the operation of the escalator is predicted and analyzed, and it can accurately predict whether an abnormal event will occur when the escalator operation gradually deteriorates but no actual operation abnormality occurs. Specifically, based on the actual motion offset state information and action intensity information, the operation offset recovery state of all steps under the escalator is identified to determine whether all steps can be restored to a state without motion offset within a preset time range, thereby identifying and determining the position information of all abnormal steps inside the escalator.

[0079] S300, when an abnormal operation event occurs in the escalator, dynamic visual recognition of the escalator steps and visual recognition of the load posture are performed to obtain the escalator stopping parameter information and the posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and the posture characteristic information, the movement state of the escalator is adjusted to switch to the stopping mode.

[0080] Furthermore, when an abnormal operation event occurs in the escalator, the escalator is subjected to step dynamic visual recognition and load posture visual recognition to obtain the escalator stopping parameter information and the posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and the posture characteristic information, the motion state of the escalator switching to the stopping mode is adjusted, specifically:

[0081] When an escalator has an abnormal operation, all abnormal steps are tracked and photographed based on their location information to obtain dynamic motion images and load dynamic images of each abnormal step.

[0082] Perform visual recognition on the dynamic motion image to obtain the allowable stopping distance information of the escalator, which is used as the escalator stopping parameter information; perform visual recognition on the load center of gravity posture of the load dynamic image to obtain the center of gravity offset posture angle information of the load on the abnormal step, which is used as the posture feature information of the load on the escalator;

[0083] Based on the allowable braking distance information and the center of gravity offset attitude angle information, the braking acceleration when the escalator switches to the braking mode is adjusted.

[0084] When an escalator has an abnormal operation, it indicates that at least one step of the escalator has an abnormal problem of running stuck or tilting. In order to prevent the step with the abnormal problem from affecting other normal steps and causing danger to the people on the escalator, the escalator needs to be stopped in time. The existing escalator stopping operation is to reduce the escalator's movement speed to zero in a short time. Considering that most people on the escalator do not hold the handrails firmly and there may be large objects such as suitcases on the escalator, people and objects will fall under the action of inertia and cause safety accidents; when people or objects on the escalator are in a forward tilting posture, people or objects on the escalator are more likely to fall due to the emergency stop of the escalator. To achieve a smooth transition of the escalator's stopping operation and reduce the probability of people or objects on the escalator falling, all abnormal steps are tracked and photographed based on the position information of all abnormal steps, and dynamic motion images and load dynamic images of each abnormal step are obtained. The dynamic motion images are visually identified to obtain the allowable stopping distance information for the escalator to stop. The load dynamic images are also visually identified to obtain the center of gravity posture information of the load on the abnormal steps. The deceleration state (i.e., the stopping acceleration) of the escalator during the stopping operation is adjusted based on the allowable stopping distance information and the center of gravity posture information, to ensure that the escalator can decelerate smoothly, avoid accidents caused by excessive deceleration, and improve the safety of escalator operation.

[0085] See also Figure 2 As shown, the present invention provides an escalator control operation system based on image recognition, which includes the following modules:

[0086] A component wear variation trend determination module is used to obtain the motion characteristic data of the components inside the escalator, perform motion pattern matching analysis on the component motion characteristic data about the escalator, and determine the mechanical wear variation trend of the components inside the escalator;

[0087] The load and motion offset correlation prediction module is used to predict the load and motion offset correlation information of the escalator based on the change trend of mechanical wear of parts;

[0088] Escalator load bearing and load identification module, used to obtain the escalator load bearing measured data and load status image;

[0089] The actual motion offset analysis module is used to jointly analyze the load-bearing measured data and the load state image to determine the actual motion offset state information of the escalator;

[0090] The load dynamic action recognition module is used to perform load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator;

[0091] An operation abnormality judgment module is used to judge whether an escalator operation abnormality event will occur based on the actual motion deviation state information and external action information;

[0092] The escalator visual recognition module is used to perform step dynamic visual recognition and load posture visual recognition on the escalator when an abnormal operation event occurs on the escalator, and obtain the escalator stopping parameter information and the posture characteristic information of the load on the escalator;

[0093] The braking motion adjustment module is used to adjust the motion state of the escalator to switch to the braking mode based on the escalator braking parameter information and posture feature information.

[0094] Furthermore, the component wear variation trend determination module obtains the component motion characteristic data inside the escalator, performs motion pattern matching analysis on the component motion characteristic data about the escalator, and determines the mechanical wear variation trend of the components inside the escalator, specifically:

[0095] Obtaining the motion friction and motion separation gap change characteristic data of all the transmission components inside the escalator, and obtaining the component transmission operation path data of all the transmission components inside the escalator in the corresponding motion mode;

[0096] Based on the characteristic data of motion friction and motion separation gap changes and the transmission operation path data of components, all transmission components inside the escalator are modeled and analyzed to determine the mechanical loss change trend of all transmission components inside the escalator under the corresponding motion mode; the mechanical loss change trend includes the change rate of the physical structure wear degree of all transmission components under the corresponding motion mode and the change rate of the power transmission loss between other adjacent transmission components;

[0097] The load and motion offset correlation prediction module predicts the load and motion offset correlation information of the escalator based on the change trend of mechanical wear of parts, specifically:

[0098] Based on the mechanical loss change trend of all transmission components in the corresponding motion mode, the motion trajectory offset information of the escalator itself under different load-bearing states is predicted, which is used as the load and motion offset correlation information of the escalator; among which, the motion trajectory offset information includes the offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

[0099] Furthermore, the escalator load bearing and load identification module obtains the measured load bearing data and load status image of the escalator, specifically:

[0100] Obtaining measured global load-bearing data of the escalator step surface and a global load state image of the escalator step surface;

[0101] The actual motion offset analysis module conducts a joint analysis of the load-bearing measured data and the load state image to determine the actual motion offset state information of the escalator, specifically:

[0102] Perform object contour recognition on the load global state image to obtain load distribution position information on the escalator step surface;

[0103] Based on the load distribution position information, the global load-bearing measured data of the step surface is processed by region identification to obtain the actual load-bearing state corresponding to the escalator; the actual load-bearing state is compared with the load and motion offset correlation information of the escalator to determine the actual motion offset state information of the escalator; wherein the actual offset state information of the escalator includes the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom;

[0104] The load dynamic action recognition module performs load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator, specifically:

[0105] The escalator is dynamically photographed for load to obtain a global dynamic image of the load on the step surface of the escalator; the load contour position time-evolution analysis is performed on the global dynamic image to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; based on the movement speed and movement amplitude, the intensity information of the load acting on the escalator along a specific direction is determined.

[0106] Furthermore, the operation abnormality judgment module judges whether an escalator operation abnormality event will occur based on the actual motion offset state information and external action information, specifically:

[0107] Based on the actual motion offset state information and action intensity information, the operation offset recovery state of all steps under the escalator is identified to determine whether all steps can be restored to a state without motion offset within a preset time range; if so, it is determined that the escalator will not have an abnormal operation event; if not, it is determined that the escalator will have an abnormal operation event and the location information of all abnormal steps is determined.

[0108] Furthermore, when an abnormal operation event occurs in the escalator, the escalator visual recognition module performs step dynamic visual recognition and load posture visual recognition on the escalator to obtain the escalator stopping parameter information and the posture characteristic information of the load on the escalator, specifically:

[0109] When an escalator has an abnormal operation, all abnormal steps are tracked and photographed based on their location information to obtain dynamic motion images and load dynamic images of each abnormal step.

[0110] Perform visual recognition on the dynamic motion image to obtain the allowable stopping distance information of the escalator, which is used as the escalator stopping parameter information; perform visual recognition on the load center of gravity posture of the load dynamic image to obtain the center of gravity offset posture angle information of the load on the abnormal step, which is used as the posture feature information of the load on the escalator;

[0111] The stop motion adjustment module adjusts the motion state of the escalator switching to the stop mode based on the escalator stop parameter information and posture feature information, specifically:

[0112] Based on the allowable braking distance information and the center of gravity offset attitude angle information, the braking acceleration when the escalator switches to the braking mode is adjusted.

[0113] The operation and effect of the escalator control operation system based on image recognition of the present invention correspond to and are consistent with the above-mentioned escalator operation abnormality identification and monitoring method, and the escalator control operation system based on image recognition will not be repeated here.

[0114] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying and monitoring abnormal operation of an escalator, characterized in that: The method comprises the following steps: Acquire motion characteristic data of components inside the escalator, perform motion pattern matching analysis on the component motion characteristic data about the escalator, and determine the mechanical wear variation trend of components inside the escalator; based on the mechanical wear variation trend of components, predict the load and motion offset correlation information of the escalator; Acquire measured load-bearing data and a load state image of the escalator, perform joint analysis on the measured load-bearing data and the load state image, and determine actual motion offset state information of the escalator; perform load dynamic image recognition on the escalator to obtain actual dynamic characteristics of the load on the escalator, thereby determining external action information on the escalator; and determine whether an abnormal operation event will occur on the escalator based on the actual motion offset state information and the external action information; When an abnormal operation event occurs in the escalator, the escalator is subjected to step dynamic visual recognition and load posture visual recognition to obtain escalator stopping parameter information and posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and the posture characteristic information, the movement state of the escalator is adjusted to switch to the stopping mode.

2. The method according to claim 1, characterized in that The acquiring of motion characteristic data of components inside the escalator, performing motion pattern matching analysis on the component motion characteristic data for the escalator, and determining a change trend of mechanical loss of components inside the escalator; and predicting the load and motion offset correlation information of the escalator based on the change trend of mechanical loss of components, are specifically as follows: Obtaining the motion friction and motion separation gap change characteristic data of each of the transmission components inside the escalator, and obtaining the component transmission operation path data of all the transmission components inside the escalator in the corresponding motion mode; Based on the motion friction and motion separation gap variation characteristic data and the component transmission operation path data, modeling and analysis are performed on all transmission components inside the escalator to determine the mechanical loss variation trend of all transmission components inside the escalator under the corresponding motion mode; wherein the mechanical loss variation trend includes the physical structure wear degree variation rate of all transmission components under the corresponding motion mode and the power transmission loss variation rate between other adjacent transmission components; Based on the mechanical loss variation trend of all transmission components in the corresponding motion mode, the motion trajectory offset information of the escalator itself under different load-bearing states is predicted, which is used as the load and motion offset correlation information of the escalator; wherein the motion trajectory offset information includes the offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

3. The method according to claim 1, characterized in that The method of obtaining measured load-bearing data and a load state image of the escalator, jointly analyzing the measured load-bearing data and the load state image to determine the actual motion offset state information of the escalator; and performing load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator, is specifically as follows: Acquire the measured global load-bearing data of the step surface of the escalator and the global load state image of the step surface of the escalator; perform object contour recognition on the global load state image to obtain the load distribution position information of the step surface of the escalator; Based on the load distribution position information, the global load-bearing measured data of the step surface is processed by region identification to obtain the actual load bearing state corresponding to the escalator; the actual load bearing state is compared with the load and motion offset association information of the escalator to determine the actual motion offset state information of the escalator; wherein the actual offset state information of the escalator includes the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom; The escalator is dynamically photographed with load to obtain a global dynamic image of the load on the step surface of the escalator; the load contour position time evolution analysis is performed on the global dynamic image to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; and based on the movement speed and the movement amplitude, the intensity information of the action on the escalator from the load along the specific direction is determined.

4. The method according to claim 3, characterized in that: The determining whether an abnormal operation event will occur in the escalator based on the actual motion offset state information and the external action information is specifically: Based on the actual motion offset state information and the action intensity information, the operation offset recovery state of all the steps under the escalator is identified to determine whether all the steps can be restored to a state without motion offset within a preset time range; if so, it is determined that the escalator will not have an abnormal operation event; if not, it is determined that the escalator will have an abnormal operation event, and the position information of all abnormal steps is determined.

5. The method according to claim 1, characterized in that When an abnormal operation event occurs in the escalator, the escalator is subjected to step dynamic visual recognition and load posture visual recognition to obtain escalator stopping parameter information and posture characteristic information of the load on the escalator; based on the escalator stopping parameter information and the posture characteristic information, the motion state of the escalator switching to the stopping mode is adjusted, specifically: When an abnormal operation event occurs in the escalator, all abnormal steps are tracked and photographed based on the position information of all abnormal steps to obtain dynamic motion images and load dynamic images of each abnormal step; Performing visual recognition on the dynamic motion image to obtain the allowable stopping distance information of the escalator for stopping, which is used as the escalator stopping parameter information; performing visual recognition on the load center of gravity posture of the load dynamic image to obtain the center of gravity offset posture angle information of the load on the abnormal step, which is used as the posture feature information of the load on the escalator; Based on the allowable braking distance information and the center of gravity offset attitude angle information, the braking acceleration when the escalator is switched to the braking mode is adjusted.

6. The escalator control operation system based on image recognition is characterized by: The system includes the following modules: A component wear variation trend determination module is used to obtain component motion characteristic data inside the escalator, perform motion pattern matching analysis on the component motion characteristic data about the escalator, and determine the component mechanical wear variation trend inside the escalator; A load and motion offset correlation prediction module, used to predict the load and motion offset correlation information of the escalator based on the mechanical wear variation trend of the components; An escalator load-bearing and load identification module, used to obtain the measured load-bearing data and load status image of the escalator; An actual motion offset analysis module, used to jointly analyze the load-bearing measured data and the load state image to determine the actual motion offset state information of the escalator; A load dynamic action recognition module is used to perform load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator; An operation abnormality judgment module is used to judge whether an operation abnormality event will occur in the escalator based on the actual motion offset state information and the external action information; An escalator visual recognition module is used to perform step dynamic visual recognition and load posture visual recognition on the escalator when an abnormal operation event occurs on the escalator, and obtain escalator stopping parameter information and posture feature information of the load on the escalator; A braking motion adjustment module is used to adjust the motion state of the escalator when it switches to a braking mode based on the escalator braking parameter information and the posture feature information.

7. The system according to claim 6, characterized in that The component wear variation trend determination module obtains component motion characteristic data inside the escalator, performs motion pattern matching analysis on the component motion characteristic data about the escalator, and determines the component mechanical wear variation trend inside the escalator, specifically: Obtaining the motion friction and motion separation gap change characteristic data of each of the transmission components inside the escalator, and obtaining the component transmission operation path data of all the transmission components inside the escalator in the corresponding motion mode; Based on the motion friction and motion separation gap variation characteristic data and the component transmission operation path data, modeling and analysis are performed on all transmission components inside the escalator to determine the mechanical loss variation trend of all transmission components inside the escalator under the corresponding motion mode; wherein the mechanical loss variation trend includes the physical structure wear degree variation rate of all transmission components under the corresponding motion mode and the power transmission loss variation rate between other adjacent transmission components; The load and motion offset correlation prediction module predicts the load and motion offset correlation information of the escalator based on the mechanical loss change trend of the components, specifically: Based on the mechanical loss variation trend of all transmission components in the corresponding motion mode, the motion trajectory offset information of the escalator itself under different load-bearing states is predicted, which is used as the load and motion offset correlation information of the escalator; wherein the motion trajectory offset information includes the offset information of the motion trajectory of all steps under the escalator in six degrees of freedom.

8. The system according to claim 6, characterized in that The escalator load bearing and load identification module obtains the measured load bearing data and load status image of the escalator, specifically: Acquiring measured global load-bearing data of the step surface of the escalator and a global load state image of the step surface of the escalator; The actual motion offset analysis module performs a joint analysis on the load-bearing measured data and the load state image to determine the actual motion offset state information of the escalator, specifically: Performing object contour recognition on the load global state image to obtain load distribution position information on the step surface of the escalator; Based on the load distribution position information, the global load-bearing measured data of the step surface is processed by region identification to obtain the actual load bearing state corresponding to the escalator; the actual load bearing state is compared with the load and motion offset association information of the escalator to determine the actual motion offset state information of the escalator; wherein the actual offset state information of the escalator includes the actual offset information of the motion trajectory of all steps under the escalator in six degrees of freedom; The load dynamic action identification module performs load dynamic image recognition on the escalator to obtain the actual dynamic characteristics of the load on the escalator, thereby determining the external action information on the escalator, specifically: The escalator is dynamically photographed with load to obtain a global dynamic image of the load on the step surface of the escalator; the load contour position time evolution analysis is performed on the global dynamic image to obtain the movement speed and movement amplitude of the load on the escalator along a specific direction; and based on the movement speed and the movement amplitude, the intensity information of the action on the escalator from the load along the specific direction is determined.

9. The system according to claim 8, characterized in that The operation abnormality judgment module judges whether an operation abnormality event will occur in the escalator based on the actual motion offset state information and the external action information, specifically: Based on the actual motion offset state information and the action intensity information, the operation offset recovery state of all the steps under the escalator is identified to determine whether all the steps can be restored to a state without motion offset within a preset time range; if so, it is determined that the escalator will not have an abnormal operation event; if not, it is determined that the escalator will have an abnormal operation event, and the position information of all abnormal steps is determined.

10. The system according to claim 6, characterized in that When an abnormal operation event occurs on the escalator, the escalator visual recognition module performs step dynamic visual recognition and load posture visual recognition on the escalator to obtain escalator stopping parameter information and posture feature information of the load on the escalator, specifically: When an abnormal operation event occurs in the escalator, all abnormal steps are tracked and photographed based on the position information of all abnormal steps to obtain dynamic motion images and load dynamic images of each abnormal step; Performing visual recognition on the dynamic motion image to obtain the allowable stopping distance information of the escalator for stopping, which is used as the escalator stopping parameter information; performing visual recognition on the load center of gravity posture of the load dynamic image to obtain the center of gravity offset posture angle information of the load on the abnormal step, which is used as the posture feature information of the load on the escalator; The stopping motion adjustment module adjusts the motion state of the escalator switching to the stopping mode based on the escalator stopping parameter information and the posture feature information, specifically: Based on the allowable braking distance information and the center of gravity offset attitude angle information, the braking acceleration when the escalator is switched to the braking mode is adjusted.

Images