Multi-area information fusion escalator speed monitoring method

Abstract

The invention provides a multi-area information fusion escalator speed monitoring method. The problems that a traditional sensor is complex in installation and a general vision algorithm is insufficient in robustness are solved through a non-contact detection means. The method specifically comprises the steps that multiple detection areas are divided in a horizontal section and an inclined section of the escalator, a speed correlation model is established based on a kinematics constraint relation, and dynamic speed calculation in a shielding scene is achieved; yellow warning frame features are extracted through a centroid tracking algorithm, and the detection precision is improved in combination with boundary crossing judgment conditions and a dynamic time compensation mechanism; a redundancy detection process and multi-region cooperative verification are designed, the system reliability under complex illumination and stain interference is guaranteed, and a complete technical closed loop from data acquisition, algorithm processing to safety early warning is formed.

Description

Technical Field

[0001] The present invention belongs to the technical fields of escalator safety monitoring and industrial application of machine vision, and particularly relates to a multi-region information fusion escalator speed monitoring method. Background Art

[0002] In the field of escalator safety monitoring, existing speed detection technologies are mainly divided into two categories: contact and non-contact. Contact methods usually rely on devices such as speed measuring wheels, laser markers, or mechanical sensors to obtain motion data by directly contacting escalator components (such as handrails or steps). Although such methods can achieve a certain degree of measurement accuracy, they have the problems of complex installation and susceptibility to mechanical wear or contact surface contamination. Long-term use may lead to error accumulation and it is difficult to adapt to the rapid deployment requirements of different escalator models. In addition, contact equipment may interfere with the operation of the escalator during the detection process, posing a safety hazard.

[0003] Among non-contact detection technologies, machine vision-based solutions are gradually becoming mainstream, using image processing algorithms to analyze the operating status of escalators. For example, some existing technologies calculate speed deviations by placing specific markers (such as colored stickers, light spots, etc.) on handrails or steps using color segmentation, contour recognition, or optical flow tracking. However, such methods are highly dependent on markers and are easily affected by factors such as lighting changes, obstruction by stains, or marker shedding in actual applications, resulting in unstable feature extraction. At the same time, dynamic target tracking algorithms in complex scenarios are computationally intensive and lack real-time performance, making it difficult to meet the requirements of high-precision and high-robustness detection.

[0004] Furthermore, existing technologies often focus on detecting a single area or single parameter, such as independent analysis of handrail or step speed, lacking multi-dimensional collaborative monitoring of the escalator's overall operating status. In abnormal situations such as occlusion and data loss, existing solutions are often unable to dynamically compensate for the failed area through multi-source data fusion, resulting in reduced reliability of detection results. Therefore, achieving non-contact, high-precision, and anti-interference escalator speed monitoring in complex environments remains a technical challenge that urgently needs to be addressed in this field. Summary of the Invention

[0005] In view of the defects and deficiencies existing in the prior art, the present invention provides an escalator speed monitoring method and system based on multi-region kinematic modeling and machine vision, which solves the problems of complex installation of traditional sensors and insufficient robustness of general vision algorithms through non-contact detection means. Specifically, it includes: dividing multiple detection regions in the horizontal section and inclined section of the escalator, establishing a speed correlation model based on kinematic constraint relationships to realize dynamic speed calculation in occlusion scenarios; extracting the characteristics of the yellow warning border with the centroid tracking algorithm, and combining the boundary crossing determination condition and the dynamic time compensation mechanism to improve the detection accuracy; designing a redundant detection process and multi-region collaborative verification to ensure the system reliability under complex lighting and stain interference, and forming a complete technical closed-loop from data acquisition, algorithm processing to safety warning.

[0006] The technical solution specifically adopted by the present invention to solve its technical problems is as follows:

[0007] A multi-region information fusion escalator speed monitoring method:

[0008] Set multiple detection regions in different operating sections of the escalator, and the operating sections include a horizontal section and an inclined section;

[0009] Based on the kinematic constraint relationships of different operating sections, establish a speed correlation model between regions;

[0010] When the speed detection in some regions is interrupted due to occlusion or data failure, based on the speed correlation model, dynamically calculate the speed value of the failed region through the speed data of other regions.

[0011] Further, the kinematic constraint relationship is realized through a linear proportional model, and the proportional coefficient is calibrated based on the multi-region speed data in the non-occlusion scenario.

[0012] Further, the different operating sections include the upper horizontal section, the middle inclined section and the lower horizontal section of the escalator;

[0013] Each section respectively covers the running tracks of the steps, the left handrail belt and the right handrail belt.

[0014] Further, set a determination boundary line at the center of each detection region, and judge the displacement state of the detection target through the change rate of the directed distance between the centroid point and the boundary line;

[0015] When the centroid point crosses the boundary from the positive side, negative side or straight line position, trigger the determination of the completion of the fixed-spacing displacement;

[0016] The running speed is calculated based on the fixed spacing of the detection target and the time difference between adjacent frames;

[0017] When the centroid point does not completely cross the boundary, dynamically correct the time difference according to the distance ratio from the centroid to the boundary, and the correction formula is:

[0018]

[0019] where d i and d i+1 are the directed distances from the centroid points of the i-th frame and the (i + 1)-th frame to the determination boundary line respectively; fps is the image acquisition frame rate.

[0020] Furthermore, when extracting the color features of the detection target and generating the mask, a color threshold range is set for the yellow warning border of the escalator, and noise interference is removed;

[0021] Edge detection is performed on the mask area, and after screening the largest connected area, the centroid coordinates are calculated.

[0022] As a preferred solution, the proportionality coefficient is determined by fitting the multi-region speed data in the unobstructed scenario using the least squares method.

[0023] Furthermore, when the percentage deviation between the left and right handrails and the step speed exceeds the preset threshold, an escalator safety warning is triggered;

[0024] When the speed calculation error of continuous n (n is a preset value, greater than or equal to 1) regions exceeds the threshold, a redundant detection process is started, and the redundant detection process includes:

[0025] Switch to the standby detection area to re-acquire data;

[0026] Or enable the auxiliary sensor for cross-verification.

[0027] Furthermore, the detection target is the yellow warning border on the escalator steps and handrails, and the chromaticity parameters of the yellow warning border are set based on the preset hue, saturation, and lightness threshold ranges;

[0028] The image acquisition device is deployed at the escalator entrance and exit, and the viewing angle covers the horizontal and inclined section running tracks of the steps and handrails to ensure that the detection target is continuously visible in the image.

[0029] And, a multi-region information fusion escalator speed monitoring device, comprising:

[0030] Image acquisition module: Deployed at the escalator entrance and exit, used to acquire the running images of the steps and handrails, and the viewing angle covers the horizontal and inclined sections;

[0031] Region division module: Divide the detection regions of the upper horizontal section, middle inclined section, and lower horizontal section in the image, and each section covers the steps, left handrail, and right handrail;

[0032] Kinematics modeling module: Establish a speed correlation model based on the kinematic constraint relationships of different sections;

[0033] Dynamic compensation module: When the centroid point does not completely cross the determination boundary, dynamically correct the time difference according to the distance ratio;

[0034] Early warning and redundancy module: Trigger a safety warning according to the speed deviation, and switch the detection area or enable the auxiliary sensor when the continuous area error exceeds the limit.

[0035] Moreover, an electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method described above are implemented.

[0036] A non-transitory computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the steps of the method described above are implemented.

[0037] Compared with the prior art, the present invention and its preferred solutions at least include the following beneficial effects:

[0038] Multi-region fusion and kinematic modeling: Through multi-region division of horizontal and inclined segments and speed correlation models, significantly improve the data robustness in occlusion scenarios and avoid misjudgment caused by single-region detection failure;

[0039] Dynamic compensation and boundary determination optimization: Based on the refined determination logic of centroid crossing the boundary and dynamic correction of time difference, reduce the detection error of low-frame-rate devices and adapt to low-cost hardware deployment;

[0040] Redundancy detection and function expansion: Through multi-region collaborative verification and sensor cross-checking, ensure the continuity of speed detection, and at the same time provide reliable data support for escalator safety early warning;

[0041] Flexibility of hardware deployment: The camera is fixed at the entrance and exit to cover the entire operation trajectory, without the need to modify the main structure of the escalator, reducing the implementation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The following further describes the present invention in detail with reference to the drawings and specific embodiments:

[0043] Figure 1 It is a schematic diagram of the core mechanism of the embodiment solution of the present invention;

[0044] Figure 2 It is a schematic diagram of the implementation manner of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] To make the features and advantages of the present invention more obvious and understandable, specific embodiments are hereinafter given and described in detail as follows:

[0046] It should be noted that the following detailed description is illustrative and aims to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0048] As Figure 1 shown, an embodiment of the present invention provides a multi-region information fusion escalator speed monitoring method, and its core design includes:

[0049] Set multiple detection regions in different running sections of the escalator, where the running sections include horizontal sections and inclined sections;

[0050] Based on the kinematic constraint relationships of different running sections, establish a speed correlation model between regions;

[0051] When the speed detection in some regions is interrupted due to occlusion or data failure, based on the speed correlation model, dynamically calculate the speed value of the failed region through the speed data of other regions.

[0052] As Figure 2 shown, as a specific implementation method for reference in this embodiment, generally consider placing the camera at the entrance and exit of the escalator, with a viewing angle that can capture the entire running process of the escalator. The detection target for the steps is a yellow warning border, the depth of the steps is L, and yellow borders with equal spacing D are added to the left and right handrails as detection targets.

[0053] Then execute the following specific implementation process and logic:

[0054] 1. Set regions of interest A, B, and C on the upper horizontal running sections of the steps, left and right handrails respectively, with corresponding speeds Va, Vb, and Vc; set regions of interest A1, B1, and C1 on the middle inclined running section, with corresponding speeds Va1, Vb1, and Vc1; set regions of interest A2, B2, and C2 on the lower horizontal running section, with corresponding speeds Va2, Vb2, and Vc2; the centers of the regions are all set with a straight line determination boundary, and the straight line equation is Ax + By + C = 0;

[0055] 2. Taking the processing of the image of region of interest A as an example, first perform filtering processing to remove image noise; extract the yellow region mask, search for the edges of the yellow region, and calculate the centroid point of the yellow region;

[0056] 3. The coordinate value u of the center of mass of each image is taken as the vertical axis, and the time point t of the shooting is taken as the horizontal axis, that is, a set of discrete points (u i , t i )(i=1,2,3…,m). Let the directed distance d from the center point to the decision boundary line, and the distance change rate V d , the following formula is given:

[0057]

[0058]

[0059] Let S i =Ax i +By i +C,S i+1 =Ax i +1+By i +1+C, if S i >0, the center of mass is on the positive side of the straight line; S i <0, the center of mass is on the negative side of the straight line; S i = 0, the centroid is on the straight line. The conditions for the centroid crossing the boundary are:

[0060] ①When d i <0,d i+1 >0 and V d >0, that is, the centroid crosses the boundary from the negative side;

[0061] ②When d i >0,d i+1 <0 and V d <0, that is, the center of mass crosses the boundary from the positive side;

[0062] ③The center of mass of the i-th frame is on the straight line (d i =0), when d i+1 >0 and V d >0, that is, the center of mass crosses the boundary from the straight line position to the positive side;

[0063] ④ The center of mass of the i-th frame is on the straight line (d i =0), when d i+1 <0 and V d <0, that is, the center of mass crosses the boundary from the straight line position to the negative side;

[0064] ⑤ The center of mass of the i+1th frame is on the straight line (d i+1 =0), when d i <0 and V d >0, that is, the centroid reaches the boundary from the negative side;

[0065] ⑥ The center of mass of the i+1th frame is on the straight line (d i+1= 0), when d i > 0 and V d < 0, that is, the centroid reaches the boundary from the positive side;

[0066] 4. According to the above judgment conditions, when the image centroid eigenvalue crosses the judgment boundary, that is, the escalator runs through the depth L of one step (the fixed distance D corresponding to the yellow tape in the handrail belt), the time difference t i - t0, then the speed is calculated as shown in the following formula.

[0067]

[0068] 5. When the above judgment conditions are in cases ③④⑤⑥, no time compensation is required; when the above judgment conditions are in cases ①②, time compensation T is required,

[0069]

[0070]

[0071]

[0072] In the formula: fps is the image acquisition frame rate.

[0073] 6. According to the descriptions in steps 2 - 5 above, calculate the speeds Va, Vb, Vc, Va1, Vb1, Vc1, Va2, Vb2, Vc2 respectively, and find the corresponding relationships between the speeds in the upper and lower regions and the speeds in the inclined region, that is, Va1 = k1Va, Vb1 = k2Vb, Vc1 = k3Vc; Va1 = k4Va2, Vb1 = k5Vb2, Vc1 = k6Vc2. In this way, even if there is occlusion in a certain region, resulting in the loss of speed detection information, it can be obtained through the speeds in other regions, thereby reducing the influence of occlusion.

[0074] 7. The calculation formulas for the speed deviation of the escalator handrail belt and the speed deviations δ between the left and right handrail belts and the step running speed are as follows:

[0075]

[0076] In the formula: δ is the handrail belt speed deviation; v1 is the step speed; v2 / v3 are the left and right handrail belt speeds respectively. The calculation principles for the left and right handrail belt speeds are the same as those for the steps.

[0077] Based on the same inventive concept, the present invention further provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is configured to execute the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is used to implement one or more instructions. Specifically, it is used to load and execute one or more instructions in the computer storage medium to implement the above method.

[0078] It should be further noted that, based on the same inventive concept, the present invention further provides a computer storage medium, on which a computer program is stored, and the computer program executes the above method when run by a processor. The storage medium may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may, for example, but not be limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device.

[0079] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the field to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. Words such as "comprising" or "including" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. Words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0080] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention in any other form. Any person skilled in the art may use the technical content disclosed above to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

[0081] The present invention is not limited to the above best mode. Anyone can obtain various other forms of multi-region information fusion escalator speed monitoring methods under the inspiration of the present invention. All equal changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope covered by the present invention.

Claims

1. A multi - region information fusion method for escalator speed monitoring, characterized in that: Multiple detection regions are set in different operating sections of the escalator, and the operating sections include a horizontal section and an inclined section; Based on the kinematic constraint relationships of different operating sections, a speed correlation model between regions is established; When the speed detection in some regions is interrupted due to occlusion or data failure, based on the speed correlation model, the speed value of the failed region is dynamically calculated through the speed data of other regions.

2. The multi-region information fusion escalator speed monitoring method according to claim 1, wherein: The kinematic constraint relationship is realized through a linear proportional model, and the proportional coefficient is calibrated based on multi - region speed data in an unoccluded scenario.

3. The multi - region information fusion method for escalator speed monitoring according to claim 1, characterized in that: The different operating sections include the upper horizontal section, the middle inclined section and the lower horizontal section of the escalator; Each section respectively covers the running tracks of the steps, the left handrail belt and the right handrail belt.

4. The multi - region information fusion method for escalator speed monitoring according to claim 1, characterized in that: A determination boundary line is set at the center of each detection region, and the displacement state of the detection target is judged by the rate of change of the directed distance between the centroid point and the boundary line; When the centroid point crosses the boundary from the positive side, negative side or the straight - line position, the determination of the completion of the fixed - pitch displacement is triggered; The running speed is calculated based on the fixed pitch of the detection target and the time difference between adjacent frames; When the centroid point does not completely cross the boundary, the time difference is dynamically corrected according to the distance ratio from the centroid to the boundary, and the correction formula is: where d i and d i+1 are the directed distances from the centroid points of the i-th frame and the (i + 1)-th frame to the decision boundary line respectively; fps is the image acquisition frame rate.

5. The multi - region information fusion method for escalator speed monitoring according to claim 4, characterized in that: The color features of the detection target are extracted. When generating the mask, a color threshold range is set for the yellow warning border of the escalator, and noise interference is removed; Edge detection is performed on the mask region, and the centroid coordinates are calculated after screening the largest connected region.

6. The multi-region information fusion escalator speed monitoring method according to claim 2, characterized in that: The proportional coefficient is determined by least - squares fitting of multi - region speed data in an unoccluded scenario.

7. The multi - region information fusion method for escalator speed monitoring according to claim 1, characterized in that: When the percentage deviation between the left and right handrail belts and the step speed exceeds a preset threshold, an escalator safety warning is triggered; When the speed calculation error in n consecutive regions exceeds the threshold, a redundant detection process is started, and the redundant detection process includes: switching to a standby detection region to re - collect data; Or enabling an auxiliary sensor for cross - verification.

8. The multi - region information fusion method for escalator speed monitoring according to claim 3, characterized in that: The detection target is the yellow warning border on the escalator steps and handrail belts, and the chromaticity parameters of the yellow warning border are set based on a preset hue, saturation and lightness threshold range; The image acquisition device is deployed at the escalator entrance and exit, and the viewing angle covers the horizontal and inclined section running tracks of the steps and handrail belts to ensure that the detection target is continuously visible in the image.

9. An escalator speed monitoring device for multi-region information fusion, characterized in that, Including: Image acquisition module: Deployed at the escalator entrance and exit, used to collect the running images of the steps and handrail belts, and the viewing angle covers the horizontal section and the inclined section; Region division module: Detecting regions of the upper horizontal segment, the middle inclined segment, and the lower horizontal segment are divided in the image, and each segment covers the steps, the left handrail belt, and the right handrail belt; Kinematics modeling module: Establish a speed correlation model based on the kinematic constraint relationships of different segments; Dynamic compensation module: When the centroid point does not completely cross the determination boundary, dynamically correct the time difference according to the distance ratio; Early warning and redundancy module: Trigger a safety warning according to the speed deviation, and switch the detection area or enable auxiliary sensors when the continuous area error exceeds the limit.

10. A non-transitory computer-readable storage medium storing computer program instructions, characterized in that, When the instructions are executed by a processor, the method according to any one of claims 1-8 is implemented.

Images