Escalator chain lubrication method and system based on visual recognition, and electronic device

Abstract

The present application relates to the field of escalator operation and maintenance technology, and provides an escalator chain lubrication method and system based on visual recognition and an electronic device, which comprises: BGR original image preprocessing and anti-interference correction of the escalator chain to obtain a target image; extracting and screening an effective contour from the target image, fitting the effective contour to generate an effective detection mask area; extracting a statistical feature vector from the effective detection mask area; extracting a semantic feature vector from the effective detection mask area; inputting the statistical feature vector and the semantic feature vector into a multi-task joint lubrication state discrimination network to output a continuous lubrication index and an operation and maintenance grade label with a corresponding relationship; and generating a lubrication control instruction based on the continuous lubrication index and the operation and maintenance grade label. The present application identifies the BGR original image of the escalator chain through machine vision technology, accurately quantifies the chain lubrication state, realizes on-demand, quantitative and fixed-point intelligent lubrication, and reduces equipment wear and maintenance costs.

Description

Technical Field

[0001] This invention relates to the field of escalator operation and maintenance technology, and in particular to a method, system and electronic device for escalator chain lubrication based on vision recognition. Background Technology

[0002] Escalators, as core passenger-carrying special equipment in public places such as shopping malls, train stations, and airports, have their operational safety and stability directly related to public travel safety. The chain is the core transmission component of an escalator, including the main drive chain and the step chains. Operating at high speeds for extended periods, the lubrication condition of the chain directly determines its wear rate, operating noise, and service life, making it a key control point for daily escalator maintenance.

[0003] Currently, the industry generally adopts traditional timed and quantitative automatic oiling or manual periodic oiling operation and maintenance mode for escalator chain lubrication, which has many technical defects. Escalator chain lubrication mostly uses timed and quantitative forced oiling, which has industry pain points such as excessive or insufficient lubrication, leading to pollution, waste, chain wear and abnormal noise, high manual maintenance costs, poor lubrication accuracy, and weak adaptability. Summary of the Invention

[0004] To address the problems of existing escalator chain lubrication methods, such as their crudeness, low precision, lack of real-time detection, inability to adaptively adjust, and the tendency of visual inspection cameras to accumulate oil and dust, suffer from imaging failure, and exhibit poor system stability, this invention aims to provide a visual recognition-based escalator chain lubrication method, system, and electronic equipment. This system enables on-demand, quantitative, and point-based intelligent lubrication, thereby reducing equipment wear and maintenance costs.

[0005] To achieve the above objectives, the first objective of this invention is to provide a method for lubricating escalator chains based on visual recognition, comprising: Preprocessing and anti-interference correction of the raw BGR image of the escalator chain are performed to obtain the target image; Extract and filter valid contours from the target image, and fit the valid contours to generate an effective detection mask region; Extract statistical feature vectors from the effective detection mask region; Extract semantic feature vectors from the effective detection mask region; The statistical feature vector and semantic feature vector are input into the multi-task joint lubrication state discrimination network, and the corresponding continuous lubrication index and maintenance level label are output. Lubrication control instructions are generated based on the continuous lubrication index and maintenance level labels.

[0006] As a further improvement of the present invention, the step of preprocessing and anti-interference correction of the BGR original image of the escalator chain to obtain the target image includes: Perform bilateral filtering on the original BGR image; The original BGR image after bilateral filtering is converted to the LAB color space to obtain a three-dimensional data model; Divide the brightness channels in the 3D model data into non-overlapping sub-blocks; Perform histogram equalization on each sub-block; The brightness channel, green-red axis channel, and blue-yellow axis channel in the 3D data model are merged to obtain the illumination-corrected image; After the illumination-corrected image is converted to the HSV color space, the highlight area is determined by binary analysis using a reflective mask. After repairing the highlights, the target image is obtained.

[0007] As a further improvement of the present invention, the step of extracting and filtering effective contours from the target image, and fitting the effective contours to generate an effective detection mask region includes: Convert the target image to a grayscale image; Use the Canny operator to extract edges from the grayscale image and construct a contour set; Calculate the area of ​​each contour in the contour set; Based on the area, valid outlines are selected; The effective contour is fitted to obtain the target contour; Generate an effective detection mask region based on the target contour.

[0008] As a further improvement of the present invention, the step of extracting statistical feature vectors from the effective detection mask region includes: Based on the effective detection mask area, the oil film coverage is calculated in the HSV color space; Based on the effective detection mask area, the percentage of rust pixels in the HSV color space is calculated; Calculate the gray-level variance of the effective detection mask area; The local binary pattern is used to extract the texture standard deviation of the effective detection mask region; Calculate the proportion of reflective pixels based on the effective detection mask area; After linearly normalizing and scaling the oil film coverage, rust pixel ratio, grayscale variance, texture standard deviation, and reflective pixel ratio, a feature vector is constructed.

[0009] As a further improvement of the present invention, the step of extracting semantic feature vectors from the effective detection mask region includes: The effective detection mask area is cropped into a pure roller effective image; The effective image of the pure roller is input into a lightweight convolutional neural network, which outputs a semantic feature vector.

[0010] As a further improvement of the present invention, the multi-task joint lubrication state discrimination network includes two parallel output branches: a regression branch task and a classification branch task; The multi-task joint lubrication state discrimination network uses a composite loss function to jointly train the regression branch task and the classification branch task.

[0011] As a further improvement to the present invention, after obtaining the corresponding continuous lubrication index and maintenance level label, the invention further includes: The mean filtering process is performed on multiple consecutive lubrication indices and maintenance level labels obtained continuously.

[0012] As a further improvement of the present invention, the present invention also includes: saving the escalator operating status, continuous lubrication index, and maintenance level labels to form a lubrication log.

[0013] A second objective of this invention is to provide a vision-based escalator chain lubrication system for implementing the aforementioned vision-based escalator chain lubrication method, comprising: An image acquisition module is installed on the side of the escalator to acquire raw BGR images of the escalator chain. The visual recognition and analysis module is used to extract and filter valid contours from the target image, fit the valid contours to generate valid detection mask regions, and statistically analyze feature vectors and semantic feature vectors from the valid detection mask regions. The visual recognition and analysis module is also used to output a continuous lubrication index and maintenance level label with corresponding relationships based on statistical feature vectors and semantic feature vectors. The intelligent control module is used to generate lubrication control commands based on the continuous lubrication index and maintenance level labels; The refueling execution module is used to execute lubrication control commands.

[0014] A third objective of this invention is to provide an electronic device, including a processor, a communication interface, a memory, and a network communication component, wherein the processor, the communication interface, and the memory communicate with each other through the network communication component. The memory is used to store computer programs; The processor, when executing the program stored in the memory, implements the above-described escalator chain lubrication method.

[0015] Compared with the prior art, the present invention has the following beneficial effects: by recognizing the original BGR image of the escalator chain using machine vision technology, the chain lubrication status can be accurately quantified, enabling on-demand, quantitative, and fixed-point intelligent lubrication, reducing equipment wear and maintenance costs, improving the safety and intelligence level of escalator operation, and realizing unmanned, precise, and highly stable intelligent operation and maintenance of escalator chain lubrication. Attached Figure Description

[0016] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is a flowchart of the escalator chain lubrication method based on vision recognition described in this invention; Figure 2 This is a schematic diagram of the structure of the visual recognition-based escalator chain lubrication system described in this invention.

[0018] Explanation of reference numerals in the attached diagram: 1. Image acquisition module; 2. Visual recognition and analysis module; 3. Intelligent control module; 4. Refueling execution module; 5. Status monitoring and feedback module. Detailed Implementation

[0019] To make the technical problems solved by the present invention, the technical solutions adopted, and the technical effects achieved clearer, the technical solutions of the embodiments of the present invention will be further described in detail below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention discloses a method for lubricating escalator chains based on visual recognition, such as... Figure 1 As shown, it includes: S1. Preprocessing and anti-interference correction of the original BGR image of the escalator chain to obtain the target image. In this process, a bilateral filtering algorithm is first used to perform edge-preserving noise reduction, filtering out dust particle noise while fully preserving the details of the chain roller edges and oil film texture, avoiding segmentation failure caused by edge blurring. Then, the filtered original BGR image is converted to the LAB color space, and block-based CLAHE adaptive histogram equalization is performed on the brightness channel to achieve dynamic correction of global and local illumination unevenness, improving the oil film feature recognition in dark areas and suppressing overexposure in bright areas. Further, the corrected image is converted to the HSV color space, and metallic highlight reflective areas are extracted based on the brightness channel threshold. A binary reflective mask is constructed, and the highlight areas are reconstructed using neighborhood texture filling and image inpainting algorithms to eliminate the pixel artifacts in the oil film caused by specular reflection, resulting in a target image with uniform illumination and no highlight interference.

[0021] As one implementation method, step S1 includes: S11. For the original BGR image Perform bilateral filtering: ; in, Spatial weights, For grayscale similarity weights, N is... The neighborhood centered on it.

[0022] S12. Convert the original BGR image after bilateral filtering to the LAB color space to obtain the three-dimensional data model. : ; For three-dimensional data models Perform channel decomposition. .

[0023] S13. Divide the brightness channel L in the 3D model data into non-overlapping sub-blocks. .

[0024] S14. For each sub-block Perform histogram equalization: ; S15. Merge the brightness channel L, green-red axis channel A, and blue-yellow axis channel B in the 3D data model to obtain the illumination-corrected image. .

[0025] S16. After the illumination-corrected image is converted to the HSV color space, the highlight area is determined by binary analysis of the reflective mask.

[0026] S17. After repairing the highlights, the target image is obtained. .

[0027] S2. To eliminate interference from invalid regions, this invention employs a structured ROI precise segmentation logic to extract and filter valid contours from the target image, and fits these valid contours to generate an effective detection mask area. Through step S2, invalid pixels such as waste oil and background debris accumulated on both sides of the chain gap can be completely shielded, retaining only the working surface of the rollers involved in equipment friction as the sole detection area, thus solving the industry pain point of misjudging over-lubrication due to oil accumulation in the gaps at its source.

[0028] As one implementation method, step S2 includes: S21. Convert the target image to grayscale: ; S22. Use the Canny operator to extract edges from a grayscale image. Construct a contour set ; S23. Calculate the area of ​​each contour in the contour set. :

[0029] S2, based on area The effective contours are obtained through filtering. Here, the roller area threshold can be preset. ≥800, aspect ratio threshold 0.6≤ r ≤1.6 can filter out abnormal interference contours such as dust, bolts, and trusses.

[0030] S24. Fit the effective contour and obtain the target contour using an ellipse fitting algorithm. Target outline That is, the profile of the roller cylindrical contact surface.

[0031] S25. Generate an effective detection mask region based on the target contour. Specifically, for the target contour... All pixels in the elliptical region are set to 255, and the rest are set to 0. The actual working friction surface of the positioning chain is precisely located to generate a dedicated and effective detection mask area. .

[0032] This invention performs dual-branch parallel feature extraction in steps S3 and S4 to achieve complementary fusion of multi-dimensional lubrication features.

[0033] S3. Extract statistical feature vectors from the effective detection mask region. This is the first branch of interpretable physical feature extraction.

[0034] In one implementation method, step S3 includes: S31. Based on the effective detection mask area, accurately segment the amber area of ​​the lubricating oil in the HSV color space, calculate the oil film coverage, and first, the lubricating oil HSV interval... Oil film pixel set Then the oil film coverage (first dimension feature) ,in This represents the number of effective pixels.

[0035] S32. Based on the effective detection mask area, accurately segment the reddish-brown rust region in the HSV color space, calculate the rust pixel ratio, and rust interval. Rust pixel set Then the percentage of rust pixels (second dimension feature) ; S33. Calculate the gray-level variance of the effective detection mask region and the set of effective gray-level pixels. mean Then the gray-level variance (third dimension) .

[0036] S34. Extract the standard deviation of texture in the effective detection mask region (fourth dimension) using local binary pattern: .

[0037] S35. Calculate the proportion of reflective pixels based on the effective detection mask area (fifth dimension). ,in, .

[0038] S36. Perform linear normalization scaling on oil film coverage, rust pixel ratio, grayscale variance, texture standard deviation, and reflective pixel ratio. Then, construct a multidimensional statistical feature vector, where The original statistics are: total pixels of oil film, total pixels of rust, mean saturation, maximum and minimum brightness, and median grayscale. A fixed scaling factor is used. It comprehensively quantifies oil film uniformity, dryness, and corrosion interference; all features are based on optical-physical properties, exhibiting extremely high stability and interpretability.

[0039] S4. Extract semantic feature vectors from the effective detection mask region. This is the lightweight semantic feature branch.

[0040] Specifically, a mini-MobileNet network based on depthwise separable convolution (DW convolution) is constructed to extract deep semantic features from the effective images of pure rollers after cropping the effective detection mask region. Oil film edges and dry / wet texture details are extracted through shallow convolution, and a 128-dimensional high-dimensional latent semantic feature vector is obtained through global pooling compression. This automatically learns the differentiated features of the oil film under complex dust accumulation, heavy corrosion, and old wear chain scenarios, overcoming the shortcomings of traditional manual feature generalization.

[0041] S5. Input the statistical feature vector and semantic feature vector into the multi-task joint lubrication state discrimination network, and output the continuous lubrication index and maintenance level label with corresponding relationship. That is, step S5 constructs a dual-task discrimination network to achieve dual output of quantification and classification.

[0042] One branch is the lubrication regression task. ,in This is the weight matrix. This is a multidimensional image fusion feature vector obtained by fusing statistical feature vectors and semantic feature vectors. As the bias vector, the output is a continuous lubrication index in the range of 0-100, mapped by the sigmoid activation function. The numerical value precisely corresponds to the integrity of the oil film coverage and the quality of lubrication, enabling refined quantitative evaluation.

[0043] The other branch is a three-level classification task, which outputs three maintenance level labels: severe oil shortage, normal lubrication, and excessive lubrication, which directly match the equipment maintenance and repair standards.

[0044] Finally, the multi-task joint lubrication state discrimination network employs a composite loss function for joint training of the regression and classification branches. Specifically, a weighted joint optimization method using regression and classification losses is adopted. ,in To regress the MSE loss, For classification crossover loss, To correct the reflection loss of rust spots, while ensuring the accuracy of the quantitative index and the accuracy of the classification judgment, the feature interference weights caused by rust and residual oil stains are suppressed.

[0045] S6. Generate lubrication control instructions based on lubrication index and maintenance level labels.

[0046] To address the instability issue in single-frame image detection, this invention introduces a multi-frame sliding window temporal smoothing mechanism, setting the sliding window queue length. Continuous storage Frame original lubrication index Timing smoothing output of final lubrication index .

[0047] The lubrication index from multiple consecutive frames is averaged to eliminate jumps in results caused by instantaneous sunlight and dust disturbances. The maintenance level is then determined based on the smoothed, stable lubrication index.

[0048] S7 saves the escalator's operating status, continuous lubrication index, and maintenance level tags to form a lubrication log. It also saves core parameters such as oil film coverage and rust pixel ratio, generating structured detection results. It can directly connect to the industrial IoT platform to realize intelligent operation and maintenance functions such as lubrication status trend statistics, oil shortage warning, and automatic generation of maintenance work orders.

[0049] In summary, this invention offers the following advantages: By integrating anti-interference correction-based visual preprocessing with a multi-task joint discrimination network, it achieves precise quantitative assessment and intelligent control of escalator chain lubrication status. By extracting statistical and semantic features and inputting them into the network, it can output continuous lubrication indices and maintenance level labels, thereby dynamically generating lubrication commands and completely overcoming the blindness of traditional timed and quantitative or manual lubrication methods. This solution effectively solves the problems of oil waste and environmental pollution caused by excessive lubrication, while avoiding chain wear and abnormal noise caused by insufficient lubrication, significantly reducing maintenance costs. Furthermore, this method has strong environmental adaptability, achieving precise lubrication on demand, greatly extending the service life of core components, ensuring the safety and stability of escalator operation, and promoting the transformation of escalator maintenance towards intelligence and refinement.

[0050] Based on the same inventive concept, this invention provides a vision-based escalator chain lubrication system, such as... Figure 2 As shown, a vision-based escalator chain lubrication method for implementing the above-mentioned method includes: an image acquisition module, a vision recognition analysis module, an intelligent control module, a lubrication execution module, and a status monitoring feedback module. The image acquisition module is positioned to the side of the escalator chain's operating area to acquire raw BGR images of the escalator chain. The vision recognition analysis module extracts and filters effective contours from the target image, fits these contours to generate effective detection mask areas, and extracts statistical and semantic feature vectors from the effective detection mask areas. The vision recognition analysis module also outputs a corresponding continuous lubrication index and maintenance level label based on the statistical and semantic feature vectors. The intelligent control module generates lubrication control commands based on the continuous lubrication index and maintenance level label. The lubrication execution module executes the lubrication control commands. The status feedback module records the escalator's operating status and lubrication effect level in real time, feeding this information back to the intelligent control module to form a closed-loop control system. Simultaneously, it records the escalator's operating status, continuous lubrication index, and maintenance level label to form a lubrication log.

[0051] Lubrication control commands include: oil flow rate, oiling interval parameters, etc.

[0052] Preferably, the image acquisition module includes a high-definition high-speed industrial camera, a supplementary light source, a dustproof protective housing, and an image transmission unit; the high-definition industrial camera uses an adjustable focus wide-angle lens, aimed at the meshing and running area of ​​the escalator chain, and the supplementary light source is an adjustable brightness ring-shaped cold light source, used to eliminate ambient light and shadow interference and ensure the clarity of the chain image acquisition.

[0053] For example, when the visual recognition analysis module analyzes that the continuous lubrication index of the step chain is 30, which belongs to the maintenance level label of slightly insufficient oil, when the intelligent control module receives the continuous lubrication index value and maintenance level label, and at the same time receives that the escalator is in operation, it generates a lubrication control command, starts the solenoid valve and oil pump of the oiling execution module, and lubricates the step chain until the continuous lubrication index of the step chain is about 70 (normal lubrication) and then stops the oiling.

[0054] When there is insufficient lighting, the image acquisition module will automatically supplement the light to solve problems such as uneven lighting, reflections from oily surfaces, and shadows, replacing the traditional fixed Gamma correction.

[0055] The refueling execution module includes an oil pump and an injection assembly. The intelligent control module drives the oil pump to work with the injection assembly to complete fixed-point, quantitative, and on-demand lubrication.

[0056] Based on the same inventive concept, the present invention also provides an electronic device, including a processor, a communication interface, a display terminal, a memory, and a network communication component, wherein the processor, the communication interface, the display terminal, and the memory communicate with each other through the network communication component; the memory is used to store computer programs; and the processor is used to implement the above-mentioned escalator chain lubrication method when executing the program stored in the memory.

[0057] The network communication component mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This network communication component can be divided into address bus, data bus, control bus, etc. The communication interface is used for communication between the aforementioned terminal and other devices.

[0058] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0059] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), graphics processing units (GPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0060] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style of the specification is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0061] The technical principles of the present invention have been described above with reference to specific embodiments. These descriptions are merely for explaining the principles of the invention and should not be construed as limiting the scope of protection of the invention in any way. Based on this explanation, those skilled in the art can readily conceive of other specific embodiments of the invention without inventive effort, and these embodiments will all fall within the scope of protection of the present invention.

Claims

1. A method for lubricating escalator chains based on visual recognition, characterized in that, include: Preprocessing and anti-interference correction of the raw BGR image of the escalator chain are performed to obtain the target image; Extract and filter valid contours from the target image, and fit the valid contours to generate an effective detection mask region; Extract statistical feature vectors from the effective detection mask region; Extract semantic feature vectors from the effective detection mask region; The statistical feature vector and semantic feature vector are input into the multi-task joint lubrication state discrimination network, and the corresponding continuous lubrication index and maintenance level label are output. Lubrication control instructions are generated based on the continuous lubrication index and maintenance level labels.

2. The escalator chain lubrication method according to claim 1, characterized in that, The steps of preprocessing and anti-interference correction of the BGR original image of the escalator chain to obtain the target image include: Perform bilateral filtering on the original BGR image; The original BGR image after bilateral filtering is converted to the LAB color space to obtain a three-dimensional data model; Divide the brightness channels in the 3D model data into non-overlapping sub-blocks; Perform histogram equalization on each sub-block; The brightness channel, green-red axis channel, and blue-yellow axis channel in the 3D data model are merged to obtain the illumination-corrected image; After the illumination-corrected image is converted to the HSV color space, the highlight area is determined by binary analysis using a reflective mask. After repairing the highlights, the target image is obtained.

3. The escalator chain lubrication method according to claim 1, characterized in that, The steps of extracting and filtering valid contours from the target image, and fitting the valid contours to generate a valid detection mask region include: Convert the target image to a grayscale image; Use the Canny operator to extract edges from the grayscale image and construct a contour set; Calculate the area of ​​each contour in the contour set; Based on the area, valid outlines are selected; The effective contour is fitted to obtain the target contour; Generate an effective detection mask region based on the target contour.

4. The escalator chain lubrication method according to claim 1, characterized in that, The step of extracting statistical feature vectors from the effective detection mask region includes: Based on the effective detection mask area, the oil film coverage is calculated in the HSV color space; Based on the effective detection mask area, the percentage of rust pixels in the HSV color space is calculated; Calculate the gray-level variance of the effective detection mask area; The local binary pattern is used to extract the texture standard deviation of the effective detection mask region; Calculate the proportion of reflective pixels based on the effective detection mask area; After linearly normalizing and scaling the oil film coverage, rust pixel ratio, grayscale variance, texture standard deviation, and reflective pixel ratio, a feature vector is constructed.

5. The escalator chain lubrication method according to claim 1, characterized in that, The step of extracting semantic feature vectors from the effective detection mask region includes: The effective detection mask area is cropped into a pure roller effective image; The effective image of the pure roller is input into a lightweight convolutional neural network, which outputs a semantic feature vector.

6. The escalator chain lubrication method according to claim 1, characterized in that, The multi-task joint lubrication state discrimination network includes two parallel output branches: a regression branch task and a classification branch task. The multi-task joint lubrication state discrimination network uses a composite loss function to jointly train the regression branch task and the classification branch task.

7. The escalator chain lubrication method according to claim 1, characterized in that, After obtaining the corresponding continuous lubrication index and maintenance level label, the following is also included: The mean filtering process is performed on multiple consecutive lubrication indices and maintenance level labels obtained continuously.

8. The escalator chain lubrication method according to claim 1, characterized in that, Also includes: Save the escalator's operating status, continuous lubrication index, and maintenance level label to form a lubrication log.

9. A visual recognition-based escalator chain lubrication system, characterized in that, A method for implementing a vision-based escalator chain lubrication system as described in any one of claims 1 to 8 includes: An image acquisition module is installed on the side of the escalator to acquire raw BGR images of the escalator chain. The visual recognition and analysis module is used to extract and filter valid contours from the target image, fit the valid contours to generate valid detection mask regions, and statistically analyze feature vectors and semantic feature vectors from the valid detection mask regions. The visual recognition and analysis module is also used to output a continuous lubrication index and maintenance level label with corresponding relationships based on statistical feature vectors and semantic feature vectors. The intelligent control module is used to generate lubrication control commands based on the continuous lubrication index and maintenance level labels; The refueling execution module is used to execute lubrication control commands.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a network communication component, wherein the processor, the communication interface, and the memory communicate with each other through the network communication component; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the escalator chain lubrication method as described in any one of claims 1-8.

Images