Elevator buffer compression state monitoring system based on machine vision

Abstract

The invention discloses an elevator buffer compression state monitoring system based on machine vision, and relates to the technical field of elevator safety. According to the elevator buffer compression state monitoring system based on machine vision, a buffer period image sequence is collected through a double-low-light camera and a double-condition linkage triggering mechanism, and after images are subjected to denoising, enhancement processing, distortion correction and cutting, the compression state of the buffer is monitored; the method comprises the following steps: marking a key point location and expanding an abnormal sample, determining an edge frame and a deformation area by using a YOLOv8n lightweight target detection model, extracting key features, training the model according to MobileNetV2 in combination with an SE attention mechanism to detect abnormality, performing secondary judgment by linking a knowledge base when suspected normal is detected, and updating the model and the knowledge base at regular intervals by an optimization module. And the communication module preferentially uploads high-risk data according to the abnormal level and performs graded response, so that accurate early warning and real-time monitoring of the compression state of the buffer are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of elevator safety technology, and in particular to an elevator buffer compression state monitoring system based on machine vision. BACKGROUND

[0002] The elevator buffer is a safety protection device to protect against the elevator from hitting the top and squatting the bottom. Real-time monitoring and early warning of the state of the elevator buffer before and after compression is essential to avoid the failure of the safety protection device in the event of a car falling, impact buffer safety accident. There are significant technical shortcomings in the current elevator buffer monitoring field. On the one hand, the traditional monitoring method mainly relies on manual regular inspection, which is not only inefficient and time-consuming, but also prone to errors in subjective judgment by humans and the limitations of the dark and narrow environment of the elevator pit, making it difficult to detect subtle deformation and hidden cracks in the buffer. On the other hand, the existing visual monitoring scheme mainly uses a single camera to collect images, which may result in insufficient image contrast in low-light environments, and the lens distortion is not systematically corrected, resulting in a large deviation between the buffer image coordinates and the actual physical coordinates, which directly affects the feature extraction accuracy. In addition, there is a high safety risk for the inspector to observe the compression state in the pit during the test, At the same time, the existing scheme faces the problem of sample imbalance, and the scarcity of abnormal samples limits the model training effect. Moreover, there is a lack of knowledge base support and secondary judgment mechanism for integrating historical cases, which may lead to misjudgment and missed judgment. In addition, there is no standardized process for model parameter updating and knowledge base iteration, making it difficult to adapt to new buffer structures and new types of abnormalities. The abnormal response is not classified, which may either lead to a delay in handling high-risk abnormalities and cause risks, or cause excessive downtime due to non-high-risk abnormalities and affect the normal operation of the elevator, severely restricting the safety and maintenance efficiency of the buffer monitoring. SUMMARY

[0003] The present application aims to provide an elevator buffer compression state monitoring system based on machine vision to solve the problems in the prior art.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solution: an elevator buffer compression state monitoring system based on machine vision, the system comprising an image acquisition module, an image preprocessing module, a data labeling and enhancement module, a feature extraction module, an anomaly detection module, a knowledge base module, an optimization module, and a communication module. The image acquisition module uses a dual low-light camera and a "dual condition linkage" trigger mechanism to continuously acquire buffer images throughout the process and add timestamps when the elevator meets the trigger mechanism, thereby constructing an image sequence. After image denoising, the image preprocessing module uses the Retinex algorithm to improve the image contrast, calculates the distortion coefficient, corrects the lens distortion according to the distortion coefficient, and crops the image. The data labeling and enhancement module selects key points of the buffer to establish a coordinate system, calculates relevant parameters, sets label judgment criteria, labels historical images according to the label judgment criteria, enhances the data, and expands abnormal samples through the SMOTE algorithm. The feature extraction module uses the target detection model to determine the buffer edge box and the deformation area, corrects the image coordinates to physical coordinates according to the distortion coefficient, and then extracts key features after correction. The anomaly detection module uses a specific basic network combined with the SE attention mechanism to train the model with samples generated by balanced sample units. The model is used to judge elevator anomalies, and when it is suspected to be normal, it switches to a secondary judgment. The knowledge base module integrates buffer features and historical cases, formulates anomaly rules to build a knowledge base and sets up data indexes, and compares features with knowledge base information when a situation is suspected to be normal to complete a secondary judgment. The optimization module is responsible for optimizing the model and updating the knowledge base; The communication module is responsible for collecting and encapsulating monitoring data, uploading abnormal data, storing normal data locally, and taking corresponding measures for different anomalies.

[0005] The image acquisition module includes an acquisition unit and an acquisition trigger unit; The acquisition unit uses dual low-light cameras. The low-light cameras are installed facing the elevator buffer. The camera bodies are adjusted with a level to ensure that both cameras are horizontal. The angle between the lens optical axis and the central axis of the buffer is ≤5° for camera deployment. The acquisition triggering unit adopts a "dual-condition linkage" mechanism. The "dual-condition linkage" mechanism pre-starts the camera when the elevator acceleration exceeds 0.5g and triggers the acquisition process when the elevator descends to the bottom. It continuously acquires images of the entire process of buffer compression and timestamps them to construct a buffer image sequence.

[0006] The image preprocessing module includes an image cleaning unit, a contrast enhancement unit, a distortion correction unit, and a cropping unit; The image cleaning unit converts the image into a grayscale image using a weighted average method. A 3×3 sliding window is set and slides the window pixel by pixel on the image. Grayscale values ​​are extracted for the nine pixels in each window. The nine grayscale values ​​in the window are sorted in ascending order, and the grayscale value of the center pixel of the window is replaced by the middle value of the sorted values. For the image edge area, virtual pixels outside the edge are supplemented by a mirror filling method. The contrast enhancement unit is equipped with an ambient light sensor, which decomposes the pixel coordinates I(x,y) of the image into the reflection component R(X,Y) of the object structure and the illumination component L(x,y) of the ambient light distribution, and outputs the enhanced image.

[0007] The distortion correction unit automatically identifies the intersection points of the checkerboard grid for each acquired calibration board image using a corner detection algorithm, records the correspondence between the image coordinates of the corner points and the world coordinates of the calibration board, and uses the mathematical model of Zhang Zhengyou's calibration method to calculate the camera's intrinsic and extrinsic parameter matrices through least squares fitting, solves for the lens's distortion coefficient, and calculates the ideal pixel coordinates after distortion-free processing for each pixel in the image based on the distortion coefficient and the distortion correction formula. Specifically, the distorted pixel coordinates are normalized, the distortion-free normalized coordinates are derived using an iterative method, the distorted pixel normalized coordinates are set as the initial values ​​of the distortion-free coordinates, iterative calculations are performed, the radial distortion factor is calculated using a polynomial calculation method, the "simulated distortion coordinates" are calculated using the distortion model substitution method, and the distortion-free coordinates are corrected using an iterative approximation method. When the difference between the distortion-free coordinates of two iterations is less than a threshold, the iteration stops. The threshold is determined by professionals based on the actual business scenario, and the distortion-free normalized coordinates are mapped to the ideal pixel coordinates. For non-integer ideal pixel coordinates, a bilinear interpolation algorithm is used to supplement the pixel grayscale values; The clipping unit pre-divides a rectangular area covering the entire process state of the buffer based on the buffer features and installation parameters. The buffer features are the physical dimensions of the buffer, and the installation parameters are the camera installation parameters of the image acquisition module. Taking the initial state image of the buffer acquired by the camera as a reference, the outer contour boundary of the buffer is marked, and the boundary is extended outward by 5 pixels to determine the coordinate range of the initial ROI. Collect historical buffer images, collect the edge corner points and texture features of the historical buffer images as key feature points, establish a buffer feature point library, use the SIFT algorithm to extract key feature points in the buffer region of the current preprocessed image, calculate the coordinate deviation between the current buffer feature point and the buffer feature library through the feature point matching method, obtain the offset, and correct the initial ROI coordinates based on the offset. The cropping unit crops the current image based on the corrected ROI coordinates, removes the background outside the ROI, retains the main body area of ​​the buffer, and then uses a bilinear interpolation algorithm to scale the image to a fixed size.

[0008] The data annotation and augmentation module includes an annotation unit, an augmentation unit, and a sample balancing unit; The labeling unit tags the buffer for each compression state, specifically: Select the top center point P1, the top left edge point P2, the top right edge point P3, and the bottom center point of the elevator buffer as the key points of the buffer; A coordinate system is constructed with the center point of the bottom of the elevator buffer as the origin, and the coordinates of the elevator buffer before compression (X0, Y0, Z0) and the coordinates of the key points under the maximum compression state (X1, Y1, Z1) are recorded. The longitudinal offset of the elevator buffer is calculated using ΔY = Y1 - Y0; In the formula, △Y is the longitudinal offset, Y0 is the ordinate of the elevator buffer when it is not compressed, and Y1 is the ordinate of the elevator buffer when it is in the maximum compressed state. The lateral offset of the elevator buffer is calculated using ΔX = X1 - X0. In the formula, △X is the longitudinal offset, X0 is the horizontal coordinate of the elevator buffer when it is not compressed, and X1 is the horizontal coordinate of the elevator buffer when it is in the maximum compressed state. The height compression of the elevator buffer is calculated using △Z=Z1-Z0; In the formula, △Z is the height compression amount, Z0 is the height of the elevator buffer before compression, and Z1 is the height of the elevator buffer under the maximum compression state. Historically processed buffer images are collected as raw data for annotation. The raw buffer images are manually reviewed and seven abnormal labels are set for the images: normal, overpressure deformation, fracture, skew, stagnation / rebound, local wear, and cracking. The longitudinal offset, lateral offset, and height compression of the seven abnormal label images are collected. The mean μ and standard deviation σ of the longitudinal offset, lateral offset, and height compression are calculated. When the longitudinal offset, lateral offset, and height compression are not in the interval [μ-3σ,μ+3σ], they are judged as outliers and removed. The longitudinal offset, lateral offset, and height compression are sorted in ascending order and the maximum and minimum values ​​are calculated. The maximum and minimum values ​​of the longitudinal offset, lateral offset, and height compression of the images under each label are set as the label judgment threshold interval. The elevator buffer images are labeled according to the label judgment threshold interval. The enhancement unit performs three types of enhancement on the labeled buffer image: brightness interference, affine transformation, and hybrid enhancement. The enhanced image is used as the base single sample. The base single samples are collected to construct the base single sample image set. Four images in different states are randomly selected from the base single sample image set. The four selected images are scaled to one-quarter of the size of the original image. Then, each image is randomly flipped horizontally. After flipping, the edge box is updated synchronously. Create a blank canvas with the same size as the original image, divide the blank canvas into 4 sub-regions of the same size, randomly assign 4 preprocessed images to the 4 sub-regions, and use Gaussian blur to process the stitching boundary of the 4 sub-regions; The coordinates of the bounding boxes of each stitched image are offset and adjusted according to the position of the sub-region where the stitched image is located, while retaining the original state label of each image to form an enhanced sample; The sample equalization unit incorporates the SMOTE algorithm, which generates abnormal samples and expands them to the same quantity as normal samples.

[0009] The feature extraction module includes an object detection unit and a feature extraction calculation unit; The object detection unit uses the YOLOv8n lightweight object detection model to detect the target bounding box coordinates of the elevator buffer and mark the deformation area of the elevator buffer; The feature extraction calculation unit obtains the bounding box coordinates and anomaly probability of the buffer deformation area in the current image from the object detection unit. When the system starts, it records the initial bounding box coordinates of the buffer in the state where the elevator has not touched the bottom as the comparison benchmark for buffer features, obtains the distortion coefficient and the frame rate of the image sequence, calculates the time interval for each frame, converts the pixel coordinates of the real-time bounding box and the initial bounding box into actual physical coordinates through the distortion coefficient, identifies abnormal frames by the adjacent frame difference judgment method, replaces them with the average value of the bounding box coordinates of the previous 3 frames, and calculates the aspect ratio, area change rate, deformation symmetry, compression rate, and recovery coefficient according to the actual physical coordinates.

[0010] The anomaly detection module includes a model training and inference unit and a real-time determination unit; The model training and inference unit uses MobileNetV2 as a specific base network, adds the SE attention mechanism, inputs the samples generated by the sample balancing unit for training, trains the elevator buffer anomaly determination model, and uses the classification result and anomaly probability output by the elevator buffer anomaly determination model; The real-time determination unit packages the trained elevator buffer anomaly determination model and deploys it to the edge device in the elevator pit. Professionals set the anomaly probability determination thresholds x1 and x2 according to the actual scenario. If the anomaly probability ≥ x2, it is directly determined as abnormal. If the anomaly probability is in the interval [x1, x2], it is marked as疑似正常 (suspiciously normal) and triggers subsequent secondary determination by the knowledge base. If the anomaly probability < x1, it is determined as normal.

[0011] The knowledge base module includes a knowledge base construction unit and a priority scheduling unit; The knowledge base construction unit integrates historical cases, and operation and maintenance experts formulate buffer anomaly rules based on historical cases, construct a knowledge base according to the buffer anomaly rules, and then build an index for the knowledge base data; When the real-time determination unit determines that the anomaly probability of the elevator buffer is less than x1, the real-time determination unit sends a request to the priority scheduling unit. The priority scheduling unit calls the elevator buffer features obtained by the feature extraction unit, compares them with the knowledge base according to the elevator buffer features, and determines the elevator buffer.

[0012] The optimization module includes an elevator buffer anomaly determination model optimization unit and a knowledge base update unit; The elevator buffer anomaly determination model optimization unit sets the model optimization period to 30 days. At the same time, each time n inference tasks are completed, a parameter update check is triggered. The elevator buffer anomaly determination model optimization unit extracts the misjudgment cases of the model within the period from the system log and associates the corresponding buffer images, buffer features, and anomaly labels to form an optimization sample set. The optimization sample set is used as a new sample set and re-input into the model training and inference unit for model optimization; The knowledge base update unit sets the knowledge base data collection period to 30 days. It collects newly generated buffer anomaly cases, new structural parameters of buffers of the same model, and newly added buffer anomaly rules every 30 days. The newly added buffer anomaly rules are reviewed and confirmed by operation and maintenance experts. Then, the index reconstruction trigger condition is set as when the number of newly added cases per month is greater than or equal to z1 or the number of expert rules is greater than or equal to z2. z1 and z2 are trigger thresholds, which are set by professionals according to the actual business scenario. The knowledge base index reconstruction process is automatically started. Using the FAISS tool, the newly added data and the original knowledge base data are integrated into a unified data set to build a new vector index; The newly added buffer anomaly rules are converted from natural language to the mapping relationship of "feature threshold - anomaly type" using a semantic recognition model and associated with the corresponding feature vector interval. Finally, the reconstructed knowledge base index and the updated data are synchronized to the knowledge base for knowledge base optimization.

[0013] The communication module includes a monitoring data upload unit and an anomaly grading response unit; The monitoring data upload unit is responsible for uploading data. During the upload process, data with an anomaly probability > x2 is defined as high-risk data and is uploaded with priority. Data with an anomaly probability in [x1, x2] is second-priority data, and data with an anomaly probability < x1 is normal data and is stored locally; The anomaly grading response unit obtains the anomaly type and anomaly probability from the anomaly detection module. It determines fragmentation, overpressure deformation, and cracking as high-risk anomalies with an anomaly probability greater than or equal to x2, and defines skew, staying unrebounded, and local wear as non-high-risk anomalies with an anomaly probability in [x1, x2]; When the anomaly grading response unit obtains the output result of the elevator buffer anomaly determination model, it responds. When the output result is a high-risk anomaly, the elevator is forced to stop running and an alarm message is sent to the operation and maintenance personnel. When the output result is a non-high-risk anomaly, the elevator operation and maintenance are not stopped, and the operation and maintenance personnel are reminded to perform operation and maintenance on the elevator within 24 hours.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention acquires images using dual low-light cameras and a "dual-condition linkage" triggering mechanism. After distortion correction, contrast enhancement, and cropping, the images are combined with a target detection model and a "basic network + SE attention mechanism" model to detect anomalies. The knowledge base is then linked for secondary judgment to improve the accuracy of buffer compression state monitoring.

[0015] 2. This invention uses a communication module to respond in a graded manner according to the level of anomaly, and an optimization module to regularly optimize the model and update the knowledge base, ensuring efficient and reasonable operation and maintenance.

[0016] 3. This invention solves the sample imbalance problem by expanding abnormal samples using the SMOTE algorithm, and optimizes data utilization and sample quality by hierarchically storing and transmitting data in the communication module. Attached Figure Description

[0017] Fig. 1 This is a flowchart illustrating a machine vision-based elevator buffer compression state monitoring system according to the present invention. Fig. 2 This is a schematic diagram of the structure of an elevator buffer compression state monitoring system based on machine vision according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: As Figs. 1-2 As shown, the present invention provides a technical solution, an elevator buffer compression state monitoring system based on machine vision, the system including an image acquisition module, an image preprocessing module, a data annotation and enhancement module, a feature extraction module, an anomaly detection module, a knowledge base module, an optimization module and a communication module; The image acquisition module uses dual low-light cameras and a "dual-condition linkage" triggering mechanism to continuously acquire images of the entire buffer process and add timestamps to construct an image sequence when the elevator meets the triggering mechanism. After the image preprocessing module removes noise, it uses the Retinex algorithm to improve image contrast, calculates the distortion coefficient, corrects lens distortion based on the distortion coefficient, and crops the image. The data labeling and enhancement module selects key points of the buffer to establish a coordinate system, calculates relevant parameters, sets label judgment criteria, labels historical images according to the label judgment criteria, enhances the data, and expands abnormal samples through the SMOTE algorithm. The feature extraction module uses the target detection model to determine the buffer edge box and the deformation area, corrects the image coordinates to physical coordinates according to the distortion coefficient, and then extracts key features after correction. The anomaly detection module uses a specific basic network combined with the SE attention mechanism to train the model with samples generated by balanced sample units. The model is used to judge elevator anomalies, and when it is suspected to be normal, it switches to a secondary judgment. The knowledge base module integrates buffer features and historical cases, formulates anomaly rules to build a knowledge base and sets up data indexes, and compares features with knowledge base information when a situation is suspected to be normal to complete a secondary judgment. The optimization module is responsible for optimizing the model and updating the knowledge base; The communication module is responsible for collecting and encapsulating monitoring data, uploading abnormal data, storing normal data locally, and taking corresponding measures for different anomalies.

[0020] The image acquisition module includes an acquisition unit and an acquisition trigger unit; The acquisition unit uses dual low-light cameras. The low-light cameras are installed facing the elevator buffer. The camera bodies are adjusted with a level to ensure that both cameras are horizontal. The angle between the lens optical axis and the central axis of the buffer is ≤5° for camera deployment. The acquisition triggering unit adopts a "dual-condition linkage" mechanism. The "dual-condition linkage" mechanism pre-starts the camera when the elevator acceleration exceeds 0.5g and triggers the acquisition process when the elevator descends to the bottom. It continuously acquires images of the entire process of buffer compression and timestamps them to construct a buffer image sequence.

[0021] The image preprocessing module includes an image cleaning unit, a contrast enhancement unit, a distortion correction unit, and a cropping unit; The image cleaning unit converts the image into a grayscale image using a weighted average method. A 3×3 sliding window is set and slides the window pixel by pixel on the image. Grayscale values ​​are extracted for the nine pixels in each window. The nine grayscale values ​​in the window are sorted in ascending order, and the grayscale value of the center pixel of the window is replaced by the middle value of the sorted values. For the image edge area, virtual pixels outside the edge are supplemented by a mirror filling method. The contrast enhancement unit incorporates an ambient light sensor to decompose the pixel coordinates I(x,y) of the image into the reflection component R(x,Y) of the object structure and the illumination component L(x,y) of the ambient light distribution, outputting an enhanced image, specifically: The pixel coordinates I(x,y) of the image are decomposed into the reflection component R(x,Y) of the object structure and the illumination component L(x,y) of the ambient light distribution, specifically: ; In the formula, These are the pixel coordinates of the image. The reflection component of the object's structure. The illumination component represents the ambient light distribution. Take the logarithm of the image, specifically: ; In the formula, This is the logarithm of the image pixel coordinates. Let logarithm be the reflection component of the object's structure. The logarithm of the illumination component in the ambient light distribution; Using multi-scale Gaussian filtering Convolution, calculating the logarithm of the illumination component in the ambient light distribution. Specifically: ; In the formula, Let be the pixel value of the illumination component at the k-th scale, where k is the multi-scale index. represents the Gaussian filter kernel at the k-th scale; * represents the convolution operator. Normalize the pixel values ​​of the illumination components to [0,1]; Applying the gamma function, specifically: ; In the formula, L(x,y) represents the pixel value of the illumination component after gamma correction, L(x,y) represents the original estimated pixel value of the illumination component, and γ represents the gamma correction coefficient. The revised Restore to the original image's grayscale range; The corrected illumination component is stripped from the original image, and the pure reflection component R(x,y) is extracted: ; In the formula, This is the logarithmic form of the reflection component. The average weight of the multi-scale results, For the summation operator, These are the pixel values ​​of the original input image. Let be the logarithmic form of the illumination component at the k-th scale.

[0022] The distortion correction unit automatically identifies the intersection points of the checkerboard pattern in each acquired calibration board image using a corner detection algorithm, records the correspondence between the image coordinates of the corner points and the world coordinates of the calibration board, and uses the mathematical model of Zhang Zhengyou's calibration method to calculate the camera's intrinsic and extrinsic parameter matrices through least squares fitting, solves for the lens distortion coefficients, and calculates the ideal pixel coordinates without distortion for each pixel in the image based on the distortion coefficients and the distortion correction formula. Specifically: The distortion coefficients include radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2; The distorted pixel coordinates are normalized as follows: ; In the formula, Here are the normalized image coordinates in the x-direction after distortion. Let x be the x-coordinate of the distorted pixel. Let x be the coordinate of the principal point of the camera. The equivalent focal length in the x-direction; ; In the formula, This is the normalized image in the y-direction after distortion. Let be the y-coordinate of the distorted pixel. Let be the y-coordinate of the camera's principal point. The equivalent focal length in the y-direction; The iterative method is used to reverse-engineer the distortion-free normalized coordinates. The normalized coordinates of the distorted pixels are set as the initial values ​​of the distortion-free coordinates, and iterative calculations are performed, specifically as follows: ; In the formula, For the i-th iteration, the squared distance from the distortion-free normalized coordinates to the origin is... Let x be the distortion-free normalized coordinate in the x-direction for the i-th iteration. The y-axis is a distortion-free normalized coordinate. The radial distortion factor is calculated as follows: ; In the formula, Let be the radial distortion factor for the i-th iteration, and k1, k2, k3 be the radial distortion coefficients; The calculation of "simulated distortion coordinates" is as follows: ; In the formula, Let p1 and p2 be the normalized coordinates of the simulated distortion in the x-direction for the i-th iteration, and p1 and p2 be the tangential distortion coefficients. Let x be the distortion-free normalized coordinate in the x-direction for the i-th iteration. The y-axis is a distortion-free normalized coordinate. ; In the formula, Let p1 and p2 be the normalized coordinates of the simulated distortion in the y-direction for the i-th iteration, and p1 and p2 be the tangential distortion coefficients. Let x be the distortion-free normalized coordinate in the x-direction for the i-th iteration. The y-axis is a distortion-free normalized coordinate. Correcting distortion-free coordinates, specifically: ; In the formula, The distortion-free normalized coordinates in the x-direction for the (i+1)th iteration; ; In the formula, The distortion-free normalized coordinates in the y-direction for the (i+1)th iteration; The iteration stops when the difference between the distortion-free coordinates of two iterations is less than a threshold. The threshold is determined by professionals based on the actual business scenario, and is as follows: ; ; In the formula, Let x be the distortion-free normalized coordinate in the x-direction for the (i+1)th iteration. Let be the distortion-free normalized coordinates in the y-direction for the (i+1)th iteration. This is the threshold for iterative convergence; The distortion-free normalized coordinates are mapped to ideal pixel coordinates, specifically as follows: ; ; In the formula, The x-coordinate of an ideal pixel without distortion. The y-coordinate of an ideal pixel without distortion. For the final distortion-free normalized x-direction coordinates, This represents the final distortion-free, normalized y-axis coordinate.

[0023] For non-integer ideal pixel coordinates, a bilinear interpolation algorithm is used to supplement the pixel grayscale values; The clipping unit pre-divides a rectangular area covering the entire process state of the buffer based on the buffer features and installation parameters. The buffer features are the physical dimensions of the buffer, and the installation parameters are the camera installation parameters of the image acquisition module. Taking the initial state image of the buffer acquired by the camera as a reference, the outer contour boundary of the buffer is marked, and the boundary is extended outward by 5 pixels to determine the coordinate range of the initial ROI. Collect historical buffer images, collect the edge corner points and texture features of the historical buffer images as key feature points, establish a buffer feature point library, use the SIFT algorithm to extract key feature points in the buffer region of the current preprocessed image, calculate the coordinate deviation between the current buffer feature point and the buffer feature library through the feature point matching method, obtain the offset, and correct the initial ROI coordinates based on the offset. The cropping unit crops the current image based on the corrected ROI coordinates, removes the background outside the ROI, retains the main body area of ​​the buffer, and then uses a bilinear interpolation algorithm to scale the image to a fixed size.

[0024] The data annotation and augmentation module includes an annotation unit, an augmentation unit, and a sample balancing unit; The labeling unit tags the buffer for each compression state, specifically: Select the top center point P1, the top left edge point P2, the top right edge point P3, and the bottom center point of the elevator buffer as the key points of the buffer; A coordinate system is constructed with the center point of the bottom of the elevator buffer as the origin, and the coordinates of the elevator buffer before compression (X0, Y0, Z0) and the coordinates of the key points under the maximum compression state (X1, Y1, Z1) are recorded. The longitudinal offset of the elevator buffer is calculated using ΔY = Y1 - Y0; In the formula, △Y is the longitudinal offset, Y0 is the ordinate of the elevator buffer when it is not compressed, and Y1 is the ordinate of the elevator buffer when it is in the maximum compressed state. The lateral offset of the elevator buffer is calculated using ΔX = X1 - X0. In the formula, △X is the longitudinal offset, X0 is the horizontal coordinate of the elevator buffer when it is not compressed, and X1 is the horizontal coordinate of the elevator buffer when it is in the maximum compressed state. The height compression of the elevator buffer is calculated using △Z=Z1-Z0; In the formula, △Z is the height compression amount, Z0 is the height of the elevator buffer before compression, and Z1 is the height of the elevator buffer under the maximum compression state. Historically processed buffer images are collected as raw data for annotation. The raw buffer images are manually reviewed and seven abnormal labels are set for the images: normal, overpressure deformation, fracture, skew, stagnation / rebound, local wear, and cracking. The longitudinal offset, lateral offset, and height compression of the seven abnormal label images are collected. The mean μ and standard deviation σ of the longitudinal offset, lateral offset, and height compression are calculated. When the longitudinal offset, lateral offset, and height compression are not in the interval [μ-3σ,μ+3σ], they are judged as outliers and removed. The longitudinal offset, lateral offset, and height compression are sorted in ascending order and the maximum and minimum values ​​are calculated. The maximum and minimum values ​​of the longitudinal offset, lateral offset, and height compression of the images under each label are set as the label judgment threshold interval. The elevator buffer images are labeled according to the label judgment threshold interval. The enhancement unit performs three types of enhancement on the labeled buffer image: brightness interference, affine transformation, and hybrid enhancement. The enhanced image is used as the base single sample. The base single samples are collected to construct the base single sample image set. Four images in different states are randomly selected from the base single sample image set. The four selected images are scaled to one-quarter of the size of the original image. Then, each image is randomly flipped horizontally. After flipping, the edge box is updated synchronously. Create a blank canvas with the same size as the original image, divide the blank canvas into 4 sub-regions of the same size, randomly assign 4 preprocessed images to the 4 sub-regions, and use Gaussian blur to process the stitching boundary of the 4 sub-regions; The coordinates of the bounding boxes of each stitched image are offset and adjusted according to the position of the sub-region where the stitched image is located, while retaining the original state label of each image to form an enhanced sample; The sample equalization unit incorporates the SMOTE algorithm, which generates abnormal samples and expands them to the same quantity as normal samples.

[0025] The feature extraction module includes a target detection unit and a feature extraction calculation unit; The target detection unit uses the YOLOv8n lightweight target detection model to detect the target bounding box coordinates of the elevator buffer and marks the deformation area of ​​the elevator buffer. The feature extraction and calculation unit obtains the bounding box coordinates and anomaly probability of the current image buffer deformation area from the target detection unit. When the system starts, it records the initial bounding box coordinates of the buffer in the state where the elevator has not hit the bottom, as a comparison benchmark for the buffer features, obtains the distortion coefficient and the frame rate of the image sequence, calculates the time interval of each frame, converts the pixel coordinates of the real-time bounding box and the initial bounding box into actual physical coordinates through the distortion coefficient, identifies abnormal frames by the adjacent frame difference judgment method, replaces them with the average value of the bounding box coordinates of the previous 3 frames, and calculates the aspect ratio, area change rate, deformation symmetry, compression rate and restoration coefficient based on the actual physical coordinates.

[0026] The anomaly detection module includes a model training and inference unit and a real-time judgment unit; The model training and inference unit uses MobileNetV2 as a specific base network, adds the SE attention mechanism, inputs the samples generated by the sample equalization unit into the training, trains the elevator buffer anomaly detection model, and uses the classification results and anomaly probability output by the elevator buffer anomaly detection model. The real-time judgment unit encapsulates the trained elevator buffer anomaly judgment model and deploys it to the edge device of the elevator pit. Professionals set anomaly probability judgment thresholds of 0.5 and 0.8 according to the actual scenario. If the anomaly probability is ≥0.8, it is directly judged as an anomaly. If the anomaly probability is in the range [0.5, 0.8], it is marked as suspected normal and triggers a secondary judgment by the knowledge base. If the anomaly probability is <0.5, it is judged as normal.

[0027] The knowledge base module includes a knowledge base construction unit and a priority scheduling unit; The knowledge base construction unit integrates historical cases. The operation and maintenance experts formulate buffer abnormality rules based on the historical cases, construct a knowledge base according to the buffer abnormality rules, and then build an index for the knowledge base data; When the real-time determination unit determines that the abnormality probability of the elevator buffer is less than 0.5, the real-time determination unit sends a request to the priority scheduling unit. The priority scheduling unit calls the elevator buffer characteristics obtained by the feature extraction unit, compares them with the knowledge base according to the elevator buffer characteristics, and determines the elevator buffer.

[0028] The optimization module includes an elevator buffer abnormality determination model optimization unit and a knowledge base update unit; The elevator buffer abnormality determination model optimization unit sets the model optimization period to 30 days, and at the same time triggers a parameter update check every time n inference tasks are completed. The elevator buffer abnormality determination model optimization unit extracts misjudged cases of the model within the system log extraction period and associates the corresponding buffer images, buffer characteristics, and abnormality labels to form an optimization sample set. The optimization sample set is used as a new sample set and re-input into the model training and inference unit for model optimization; The knowledge base update unit sets the knowledge base data collection period to 30 days, and collects newly generated buffer abnormality cases, new structure parameters of the same type of buffer, and newly added buffer abnormality rules every 30 days. The newly added buffer abnormality rules are reviewed and confirmed by the operation and maintenance experts. Then, the index reconstruction trigger condition is set to when the number of newly added cases per month is greater than or equal to z1 or the expert rules are greater than or equal to z2. z1 and z2 are trigger thresholds, which are set by professionals according to the actual business scenario. Automatically start the knowledge base index reconstruction process, use the FAISS tool, integrate the newly added data with the original knowledge base data into a unified data set, and build a new vector index; Convert the newly added buffer abnormality rules from natural language to the mapping relationship of "feature threshold - abnormality type" using the semantic recognition model, and associate them with the corresponding feature vector interval. Finally, synchronize the reconstructed knowledge base index and updated data to the knowledge base for knowledge base optimization.

[0029] The communication module includes a monitoring data upload unit and an abnormality classification response unit; The monitoring data upload unit is responsible for uploading data. During the upload process, data with an abnormality probability > x2 is defined as high-risk data and is uploaded preferentially. Data with an abnormality probability in [x1, x2] is second-priority data, and data with an abnormality probability < x1 is normal data and is stored locally; The anomaly classification response unit obtains the anomaly type and anomaly probability from the anomaly detection module. It identifies fragmentation, overpressure deformation and cracking as high-risk anomalies with an anomaly probability greater than or equal to x2. It defines skewness, retention without rebound and local wear as non-high-risk anomalies with an anomaly probability in the range of [x1, x2]. When the anomaly classification response unit obtains the output result of the elevator buffer anomaly judgment model, it responds. When the output result is a high-risk anomaly, it forces the elevator to stop running and sends an alarm message to the operation and maintenance personnel. When the output result is a non-high-risk anomaly, it does not stop the elevator operation and maintenance, but reminds the operation and maintenance personnel to perform operation and maintenance on the elevator within 24 hours.

[0030] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A machine vision based elevator buffer compression state monitoring system, characterized by: The system comprises an image acquisition module, an image preprocessing module, a data labeling and enhancement module, a feature extraction module, an anomaly detection module, a knowledge base module, an optimization module and a communication module; The image acquisition module acquires images of the whole process of the buffer through a double low-light camera and a double-condition linkage triggering mechanism, and adds timestamps to the images to construct an image sequence when the elevator meets the triggering mechanism; The image preprocessing module uses a Retinex algorithm to improve the contrast of the image after image denoising, calculates a distortion coefficient, corrects lens distortion according to the distortion coefficient, and crops the image; The data labeling and enhancement module sets a coordinate system to calculate relevant parameters, sets a label judgment basis, labels historical images according to the label judgment basis, and enhances the data by using the SMOTE algorithm to expand abnormal samples; The feature extraction module uses a target detection model to determine the buffer edge frame and the deformation area, corrects the image coordinates to physical coordinates according to the distortion coefficient, and extracts key features after correction; The anomaly detection module uses a balanced sample unit to generate a model for training, judges elevator anomalies according to the model, and switches to secondary judgment when it is suspected to be normal; The knowledge base module integrates buffer features and historical cases, formulates abnormal rules, establishes a knowledge base, and sets data indexes, compares features with knowledge base information when it is suspected to be normal, and completes secondary judgment; The optimization module is responsible for optimizing the model and updating the knowledge base; The communication module is responsible for collecting and packaging monitoring data, uploading abnormal data, locally storing normal data, and taking corresponding measures for different anomalies.

2. A machine vision-based elevator bumper compression state monitoring system according to claim 1, characterized in that: The image acquisition module comprises an acquisition unit and an acquisition triggering unit; The acquisition unit uses a double low-light camera, which is installed directly opposite the elevator buffer, and uses a level to adjust the camera body so that the two cameras are horizontal, and the included angle between the lens optical axis and the buffer center axis is ≤5° for camera deployment; The acquisition triggering unit adopts a double-condition linkage mechanism, which pre-starts the camera when the elevator acceleration exceeds 0.5g, and triggers the acquisition process when the elevator is descending to the bottom end; The acquisition unit continuously acquires images of the whole process of buffer compression and adds timestamps to construct a buffer image sequence.

3. A machine vision-based elevator bumper compression state monitoring system according to claim 2, characterized in that: The image preprocessing module comprises an image cleaning unit, a contrast enhancement unit, a distortion correction unit and a cropping unit; The image cleaning unit converts the image into a grayscale image by using a weighted average method, sets a 3x3 sliding window, slides the sliding window on the image pixel by pixel, extracts the grayscale values of 9 pixels in each window and sorts them in ascending order, replaces the grayscale value of the center pixel in the window with the sorted middle value, and uses mirror padding to supplement virtual pixels outside the edge area for the image edge area; The contrast enhancement unit is provided with an ambient light sensor, which decomposes the pixel coordinates I(x,y) of the image into a reflection component R(X,Y) of the object structure and a light component L(x,y) of the ambient light distribution, and outputs an enhanced image.

4. A machine vision-based elevator bumper compression state monitoring system according to claim 3, characterized in that: The distortion correction unit identifies the corner points through a corner point detection algorithm for each collected calibration plate image, records the corresponding relationship between the image coordinates of the corner points and the world coordinates of the calibration plate, uses the mathematical model of Zhang Zhengyou's calibration method, and calculates the camera intrinsic matrix and extrinsic matrix through least squares fitting, so as to solve the lens distortion coefficient. For each pixel point of the image, the ideal pixel coordinates without distortion are calculated according to the distortion coefficient and the distortion correction formula; For non-integer ideal pixel coordinates, a bilinear interpolation algorithm is used to supplement the pixel gray value; The cropping unit pre-divides the rectangular area covering the whole state of the buffer based on the physical size of the buffer and the installation parameters of the camera of the image acquisition module. The installation parameters are used as the reference for the initial state image of the buffer collected by the camera, and the outer contour boundary of the buffer is marked and expanded outward by 5 pixels to determine the coordinate range of the initial ROI. The historical buffer images are collected, the corner points and texture features of the historical buffer images are collected as key feature points, a buffer feature point library is established, the key feature points in the buffer region of the current preprocessed image are extracted using the SIFT algorithm, the coordinate deviation between the current buffer feature points and the buffer feature library is calculated through the feature point matching method, the offset is obtained, and the initial ROI coordinates are corrected according to the offset. The cropping unit crops the current image based on the corrected ROI coordinates, removes the background outside the ROI, and retains the main body region of the buffer. Then, the bilinear interpolation algorithm is used to scale the image to a fixed size.

5. The machine vision-based elevator bumper compression state monitoring system of claim 1, wherein: The data labeling and enhancement module includes a labeling unit, an enhancement unit, and a sample balancing unit. The labeling unit labels each compression state of the buffer, specifically: Select the top center point P1 of the elevator buffer, the top left edge point P2, the top right edge point P3, and the bottom center point of the elevator buffer as the key points of the buffer. A coordinate system is constructed with the bottom center point of the elevator buffer as the coordinate origin, and the coordinates (X0, Y0, Z0) of the elevator buffer in the uncompressed state and the coordinates (X1, Y1, Z1) of the key points in the maximum compression state are recorded. The three-dimensional variables of the elevator buffer are calculated, including the longitudinal displacement, the lateral displacement, and the height compression. The longitudinal displacement of the elevator buffer is calculated by ΔY=Y1-Y0. In the formula, ΔY is the longitudinal displacement, Y0 is the longitudinal coordinate of the elevator buffer in the uncompressed state, and Y1 is the longitudinal coordinate of the elevator buffer in the maximum compression state. The lateral displacement of the elevator buffer is calculated by ΔX=X1-X0. In the formula, ΔX is the longitudinal displacement, X0 is the lateral coordinate of the elevator buffer in the uncompressed state, and X1 is the lateral coordinate of the elevator buffer in the maximum compression state. The height compression of the elevator buffer is calculated by ΔZ=Z1-Z0. In the formula, ΔZ is the height compression, Z0 is the height of the elevator buffer in the uncompressed state, and Z1 is the height of the elevator buffer in the maximum compression state. The processed buffer image is collected as labeled original data, the original buffer image is manually audited, and the image is labeled as normal, overpressure deformation, fragmentation, skew, and seven types of abnormal labels, three-dimensional variables of the seven types of abnormal label images are collected, the mean μ and the standard deviation σ of the three-dimensional variables are calculated, when the three-dimensional variables are not in the interval [μ-3σ, μ+3σ], it is determined that the value is abnormal, the abnormal value is removed, the three-dimensional variables are sorted in ascending order to calculate the maximum value and the minimum value, and the maximum value and the minimum value of the three-dimensional variables of the image under each type of label are set as the label determination threshold interval, and the elevator buffer image is labeled according to the label determination threshold interval; The enhancement unit performs three types of enhancement on the labeled buffer image, and the three types of enhancement are brightness interference, affine transformation and mixed enhancement; The enhanced image is taken as a basic single sample, the basic single sample is collected, a basic single sample image set is constructed, four images in different states are randomly selected from the basic single sample image set, the four selected images are scaled to one fourth of the original image size, then each image is randomly executed horizontal flip, and the edge frame is updated synchronously after the flip; A blank canvas with the same size as the original image is created, the blank canvas is divided into four sub-regions with the same size, the four preprocessed images are randomly allocated to the four sub-regions, and the four sub-regions are processed using Gaussian blur on the splicing boundary; The edge frame coordinates of each spliced image are offset adjusted according to the position of the sub-region where the spliced image is located, the original state label of each image is retained, and an enhanced sample is formed. The sample balancing unit is internally provided with an SMOTE algorithm, and the SMOTE algorithm is used to generate abnormal samples, and the abnormal samples are expanded to the same amount as the normal samples.

6. The machine vision-based elevator bumper compression state monitoring system of claim 1, wherein: The feature extraction module comprises a target detection unit and a feature extraction calculation unit. The target detection unit uses a YOLOv8n lightweight target detection model to detect the target edge frame coordinates of the elevator buffer, and labels the deformation area of the elevator buffer. The feature extraction calculation unit obtains the edge frame coordinates and abnormal probability of the deformation area of the current image buffer from the target detection unit, records the initial edge frame coordinates of the buffer under the non-touching bottom state of the elevator as a comparison reference when the system starts, obtains the distortion coefficient and the frame rate of the image sequence, calculates the time interval of each frame, converts the pixel coordinates of the real-time edge frame and the initial edge frame into actual physical coordinates through the distortion coefficient, identifies the abnormal frame through the adjacent frame difference value judgment method, replaces the average value of the edge frame coordinates of the previous three frames, and calculates the height-width ratio, the area change rate, the deformation symmetry, the compression rate and the recovery coefficient according to the actual physical coordinates.

7. The machine vision-based elevator bumper compression state monitoring system of claim 1, wherein: The abnormal detection module comprises a model training and reasoning unit and a real-time determination unit. The model training and reasoning unit uses MobileNetV2 as a specific basic network, adds an SE attention mechanism, inputs the samples generated by the sample balancing unit, trains an elevator buffer abnormality judgment model, and uses the classification results and abnormal probability output by the elevator buffer abnormality judgment model. The real-time determination unit deploys the trained elevator buffer abnormality determination model to the edge device of the elevator pit after encapsulation, and a professional sets the abnormal probability determination thresholds x1 and x2 according to an actual scene, determines that the abnormal probability is greater than or equal to x1 as abnormal, determines that the abnormal probability is in the interval [x1, x2] as suspected normal, triggers secondary determination of the knowledge base, and determines that the abnormal probability is less than x1 as normal.

8. The machine vision-based elevator bumper compression state monitoring system of claim 1, wherein: The knowledge base module comprises a knowledge base construction unit and a priority scheduling unit; The knowledge base construction unit integrates historical cases, and an operation and maintenance expert formulates buffer abnormality rules according to the historical cases, constructs the knowledge base according to the buffer abnormality rules, and constructs an index for the knowledge base data; When the real-time determination unit determines that the abnormal probability of the elevator buffer is less than x1, the real-time determination unit sends a request to the priority scheduling unit, the priority scheduling unit calls the elevator buffer features obtained by the feature extraction unit, compares the elevator buffer features with the knowledge base, and determines the elevator buffer.

9. The machine vision-based elevator bumper compression state monitoring system of claim 1, wherein: The optimization module comprises an elevator buffer abnormality determination model optimization unit and a knowledge base updating unit; The elevator buffer abnormality determination model optimization unit sets a model optimization period of 30 days, triggers a parameter update check once every n reasoning tasks are completed, extracts misjudgment cases of the model in the period through system logs, and associates corresponding buffer images, buffer features and abnormal labels to form an optimization sample set, takes the optimization sample set as a new sample set, and re-inputs the new sample set to the model training and reasoning unit for model optimization; The knowledge base updating unit sets a knowledge base data acquisition period of 30 days, acquires new buffer abnormality cases, new structure parameters of the same type of buffer and new buffer abnormality rules every 30 days, the new buffer abnormality rules are audited and confirmed by an operation and maintenance expert, and then sets an index reconstruction trigger condition that when the number of new cases is greater than or equal to z1 or the number of expert rules is greater than or equal to z2 per month, z1 and z2 are trigger thresholds, which are set by a professional according to an actual business scene, automatically starts the knowledge base index reconstruction process, integrates the new data and the original knowledge base data into a unified data set using the FAISS tool, and constructs a new vector index; The new buffer abnormality rules are converted into a mapping relationship of "feature threshold-abnormal type" from natural language using a semantic recognition model, and are associated with corresponding feature vector intervals, and finally the reconstructed knowledge base index and the updated data are synchronized to the knowledge base for knowledge base optimization.

10. The machine vision-based elevator bumper compression state monitoring system of claim 1, wherein: The communication module comprises a monitoring data uploading unit and an abnormality grading response unit; The monitoring data uploading unit is responsible for uploading data, and in the uploading process, data with an abnormal probability greater than x2 is defined as high-risk data and is uploaded preferentially, data with an abnormal probability in [x1, x2] is second-priority data, and data with an abnormal probability less than x1 is normal data and is stored locally. The abnormal grading response unit obtains the abnormal type and abnormal probability from the abnormal detection module, determines fragmentation, overpressure deformation and cracking as high-risk abnormalities, and abnormal probability greater than or equal to x2, defines deflection, retention without rebound and local wear as non-high-risk abnormalities, and abnormal probability is in [x1, x2]; When the abnormal grading response unit obtains the output result of the elevator buffer abnormality judgment model, it responds, when the output result is a high-risk abnormality, it forces the elevator to stop running and sends an alarm information to the operation and maintenance personnel, when the output result is a non-high-risk abnormality, it does not stop the elevator operation and maintenance, and reminds the operation and maintenance personnel to operate and maintain the elevator within 24 hours.

Images