Elevator guide rail automatic detection and calibration system based on machine vision

Abstract

The invention discloses an elevator guide rail automatic detection and calibration system based on machine vision. The elevator guide rail automatic detection and calibration system comprises an image acquisition module based on system architecture, an image preprocessing module, a feature extraction and matching module, a straightness and flatness calculation module and a calibration execution module. Compared with the prior art, the method has the advantages that the detection precision is high; the efficiency is high; the real-time performance is high; the automation degree is high.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator guide rail detection and calibration, and in particular to an automatic elevator guide rail detection and calibration system based on machine vision. Background Art

[0002] The inspection and calibration of traditional elevator guide rails mostly rely on manual operations, which is not only inefficient but also difficult to ensure the accuracy and consistency of the test results. Manual inspection is easily affected by environmental and human factors, resulting in large errors and making it difficult to meet the high-precision and high-efficiency requirements of the modern elevator manufacturing industry. At the same time, with the continuous development of elevator technology, the requirements for the straightness and flatness of guide rails are becoming increasingly stringent, and traditional inspection methods can no longer meet this demand. In the existing technology, although some inspection equipment uses machine vision technology, most of them have problems such as high complexity of image processing algorithms, large computational workload, and poor real-time performance. In addition, the calibration process still requires manual intervention, and true full automation cannot be achieved. Therefore, it is particularly important to develop an efficient, accurate, and automated elevator guide rail inspection and calibration system. Summary of the Invention

[0003] The technical problems to be solved by the present invention are: 1) low detection accuracy: traditional detection methods are easily affected by environmental and human factors, resulting in low detection accuracy; 2) low efficiency: manual detection and calibration processes are time-consuming and labor-intensive, and cannot meet the needs of large-scale production; 3) poor real-time performance: existing machine vision detection equipment often has problems such as large computational complexity and slow processing speed, making it difficult to achieve real-time detection; 4) low degree of automation: the calibration process still requires manual intervention and cannot be fully automated.

[0004] To solve the above technical problems, the present invention provides a technical solution: a machine vision-based elevator guide rail automatic detection and calibration system, comprising an image acquisition module, an image preprocessing module, a feature extraction and matching module, a straightness and flatness calculation module, and a calibration execution module based on the system architecture. The image acquisition module is used to capture image data of the elevator guide rail through a high-resolution camera; the image preprocessing module performs preprocessing operations such as denoising and contrast enhancement on the acquired image to improve the accuracy and efficiency of subsequent processing; the feature extraction and matching module uses an image processing algorithm to extract edge and corner feature information of the elevator guide rail, and performs feature matching to determine the accurate position and shape of the guide rail; the straightness and flatness calculation module uses an algorithm to calculate the straightness and flatness of the elevator guide rail based on the extracted feature information, and determines whether it meets the design requirements; the calibration execution module controls the robotic arm or other actuator to automatically calibrate the elevator guide rail according to the calculation results of the straightness and flatness, and adjusts its position and posture to achieve the predetermined accuracy requirements.

[0005] Compared with the existing technology, the advantages of the present invention are: 1) high detection accuracy: through high-resolution cameras and advanced image processing algorithms, accurate detection of the straightness and flatness of elevator guide rails is ensured, greatly improving the detection accuracy; 2) high efficiency: the automation of the detection and calibration process is realized, which greatly improves work efficiency and reduces labor costs; 3) strong real-time performance: optimized image processing algorithms and efficient calculation methods ensure the real-time output of detection results, meeting the real-time requirements of the modern elevator manufacturing industry; 4) high degree of automation: the calibration process does not require human intervention, and full automation from image acquisition to calibration execution is achieved, thereby improving the automation level of the production line.

[0006] Furthermore, the image acquisition module includes at least one high-resolution camera, which is mounted on an adjustable bracket to adapt to the detection requirements of elevator guide rails of different sizes and shapes.

[0007] Furthermore, the image preprocessing module includes the following functions: 1) Denoising: using median filtering and Gaussian filtering algorithms to denoise the image and remove random noise and interference in the image; 2) Contrast enhancement: adjusting the contrast of the image to make the edges and corner features of the elevator guide rails more prominent, facilitating subsequent feature extraction.

[0008] Furthermore, the median filtering denoising process includes the following steps: 1) determining the neighborhood window size: usually selecting odd-sized windows of 3x3 or 5x5 to ensure that the median can be found accurately; 2) sliding the window: sliding the window across the image, covering a small area in the image each time; 3) sorting and taking the median: sorting the pixel values ​​in each window, and taking the median as the new grayscale value of the center pixel of the window; 4) replacing the pixel value: replacing the original grayscale value of the center pixel of the window with the sorted median.

[0009] Furthermore, the Gaussian filtering denoising process includes the following steps: 1) determining the Gaussian kernel size and standard deviation: the size and standard deviation of the Gaussian kernel are important parameters that affect the filtering effect and are selected according to the severity of the noise and the detail requirements of the image; 2) constructing the Gaussian kernel: constructing a two-dimensional Gaussian kernel based on the selected size and standard deviation; 3) convolution operation: convolving the Gaussian kernel with each pixel point in the image and its neighborhood, and calculating the weighted average; 4) replacing pixel values: replacing the grayscale value of the original pixel point with the calculated weighted average.

[0010] Furthermore, the contrast enhancement function adopts two methods: histogram adjustment or automatic contrast adjustment, among which: the histogram adjustment method is: 1) Analyze histogram: First, analyze the histogram of the image to understand the brightness distribution of the image. The histogram shows the number of pixels at each brightness level in the image, through which it can be judged whether the contrast of the image is sufficient; 2) Stretching histogram: If the histogram distribution of the image is narrow, it means that the contrast is low. At this time, the contrast can be increased by stretching the histogram. The pixels at the darkest and brightest ends of the histogram are expanded to both ends, and the pixels in the middle part are compressed, so that the brightness range of the image is expanded and the contrast is enhanced; 3) Apply adjustment: Apply the adjusted histogram to the original image to obtain a contrast-enhanced image; the automatic contrast adjustment method is implemented through third-party image processing software installed in the system.

[0011] Furthermore, the feature extraction and matching module uses the SURF feature point extraction algorithm to extract edge and corner feature information from the preprocessed image. The RANSAC algorithm is then used to match the extracted features to determine the exact position and shape of the elevator guide rail. The matching results can be used to identify guide rail deviations or defects. The SURF algorithm uses the determinant of the Hessian matrix as the basis for feature point detection and utilizes integral graphs to accelerate the calculation. The steps include: 1) Feature point detection: Detecting feature points using the determinant of the Hessian matrix; 2) Feature point localization: Locating feature points by interpolating neighboring information; 3) Direction localization: Determining the direction by calculating the Haar wavelet transform of the pixels surrounding the feature point; 4) Feature descriptor: Generating a 64-dimensional descriptor to describe the image information surrounding the feature point. The RANSAC algorithm iteratively estimates mathematical model parameters from samples containing abnormal data to obtain the optimal data matching result. The algorithm includes five steps: randomly selecting sample points, calculating the homography matrix, evaluating inliers and outliers, iterative optimization, and determining the position and shape of the elevator guide rail. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of the architecture of the patent of this invention.

[0013] Figure 2 This is a schematic diagram of the workflow of the patent of this invention.

[0014] As shown in the figure: 1. Image acquisition module, 2. Image preprocessing module, 3. Feature extraction and matching module, 4. Straightness and flatness calculation module, 5. Calibration execution module. DETAILED DESCRIPTION

[0015] The present invention will be described in further detail below with reference to the accompanying drawings.

[0016] Example 1

[0017] When the present invention is specifically implemented, Figure 1 and Figure 2 As shown in the figure, the workflow of this system includes:

[0018] 1) Image Acquisition: The system is equipped with high-resolution cameras mounted on adjustable brackets to accommodate inspection of elevator guide rails of varying sizes and shapes. The cameras capture image data as the guide rail enters the inspection area. This image data contains key information such as the rail's shape and position.

[0019] 2) Image Preprocessing: De-noising is performed on the captured images using algorithms such as median filtering and Gaussian filtering to remove random noise and interference, improving image quality. By adjusting the image contrast, features such as the edges and corners of the elevator guide rails are more prominent, facilitating subsequent feature extraction and recognition.

[0020] 3) Feature Extraction and Matching: Image processing algorithms (such as the SURF feature point extraction algorithm) are used to extract features such as edges and corners of the elevator guide rail from the preprocessed image. This feature information is used in subsequent straightness and flatness calculations. Algorithms such as RANSAC are used to match the extracted features to determine the precise position and shape of the elevator guide rail. The matching results can be used to identify guide rail deviations or defects.

[0021] 4) Straightness and Flatness Calculation: Based on the extracted feature information, the system uses specific algorithms to calculate the straightness and flatness of the elevator guide rail. These algorithms may include the two-end line method and the least squares method to evaluate the guide rail's straightness error and flatness. The calculated results are compared with the design requirements to determine whether the elevator guide rail meets the design requirements.

[0022] 5) Calibration Execution: Based on the straightness and flatness calculations, the system controls a robotic arm or other actuator to automatically calibrate the elevator guide rails. The actuator adjusts the position and orientation of the guide rails according to preset calibration parameters and strategies to achieve the desired accuracy.

[0023] 6) Feedback and Optimization: After calibration is complete, the system will provide feedback to the operator or control system for subsequent processing or recording. Based on the feedback and actual needs, the system can perform iterative operations such as algorithm optimization and parameter adjustment to continuously improve the accuracy and efficiency of detection and calibration.

[0024] Example 2

[0025] Based on the first embodiment, Figure 1 and Figure 2 As shown, the specific steps of the RANSAC algorithm for matching feature points are as follows:

[0026] 1) Randomly select sample points: Randomly select 4 non-collinear sample points from the extracted feature points. These 4 sample points will be used to calculate the initial homography matrix, which describes the mapping relationship from one image plane to another image plane; 2) Calculate the homography matrix: Use the 4 selected sample points to calculate the homography matrix H. In a two-dimensional image, the homography matrix is ​​a 3x3 matrix that can map points in one image to corresponding points in another image; 3) Evaluate inliers and outliers: Substitute the remaining feature points into the calculated homography matrix H, and evaluate whether they are inliers (points that satisfy H) or outliers (points that do not satisfy H) by calculating the distance between these points and the points mapped by H (i.e., the reprojection error). Generally, points with a distance less than a set threshold are considered inliers.

[0027] 4) Iterative Optimization: Repeat the above steps multiple times (usually with a set maximum number of iterations). Each iteration randomly selects four new sample points and calculates the homography matrix H. In each iteration, the number of inliers is recorded. At the end of each iteration, the homography matrix H resulting from the iteration with the largest number of inliers is selected as the final result. This matrix is ​​most likely to correctly describe the transformation relationship between the elevator guide rails in the two images.

[0028] 5) Determine the position and shape of the elevator guide rail: Using the resulting homography matrix H, the elevator guide rail in one image can be mapped to another image, thereby determining its exact position and shape from different perspectives.

[0029] Example 3

[0030] Based on the second embodiment, Figure 1 and Figure 2 As shown in the figure, fitting the straight line features of the elevator guide rail using algorithms such as the least squares method based on the results of feature matching is an effective method for calculating its straightness parameters. The straightness parameter is usually used to describe the degree to which a straight line deviates from the ideal straight line. The specific steps are as follows:

[0031] 1) Data Preparation

[0032] Feature point screening: From the feature matching results, filter out feature points belonging to the straight sections of the elevator guide rail. These points should be distributed along the straight sections of the guide rail. Coordinate transformation: If the feature points are in the image coordinate system and need to be converted to the real world coordinate system, the corresponding coordinate transformation is required. This usually involves the camera's intrinsic and extrinsic matrix (i.e., the camera calibration parameters).

[0033] 2) Linear fitting

[0034] Least squares method to fit a straight line:

[0035] Assume the equation of the line is y = mx + b (or equivalently, Ax + By + C = 0 in two-dimensional space, where A² + B² = 1). For each feature point (xi, yi), calculate the square of its distance to the line and sum it to obtain the sum of squared errors S. Minimize the expression of S with respect to m and b (or A, B, C). By taking the partial derivative and setting it to 0, the optimal values ​​of m and b (or A, B, C) can be solved. In practical applications, due to the constraint of A² + B² = 1, the form Ax + By + C = 0 is often used, and the solution is obtained through methods such as SVD (Singular Value Decomposition).

[0036] Straightness parameter calculation:

[0037] The straightness parameter is typically defined as the maximum deviation between the actual line and the ideal line. After fitting the line, the perpendicular distance from each feature point to the fitted line is calculated, and the maximum of these distances is found to be the straightness parameter. The following is a simple example of line fitting using the NumPy library. Note that this does not cover the specific steps of image processing and feature matching.

[0038] import numpy as np

[0039] #Assume that there is already a set of feature point coordinates

[0040] points=np.array([[x1,y1],[x2,y2],...,[xn,yn]])

[0041] #Use np.polyfit to perform straight line fitting (here the fitting is in the form of y=mx+b)

[0042] #Note: polyfit returns coefficients from highest power to lowest power

[0043] m,b=np.polyfit(points[:,0],points[:,1],1)

[0044] #The equation of the fitted line is y=mx+b

[0045] #Calculate the distance from each point to the line

[0046] distances=np.abs(m*points[:,0]-points[:,1]+b) / np.sqrt(m**2+1)

[0047] #Straightness parameter is the maximum distance

[0048] straightness=np.max(distances)

[0049] print(f"Slope of the line: {m}, intercept: {b}")

[0050] print(f"Straightness parameter:{straightness}")

[0051] Example 4

[0052] Based on the third embodiment, Figure 1 and Figure 2 As shown, for the guide rail part that needs to be evaluated for flatness, a mathematical algorithm can be used to perform plane fitting and calculate the flatness parameters. Plane fitting usually involves finding the plane equation that best describes a set of three-dimensional points and calculating the maximum distance from these points to the plane as the flatness parameter. The following are the specific steps

[0053] 1) Plane fitting

[0054] Plane equation: In three-dimensional space, the equation of a plane can be expressed as Ax + By + Cz + D = 0, where A, B, and C are the plane's normal vectors, and D is the distance from the plane to the origin. Typically, A² + B² + C² = 1 is required to normalize the normal vector.

[0055] Least squares fitting: For a set of three-dimensional points (xi, yi, zi), the least squares method can be used to fit the plane. The goal is to find A, B, C, D so that the sum of squared errors is minimized, that is, i∑(Axi+Byi+Czi+D)2 is minimized. This can be done by solving a system of linear equations or using matrix operations. Specifically, a design matrix X can be constructed, in which each row is a point (xi, yi, zi, 1), and the linear system XTXp=XT0 can be solved, where p=(A, B, C, D)T and 0 is an all-zero vector. But note that since A, B, C need to be standardized, they are usually normalized after solving. It is more common to use SVD (singular value decomposition) to solve this problem directly because it can handle numerical problems more stably.

[0056] SVD solution: Construct a matrix M with each row being (xi,yi,zi), and then compute the SVD of M. Using SVD, we can find the right singular vector corresponding to the smallest singular value of M. This vector (after ignoring the last element and properly normalizing) is the plane normal (A,B,C). We can then use any point (xk,yk,zk) to solve for D.

[0057] 2) Calculate flatness parameters

[0058] Distance from a point to a plane:

[0059] For the plane Ax+By+Cz+D=0 and the point (xi,yi,zi), the distance from the point to the plane is:

[0060]

[0061] Since the normal vector has been normalized, the denominator is 1.

[0062] Flatness parameter: The flatness parameter is defined as the maximum distance from all points to the fitted plane, i.e., flatness = maxdi. Below is a simplified Python code example using the NumPy library to fit a plane and calculate the flatness parameter.

[0063] import numpy as np

[0064] #Assume that there is already a set of three-dimensional points

[0065] points=np.array([[x1,y1,z1],[x2,y2,z2],...,[xn,yn,zn]])

[0066] #Build the design matrix (note that D is not included here because D will be calculated later)

[0067] A=points[:,:3]

[0068] #Use SVD to solve the normal vector of the plane

[0069] U,s,Vt=np.linalg.svd(A-np.mean(A,axis=0),full_matrices=False)

[0070] normal=Vt[-1,:]#Take the right singular vector corresponding to the minimum singular value as the normal vector

[0071] normal / =np.linalg.norm(normal)#normalized normal vector

[0072] #Select a point to calculate D

[0073] point=points[0]

[0074] D = -np.dot(normal,point)

[0075] #Calculate the distance from all points to the plane

[0076] distances=np.abs(np.dot(points,normal)+D)

[0077] # Planarity parameter is the maximum distance

[0078] flatness = np.max(distances)

[0079] print(f"plane normal vector:{normal}")

[0080] print(f"Plane equation parameter D:{D}")

[0081] print(f"Flatness parameter:{flatness}")

[0082] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic inspection and calibration system for elevator guide rails based on machine vision, comprising an image acquisition module (1), an image preprocessing module (2), a feature extraction and matching module (3), a straightness and flatness calculation module (4), and a calibration execution module (5) based on a system architecture, characterized in that: The image acquisition module (1) is used to capture image data of an elevator guide rail through a high-resolution camera; the image preprocessing module (2) performs preprocessing operations of denoising and contrast enhancement on the acquired image to improve the accuracy and efficiency of subsequent processing; the feature extraction and matching module (3) uses an image processing algorithm to extract edge and corner feature information of the elevator guide rail, and performs feature matching to determine the accurate position and shape of the guide rail; the straightness and flatness calculation module (4) uses an algorithm to calculate the straightness and flatness of the elevator guide rail based on the extracted feature information, and determines whether it meets the design requirements; the calibration execution module (5) controls a mechanical arm or other actuator to automatically calibrate the elevator guide rail according to the calculation results of the straightness and flatness, and adjusts its position and posture to achieve a predetermined accuracy requirement.

2. The automatic detection and calibration system for elevator guide rails based on machine vision according to claim 1, characterized in that: The image acquisition module (1) comprises at least one high-resolution camera, which is mounted on an adjustable bracket to meet the detection requirements of elevator guide rails of different sizes and shapes.

3. According to claim 1, it is characterized in that The image preprocessing module (2) includes the following functions: 1) denoising: using median filtering and Gaussian filtering algorithms to denoise the image and remove random noise and interference in the image; 2) contrast enhancement: adjusting the contrast of the image to make the edge and corner features of the elevator guide rail more prominent, which is convenient for subsequent feature extraction.

4. The automatic detection and calibration system for elevator guide rails based on machine vision according to claim 3 is characterized in that The median filtering denoising processing steps include: 1) determining the neighborhood window size: usually selecting a window of odd size of 3x3 or 5x5 to ensure that the median can be found accurately; 2) sliding the window: sliding the window on the image, covering a small area in the image each time; 3) sorting and taking the median: sorting the pixel values ​​in each window, taking the median as the new grayscale value of the pixel point at the center of the window; 4) replacing the pixel value: replacing the original grayscale value of the pixel point at the center of the window with the sorted median.

5. The automatic detection and calibration system for elevator guide rails based on machine vision according to claim 3 is characterized in that The Gaussian filtering denoising processing steps include: 1) determining the Gaussian kernel size and standard deviation: the size and standard deviation of the Gaussian kernel are important parameters that affect the filtering effect, and are selected according to the severity of the noise and the detail requirements of the image; 2) constructing a Gaussian kernel: constructing a two-dimensional Gaussian kernel based on the selected size and standard deviation; 3) convolution operation: convolving the Gaussian kernel with each pixel point and its neighborhood in the image, and calculating the weighted average; 4) replacing pixel values: replacing the grayscale value of the original pixel point with the calculated weighted average.

6. The automatic detection and calibration system for elevator guide rails based on machine vision according to claim 3 is characterized in that The contrast enhancement function adopts two methods: histogram adjustment or automatic contrast adjustment, among which: the histogram adjustment method is: 1) analyzing the histogram: first, analyzing the histogram of the image to understand the brightness distribution of the image. The histogram shows the number of pixels at each brightness level in the image, through which it can be judged whether the contrast of the image is sufficient; 2) stretching the histogram: if the histogram distribution of the image is narrow, it means that the contrast is low. At this time, the contrast can be increased by stretching the histogram, expanding the pixels at the darkest and brightest ends of the histogram to both ends, and compressing the pixels in the middle part, so as to expand the brightness range of the image and enhance the contrast; 3) applying adjustment: applying the adjusted histogram to the original image to obtain a contrast-enhanced image; the automatic contrast adjustment method is implemented by a third-party image processing software installed in the system.

7. The automatic detection and calibration system for elevator guide rails based on machine vision according to claim 1, characterized in that: The feature extraction and matching module (3) uses the SURF feature point extraction algorithm to extract the edge and corner feature information of the elevator guide rail from the preprocessed image, and uses the RANSAC algorithm to match the extracted features to determine the accurate position and shape of the elevator guide rail. Through the matching results, the deviation or defect of the guide rail can be identified; wherein, the SURF algorithm uses the determinant value of the Hessian matrix as the basis for feature point detection, and uses the integral map to accelerate the operation, and the steps include: 1) feature point detection: using the determinant value of the Hessian matrix to detect feature points; 2) feature point positioning: locating feature points by interpolating feature point neighboring information; 3) direction positioning: determining the direction by calculating the Haar wavelet transform of pixels around the feature point; 4) feature descriptor: generating a 64-dimensional descriptor for describing the image information around the feature point; the RANSAC algorithm estimates the mathematical model parameters from the sample containing abnormal data in an iterative manner, thereby obtaining the best data matching result, including five steps of randomly selecting sample points, calculating the homography matrix, evaluating the inner point and the outer point, iterative optimization, and determining the position and shape of the elevator guide rail.

Images