Elevator steel wire rope defect detection device and method combining AR coding and magnetic flux leakage

Abstract

The invention discloses an elevator steel wire rope defect detection device and method combining AR coding and magnetic flux leakage. The elevator steel wire rope defect detection device comprises a magnetic flux leakage detection sensor array device and a steel wire rope movement robot. The magnetic flux leakage detection sensor array device is installed on the steel wire rope movement robot and used for obtaining magnetic flux leakage signals of a steel wire rope in real time. And the steel wire rope movement robot is used for carrying the high-precision industrial camera and the magnetic flux leakage detection sensor array and moving along the steel wire rope to realize full-coverage detection of the steel wire rope. According to the invention, through excessive sensor data fusion, a deep learning algorithm and a virtual information generation technology, high-precision and real-time detection and positioning of elevator steel wire rope defects are realized, and the detection efficiency is remarkably improved. The device not only can detect the broken wire of the elevator steel wire rope, but also can detect the crack and stress change conditions of the steel wire rope, and gives an early warning for the safety condition of the elevator steel wire rope.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator safety detection, and in particular to an elevator wire rope defect detection device and method combining AR coding and magnetic leakage. Background Art

[0002] Elevators, an indispensable means of vertical transportation in modern high-rise buildings, are of paramount importance for their safety. The condition of the wire rope, the elevator's primary load-bearing component, is directly linked to its safe operation. However, wire ropes are prone to defects such as wear, corrosion, and cracks during use. These defects, if not discovered and addressed promptly, can lead to serious safety accidents. Therefore, regular inspection and maintenance of elevator wire ropes is crucial.

[0003] Traditional wire rope inspection methods primarily rely on manual visual inspection and simple nondestructive testing techniques (such as ultrasonic testing and magnetic particle testing), which have significant technical limitations. Manual visual inspection can only detect surface damage (such as broken strands and rust), but cannot detect internal broken wires, wear, or fatigue cracks. This results in low inspection efficiency, is dependent on the inspector's experience, and is highly subjective. Patent CN119399114A proposes a binocular vision-based inspection technology designed to address the efficiency and coverage issues of traditional magnetic particle testing. This existing technology only allows for local inspection, failing to cover the entire wire rope. It requires the application of magnetic powder, which is complex to operate and significantly interferes with oil or rust, making it difficult to quantitatively analyze the extent of defects. Patent CN117765435A (A Vision-Based Elevator Wire Rope Defect Detection Method) points out that ultrasonic testing lacks separation capabilities in complex backgrounds and requires the assistance of computer vision. This existing technology requires specialized equipment and complex signal processing, is sensitive to surface oil or rust, and its detection accuracy is affected by coupling agents. It also cannot detect dynamic defects in real time. These methods have disadvantages such as low detection accuracy, low efficiency, and inability to detect in real time; the detection system is complex and cannot be integrated with the existing control system, and the use and operation are complicated and the degree of automation is not high. Summary of the Invention

[0004] In order to overcome the above technical problems, the purpose of the present invention is to provide an elevator wire rope defect detection device and method that combines AR coding and magnetic leakage. Through multi-sensor data fusion, deep learning algorithms and virtual information generation technology, high-precision, real-time detection and positioning of elevator wire rope defects are achieved, significantly improving detection efficiency. Not only can broken wires in elevator wire ropes be detected, but cracks and stress changes in wire ropes can also be detected, providing early warning of the safety status of elevator wire ropes. Its application is not only suitable for elevator maintenance in various high-rise buildings, but can also be extended to other mechanical equipment safety detection fields that require wire ropes to bear weight. It has broad application prospects and significant social and economic benefits.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] An elevator wire rope defect detection device combining AR coding and magnetic leakage includes a magnetic leakage detection sensor array device 1 and a wire rope motion robot 2; the magnetic leakage detection sensor array device 1 is installed on the wire rope motion robot 2 to obtain the magnetic leakage signal of the wire rope in real time;

[0007] The wire rope motion robot 2 is used to carry a camera and a magnetic flux leakage detection sensor array, and move along the wire rope to achieve full coverage detection of the wire rope.

[0008] The high-precision industrial camera is used to capture real-time video frames of the wire rope, and extract the contour information of the wire rope through Gaussian blur filtering and Canny edge detection algorithm to assist in the preliminary positioning of defects;

[0009] The provided video stream serves as the underlying image for AR technology. Detected defect information (such as location, size, and type) is superimposed on the real-time video through virtual markers (such as highlight boxes and text annotations) and presented intuitively to users.

[0010] Combined with the camera calibration parameters (intrinsic parameter matrix K, extrinsic parameter matrix R / t), the defect coordinates in the 2D image are converted into 3D space coordinates to achieve accurate defect positioning;

[0011] The video data and the magnetic flux leakage signal are fused through a Kalman filter or a particle filter to improve the robustness of the detection of wire rope position, posture and defects.

[0012] The magnetic flux leakage detection sensor array is used to apply a magnetic field on the surface of the wire rope through an excitation module to detect magnetic field distortion signals (magnetic flux leakage signals) caused by defects (such as broken wires, cracks, and corrosion), reflecting the damage inside and on the surface of the wire rope.

[0013] The magnetic flux leakage signal is sensitive to minute defects in wire ropes (such as microcracks and stress concentrations), enabling detection of hidden defects difficult to detect using traditional methods. Combined with a correction formula, it eliminates noise interference and improves detection accuracy. The magnetic flux leakage signal's amplitude, range, and peak intensity are used to estimate physical parameters such as defect length and depth. The magnetic flux leakage signal is fused with camera images and IMU attitude data to enable cross-modal defect verification (e.g., verifying the consistency of defect locations indicated by the magnetic flux leakage signal through video), reducing false detection rates.

[0014] The magnetic flux leakage detection sensor array device 1 includes a detection device mounting bracket 101, a clamping device driving motor 102, a clamping device driving gear 103, a clamping device driving rack 104, a clamping device guide block 105, a clamping arm 106, a clamping arm 2 107, a magnetic flux leakage sensor array mounting bracket 108, a magnetic flux leakage sensor array mounting bracket 2 109, an excitation module 110, a magnetic flux leakage sensor array 111, and a high-precision industrial camera 112;

[0015] The clamping device driving motor 102 is mounted on the detection device mounting bracket 101, the clamping device driving gear 103 is fixedly connected to the clamping device driving motor 102, the clamping device guide block 105 is fixedly mounted on the detection device mounting bracket 101, the clamping device driving rack 104 is movably connected to the clamping device guide block 105; the clamping device driving gear 103 cooperates with the clamping device driving rack 104; the clamping arm 106 and the clamping arm 2 107 are movably connected to the clamping device driving rack 10;

[0016] The shape structure of the clamping arm 106 and the clamping arm 2 107 is two arcuate parallel clamping claws.

[0017] The leakage magnetic sensor array mounting bracket 108 is fixedly installed at the end of the clamping arm 106, and the leakage magnetic sensor array mounting bracket 2 is fixedly installed at the end of the clamping arm 107; one excitation module 110 is respectively installed at both ends of the leakage magnetic sensor array mounting bracket 108, and a group of the leakage magnetic sensor array 111 is installed in the middle part of the leakage magnetic sensor array mounting bracket 108; one excitation module 110 is respectively installed at both ends of the leakage magnetic sensor array mounting bracket 2 109, and a group of the leakage magnetic sensor array 111 is installed on the leakage magnetic sensor array mounting bracket 2 109.

[0018] The wire rope motion robot 2 includes a mounting bracket 201, a driving wheel 1 202, a driving wheel 203, a guide device 1 204, a pressure wheel 205, a control module 206, a tensioning wheel 207, a guide device 208, a robot mounting plate 1 209, a mounting bearing 1 210, a mounting bearing 211, a robot mounting plate 212, a driving motor 1 213, and a driving motor 214.

[0019] The second mounting bracket 201 is fixedly mounted on the robot mounting plate 1 209, and the detection device mounting bracket 101 is fixedly mounted on the front end of the second mounting bracket 201; the drive motor 1 213 and the drive motor 2 214 are fixedly mounted on the outside of the robot mounting plate 1 209, the drive wheel 1 202 is fixedly connected to the drive motor 1 213 via a connecting shaft, and is movably connected to the robot mounting plate 2 212 via a connecting shaft and the mounting bearing 1 210; the drive wheel 203 is fixedly connected to the drive motor 2 214 via a connecting shaft, and is movably connected to the robot mounting plate 2 212 via a connecting shaft and the mounting bearing 2 211; the guide device 1 204 is movably mounted on one end of the robot mounting plate 1 209, and the guide device 2 208 is movably mounted on the other end of the robot mounting plate 1 209;

[0020] The pressure wheel 205 is movably mounted on the rear end of the second mounting bracket 201; the control module 206 is mounted on the lower side of the first robot mounting plate 209, and the control module 206 includes an inertial measurement unit (IMU), a data processing unit, and a wireless transmission unit.

[0021] The control module 206 is used to obtain the robot's posture and position information in real time, assist in the spatial alignment and posture adjustment of sensor data, and is used to fuse video data and leakage magnetic signals, estimate the position and posture of the wire rope, perform defect detection and positioning, generate virtual markers, and perform video encoding and real-time signal transmission; the tensioning wheel 207 is movably connected to the robot mounting plate 1 209 at the lower middle part of the drive motor 1 213 and the drive motor 2 214; the robot mounting plate 212 is fixedly connected to the robot mounting plate 1 209.

[0022] The driving wheel 1 202 is located at the front end of the robot device, and the driving wheel 2 203 is located at the rear side of the driving wheel 1 202. The driving wheel 1 202 and the driving wheel 2 203 are arranged axially in parallel. The wheel surfaces of the driving wheel 1 202 and the driving wheel 2 203 are both provided with grooves. The wire rope is in the groove and is driven to rotate by a motor to drive the robot device to walk on the wire rope; the guide device 1 204 is located at the front end of the robot device, and the wire rope passes through the guide device 1 204 to guide the robot device to walk on the wire rope; the pressure wheel 205 is located on one side of the driving wheel 1 202 to press the wire rope into the groove of the driving wheel 1 202. Tensioning pulley 207 is located between drive wheel 1 202 and drive wheel 2 203. As the wire rope passes through the grooves of drive wheels 1 202 and 203, it passes around tensioning pulley 207, tightening the wire rope. This allows friction between drive wheels 1 202 and 203 and the wire rope to propel the robot along the rope. Guide device 2 208 is located at the end of the robot, through which the wire rope passes, providing guidance.

[0023] A method for detecting elevator wire rope defects by combining augmented reality (AR) video coding and magnetic flux leakage comprises the following steps:

[0024] Step 1: System initialization and calibration

[0025] First, a high-precision industrial camera, a magnetic flux leakage sensor array 1, and an inertial measurement unit (IMU) were installed on a wire rope robot 2, ensuring that these sensors fully cover the wire rope's detection area. The magnetic flux leakage sensor array was evenly spaced along the wire rope to ensure comprehensive detection of surface and internal defects. Simultaneously, the IMU was connected to obtain real-time posture and position information about the robot on the wire rope, assisting with spatial alignment of sensor data and posture adjustment.

[0026] Next, the device is calibrated. The camera is calibrated using a calibration plate to obtain the camera's intrinsic parameter matrix K and extrinsic parameter matrix R, t to ensure the consistency of the image with the actual space coordinate system.

[0027] Camera calibration involves determining focal length, optical center position, and distortion parameters. For magnetic flux leakage (MFL) sensor arrays, calibration is performed to determine the transformation matrix A and offset b to ensure the accuracy of the MFL signal. This calibration process involves collecting MFL signals from a sample with known defect locations and dimensions and performing linear or nonlinear fitting to determine the correction parameters.

[0028] Step 2: Perform the robot inspection. Remove the second robot mounting plate 212 and pass the elevator wire rope through the first magnetic flux leakage sensor array mounting bracket 108, the middle of the second magnetic flux leakage sensor array mounting bracket 109, through the first guide 204, between the first drive wheel 202, the second drive wheel 203, and the tensioning wheel 207, and through the 208. The pressure wheel 205 firmly presses the wire rope into the groove of the first drive wheel 202. Then, install the second robot mounting plate 212.

[0029] The robot is then powered on, and the high-precision industrial camera 112 is powered on to capture images of the wire rope in real time, performing image preprocessing and edge detection. Drive motor 1 213 rotates drive wheel 1 202 , and drive motor 2 214 rotates drive wheel 2 203 , thereby driving the robot to walk on the wire rope.

[0030] During testing, the clamping device drive motor 102 is energized, and the clamping device drive gear 103 drives the mating clamping device drive rack 104 downward along the clamping device guide block 105, thereby rotating the clamping arm 106 and the second clamping arm 107. This causes the excitation module 110 and the magnetic flux leakage sensor array 111, mounted on the magnetic flux leakage sensor array mounting bracket 108 and the magnetic flux leakage sensor array mounting bracket 2 109, to clamp the wire rope, creating a uniform, small gap between the excitation module 110 and the magnetic flux leakage sensor array 111 and the wire rope surface. The excitation module 110 is then energized, generating an excitation magnetic field on the wire rope surface, and the magnetic flux leakage sensor array 111 collects signals.

[0031] Step 3: Real-time data collection and preprocessing

[0032] After the system initialization is completed, real-time data acquisition begins. The high-precision industrial camera captures the real-time video frame of the wire rope and applies a Gaussian blur filter to reduce image noise:

[0033] I blur =G σ *I t

[0034] Among them, G σ is the Gaussian kernel, * represents the convolution operation, and σ is the standard deviation of the Gaussian kernel. Subsequently, the Canny operator is used for edge detection:

[0035] I edges =G anny (I blur )

[0036] Edge detection helps to extract the contour information of the wire rope and assists in the location and identification of defects.

[0037] At the same time, the magnetic flux leakage detection sensor array device 1 obtains the original magnetic flux leakage signal M in real time. mea (x,y), and the real magnetic flux leakage signal is obtained through the correction formula:

[0038] M true (x,y)=A·M mea (x,y)+b

[0039] The data fusion part uses a Kalman filter or a particle filter to combine the video data and the magnetic flux leakage signal to estimate the precise position and attitude of the wire rope:

[0040] x k =Ax k-1 +Bu k +ω k

[0041] Among them, x k Contains the position and attitude information of the wire rope, A is the state transfer matrix, B is the control matrix, u k is the control input, ω k Data fusion helps improve the accuracy of defect detection and the robustness of the system, filter out noise interference, and provide stable detection results.

[0042]

[0043] Where θ is a preset threshold value, which is determined based on the actual application scenario and experimental results. Then, Fourier transform filtering is applied to remove high-frequency noise:

[0044] F{M abnormal (x,y)}=F(u,v)

[0045] F filtered (u,v)=F(u,v)·H(ω)

[0046] M filtered (x,y)=F -1 {F filtered (u,v)}

[0047] Where F and F-1 represent the Fourier transform and inverse Fourier transform, respectively, and H(ω) is a low-pass filter used to remove high-frequency noise. Through the above processing, the accuracy of detection is further improved.

[0048] The location of the defect is achieved by converting the 2D image coordinates (u,v) into 3D space coordinates (X,Y,Z):

[0049]

[0050] Where Z is provided by the depth sensor to ensure the accuracy of the defect location. The defect size estimation includes the calculation of length, depth and image size. The length L is calculated by integrating the amplitude variation range of the leakage magnetic signal:

[0051]

[0052] Where k1 is the proportional constant, x1 and x2 are the starting and ending positions of the defect. The depth D is estimated by the peak intensity Mpeak of the magnetic flux leakage signal:

[0053] D=k2·M peak

[0054] Where k2 is a proportional constant. The image size Areal is obtained by identifying the pixel area Apixel of the defect area in the image and converting it into the actual area:

[0055] A real =A pixel Pixel density Distance factor

[0056] Step 4: Virtual Information Generation and AR Overlay

[0057] The detected defect information is intuitively displayed to the user through augmented reality (AR) technology. First, a virtual marker is generated based on the 3D defect location and the size of the virtual marker is adjusted according to the actual size:

[0058] Mark size = actual size × L scale factor × D scale factor

[0059] Among them, the L scale factor and the sub-D scale factor are set according to the display ratio and resolution of the AR display device to ensure that the size of the virtual mark is consistent with the actual defect ratio. The depth information Z is predicted by the deep neural network fdepth:

[0060] Z=f depth (I blur )

[0061] Finally, the virtual marker is superimposed into the video frame:

[0062] I final (x,y)=I blur (x,y)+α·label(x,y)

[0063] Among them, α is the overlay transparency, which ensures the natural fusion effect of the virtual marker and the actual image.

[0064] Step 5: Video encoding and transmission

[0065] To achieve efficient video transmission, the enhanced AR video stream I final (x,y) is encoded using HEVC (H.265):

[0066] I encoded =HEVC_encode(I final )

[0067] HEVC uses inter-frame prediction and transform coding techniques to achieve higher compression ratios while maintaining video quality. To adapt to different network bandwidth conditions, adaptive bitrate (ABR) technology is used to dynamically adjust the resolution and bit rate of video encoding:

[0068] Bitrate = α·ABR current +(1-α)·ABR minimum

[0069] Among them, α is the adaptive factor, ABR current is the current bit rate, ABR minimum Finally, use a low-latency, high-reliability transmission protocol (such as WebRTC) to transmit the encoded video stream I encoded Real-time transmission to AR display devices ensures the continuity and smoothness of video streaming.

[0070] Step 6. Data recording and intelligent analysis

[0071] All inspection data, video streams, and virtual information are stored in a local server or cloud database for easy access and analysis. The system uses historical defect data and environmental factors to train a random forest regression model or a support vector regression model to predict the remaining useful life (RUL) of the wire rope:

[0072] RUL=Model(F defect (t),environment(t))

[0073] Among them, F defect (t) contains the current defect characteristics (location, size, type, and severity), environment(t) includes working environment factors, and t is time. The real-time prediction component provides a wire rope RUL prediction based on the current defect characteristics and environmental factors. The system also performs time series analysis to analyze the changing trends in defect quantity and severity over time. It uses models such as ARIMA and LSTM to predict potential failure risks, providing a scientific basis for maintenance decisions.

[0074] Beneficial effects of the present invention:

[0075] The elevator wire rope defect detection device and method provided by the present invention, which combines augmented reality (AR) video coding with magnetic flux leakage detection technology, has the following significant beneficial effects:

[0076] High-precision detection and real-time positioning

[0077] By integrating magnetic flux leakage detection technology with a high-precision industrial camera and employing an inertial measurement unit (IMU) to acquire the robot's posture and position in real time, this invention enables high-precision detection of surface and internal defects in elevator ropes. The real-time acquisition of magnetic flux leakage signals and the deep integration of video images result in more accurate detection results and improve the precision of defect location, particularly in complex rope environments.

[0078] 1. Augmented reality (AR) technology supports intuitive display

[0079] This technology overlays detected defect information onto a live video stream as virtual markers, leveraging augmented reality (AR) technology to provide users with an intuitive and easy-to-understand representation of the defects. The virtual markers are the same size as the actual defects, significantly enhancing the visualization of inspection results and enabling maintenance personnel to quickly identify and determine the severity and repair priority of defects.

[0080] 2. Intelligent data fusion and processing

[0081] This method uses a Kalman filter or particle filter to fuse multi-sensor data, effectively improving the robustness and accuracy of the detection system. During the simultaneous processing of magnetic flux leakage signals and video images, filtering and Fourier transforms are used to remove noise interference, ensuring data accuracy and stability. By accurately estimating the position and posture of the wire rope, defects can be precisely identified and located.

[0082] 3.Automated and full coverage detection

[0083] Combined with a wire rope motion robot, this invention achieves full coverage inspection of elevator wire ropes, eliminating the blind spots and omissions often associated with manual inspection. The robot can move freely along the length of the wire rope and, equipped with a magnetic flux leakage sensor array and industrial camera, conducts efficient, all-around inspections, improving both efficiency and safety.

[0084] 4. Efficient video encoding and real-time transmission

[0085] This system uses advanced HEVC (H.265) video coding technology, combined with adaptive bitrate (ABR) adjustment and low-latency transmission protocols such as WebRTC, to ensure efficient transmission of real-time detection results. Even in bandwidth-constrained environments, the system can ensure the clarity and stability of the video stream, facilitating remote monitoring and analysis.

[0086] 5. Intelligent prediction and maintenance decision support

[0087] This system combines historical test data with environmental factors through data logging and intelligent analysis, and uses machine learning models (such as random forest regression and support vector regression) to predict the remaining useful life (RUL) of wire ropes. Furthermore, based on time series analysis methods such as ARIMA and LSTM, the system can predict potential failure risks and defect development trends, providing a scientific basis for elevator maintenance and management, and effectively avoiding safety hazards caused by wire rope failures.

[0088] In summary, the present invention provides an efficient, accurate and intuitive elevator wire rope defect detection solution through an innovative combination of technologies, which significantly improves detection efficiency, reduces labor costs, and also provides strong technical support for the safe operation of elevators. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 Schematic diagram of a wire rope motion robot.

[0090] Figure 2 Schematic diagram of a wire rope motion robot.

[0091] Figure 3 Schematic diagram of a wire rope motion robot.

[0092] Figure 4 Schematic diagram of a wire rope motion robot.

[0093] Figure 5 Schematic diagram of a wire rope motion robot. DETAILED DESCRIPTION

[0094] The present invention will be described in further detail below with reference to the accompanying drawings.

[0095] like Figure 1-5 As shown, the detection device includes a magnetic flux leakage detection sensor array device 1 and a wire rope motion robot 2; the magnetic flux leakage detection sensor array device 1 is installed on the wire rope motion robot and is used to obtain the magnetic flux leakage signal of the wire rope in real time. The magnetic flux leakage detection sensor array device 1 includes a detection device mounting bracket 101, a clamping device drive motor 102, a clamping device drive gear 103, a clamping device drive rack 104, a clamping device guide block 105, a clamping arm 106, a clamping arm 2 107, a magnetic flux leakage sensor array mounting bracket 108, a magnetic flux leakage sensor array mounting bracket 2 109, an excitation module 110, a magnetic flux leakage sensor array 111, and a high-precision industrial camera 112;

[0096] The clamping device driving motor 102 is installed on the detection device mounting bracket 101, and the clamping device driving gear 103 is fixedly connected to the clamping device driving motor 102. The clamping device guide block 105 is fixedly mounted on the detection device mounting bracket 101, and the clamping device driving rack 104 is movably connected to the clamping device guide block 105; the clamping device driving gear 103 cooperates with the clamping device driving rack 104; the clamping arm 106 and the clamping arm 2 107 are movably connected to the clamping device driving rack 10; the leakage magnetic sensor array mounting bracket 108 is fixedly mounted on the end of the clamping arm 106, and the leakage magnetic sensor array mounting bracket 2 is fixedly mounted on the end of the clamping arm 2 107; an excitation module 110 is respectively mounted at both ends of the leakage magnetic sensor array mounting bracket 108, and a group of the leakage magnetic sensor array 111 is mounted in the middle part of the leakage magnetic sensor array mounting bracket 108; an excitation module 110 is respectively mounted at both ends of the leakage magnetic sensor array mounting bracket 2 109, and a group of the leakage magnetic sensor array 111 is mounted on the leakage magnetic sensor array mounting bracket 2 109.

[0097] The wire rope motion robot 2 is used to carry a camera and a magnetic flux leakage detection sensor array, move along the wire rope, and achieve full coverage detection of the wire rope. The wire rope motion robot 2 includes a second mounting bracket 201, a first driving wheel 202, a second driving wheel 203, a first guide device 204, a pressure wheel 205, a control module 206, a tensioning wheel 207, a second guide device 208, a first robot mounting plate 209, a first mounting bearing 210, a second mounting bearing 211, a second robot mounting plate 212, a first drive motor 213, and a second drive motor 214.

[0098] The mounting bracket 201 is fixedly mounted on the robot mounting plate 1 209, and the detection device mounting bracket 101 is fixedly mounted on the front end of the mounting bracket 201; the drive motor 1 213 and the drive motor 2 214 are fixedly mounted on the outside of the robot mounting plate 1 209, and the drive wheel 1 202 is fixedly connected to the drive motor 1 213 through a connecting shaft, and is movably connected to the robot mounting plate 2 212 through a connecting shaft and the mounting bearing 1 210; the drive wheel 203 is fixedly connected to the drive motor 2 214 through a connecting shaft, and is movably connected to the robot mounting plate 2 212 through a connecting shaft and the mounting bearing 2 211; the guide device 1 204 is movably mounted on one end of the robot mounting plate 1 209, and the guide device 2 208 is movably mounted on the robot mounting plate 1 209 the other end of the robot; the pressure wheel 205 is movably mounted on the rear end of the mounting bracket 201; the control module 206 is mounted on the lower side of the robot mounting plate 1 209, and the control module 206 internally includes an inertial measurement unit (IMU), a data processing unit and a wireless transmission unit; the control module 206 is mainly used for real-time acquisition of the robot's posture and position information, auxiliary spatial alignment and posture adjustment of sensor data, and for fusing video data and leakage magnetic signals, estimating the position and posture of the wire rope, performing defect detection and positioning, generating virtual markers, and performing video encoding and real-time signal transmission; the tensioning wheel 207 is movably connected to the robot mounting plate 1 209 at the lower middle part of the drive motor 1 213 and the drive motor 2 214; the robot mounting plate 212 is fixedly connected to the robot mounting plate 1 209.

[0099] The working principle of the present invention is as follows:

[0100] When testing is required, remove the robot mounting plate 212 and pass the elevator wire rope through the magnetic flux leakage sensor array mounting bracket 108, the magnetic flux leakage sensor array mounting bracket 209, the guide device 104, and the space between the drive wheel 102, the drive wheel 203, and the tensioning wheel 207, and finally through the 208. The pressure wheel 205 firmly presses the wire rope into the groove of the drive wheel 102. Then, install the robot mounting plate 212.

[0101] The robot is then powered on, and the high-precision industrial camera 112 is powered on to capture images of the wire rope in real time, performing image preprocessing and edge detection. Drive motor 1 213 rotates drive wheel 1 202 , and drive motor 2 214 rotates drive wheel 2 203 , thereby driving the robot to walk on the wire rope.

[0102] During testing, the clamping device drive motor 102 is energized, and the clamping device drive gear 103 drives the mating clamping device drive rack 104 downward along the clamping device guide block 105, thereby rotating the clamping arm 106 and the second clamping arm 107. This causes the excitation module 110 and the magnetic flux leakage sensor array 111, mounted on the magnetic flux leakage sensor array mounting bracket 108 and the magnetic flux leakage sensor array mounting bracket 2 109, to clamp the wire rope, creating a uniform, small gap between the excitation module 110 and the magnetic flux leakage sensor array 111 and the wire rope surface. The excitation module 110 is then energized, generating an excitation magnetic field on the wire rope surface, and the magnetic flux leakage sensor array 111 collects signals.

[0103] The elevator wire rope defect detection device of the present invention achieves high-precision, real-time defect detection and positioning through the following specific implementation steps:

[0104] Step 1: System initialization and calibration

[0105] First, a high-precision industrial camera, a magnetic flux leakage sensor array, and an inertial measurement unit (IMU) were installed on the wire rope robot to ensure that these sensors fully cover the wire rope's detection area. The magnetic flux leakage sensor array was evenly spaced along the wire rope to ensure comprehensive detection of surface and internal defects. Simultaneously, the IMU was connected to obtain real-time posture and position information of the robot on the wire rope, assisting with spatial alignment of sensor data and posture adjustment.

[0106] Next, the equipment is calibrated. Using a calibration plate, the camera is calibrated to obtain the camera's intrinsic parameter matrix K and extrinsic parameter matrices R and t, ensuring image consistency with the real-world coordinate system. Camera calibration includes determining focal length, optical center position, and distortion parameters. For magnetic flux leakage detection sensors, calibration is performed to determine the transformation matrix A and offset b to ensure the accuracy of the magnetic flux leakage signal. The calibration process collects magnetic flux leakage signals on a sample with known defect locations and dimensions and performs linear or nonlinear fitting to determine the correction parameters.

[0107] Step 2: Real-time data collection and preprocessing

[0108] After the system initialization is completed, real-time data acquisition begins. High-precision industrial cameras capture real-time video frames of the wire rope. t , and apply a Gaussian blur filter to reduce image noise:

[0109] I blur =G σ *I t

[0110] Among them, G σis the Gaussian kernel, * represents the convolution operation, and σ is the standard deviation of the Gaussian kernel. Subsequently, the Canny operator is used for edge detection:

[0111] I edges =G anny (I blur )

[0112] Edge detection helps to extract the contour information of the wire rope and assists in the location and identification of defects.

[0113] At the same time, the magnetic flux leakage detection sensor array obtains the original magnetic flux leakage signal M in real time mea (x,y), and the real magnetic flux leakage signal is obtained through the correction formula:

[0114] M true (x,y)=A·M mea (x,y)+b

[0115] The data fusion part uses a Kalman filter or a particle filter to combine the video data and the magnetic flux leakage signal to estimate the precise position and attitude of the wire rope:

[0116] x k =Ax k-1 +Bu k +ω k

[0117] Among them, x k Contains the position and attitude information of the wire rope, A is the state transfer matrix, B is the control matrix, u k is the control input, ω k Data fusion helps improve the accuracy of defect detection and the robustness of the system, filter out noise interference, and provide stable detection results.

[0118]

[0119] Where θ is a preset threshold value, which is determined based on the actual application scenario and experimental results. Then, Fourier transform filtering is applied to remove high-frequency noise:

[0120] F{M abnormal (x,y)}=F(u,v)

[0121] F filtered (u,v)=F(u,v)·H(ω)

[0122] M filtered (x,y)=F -1 {F filtered (u,v)}

[0123] Among them, F and F -1Denotes Fourier transform and inverse Fourier transform respectively, and H(ω) is a low-pass filter used to remove high-frequency noise. Through the above processing, the accuracy of detection is further improved.

[0124] The location of the defect is achieved by converting the 2D image coordinates (u,v) into 3D space coordinates (X,Y,Z):

[0125]

[0126] Where Z is provided by the depth sensor to ensure the accuracy of the defect location. The defect size estimation includes the calculation of length, depth and image size. The length L is calculated by integrating the amplitude variation range of the leakage magnetic signal:

[0127]

[0128] Where k1 is the proportional constant, x1 and x2 are the starting and ending positions of the defect. The depth D is determined by the peak intensity M of the leakage magnetic signal. peak Estimate:

[0129] D=k2·M peak

[0130] Where k2 is the proportional constant. Image size A real By identifying the pixel area A of the defect area in the image pixel And convert to actual area:

[0131] A real =A pixel Pixel density Distance factor

[0132] Step 4: Virtual Information Generation and AR Overlay

[0133] The detected defect information is intuitively displayed to the user through augmented reality (AR) technology. First, a virtual marker is generated based on the 3D defect location and the size of the virtual marker is adjusted according to the actual size:

[0134] Mark size = actual size × L scale factor × D scale factor

[0135] Among them, the L scale factor and the D scale factor are set according to the display ratio and resolution of the AR display device to ensure that the size of the virtual mark is consistent with the actual defect ratio. The depth information Z is obtained through the deep neural network f depth predict:

[0136] Z=f depth (I blur )

[0137] Finally, the virtual marker is superimposed into the video frame:

[0138] Ifinal (x,y)=I blur (x,y)+α·label(x,y)

[0139] Among them, α is the overlay transparency, which ensures the natural fusion effect of the virtual marker and the actual image.

[0140] Step 5: Video encoding and transmission

[0141] To achieve efficient video transmission, the enhanced AR video stream I final (x,y) is encoded using HEVC (H.265):

[0142] I encoded =HEVC_encode(I final )

[0143] HEVC uses inter-frame prediction and transform coding techniques to achieve higher compression ratios while maintaining video quality. To adapt to different network bandwidth conditions, adaptive bitrate (ABR) technology is used to dynamically adjust the resolution and bit rate of video encoding:

[0144] Bitrate = α·ABR current +(1-α)·ABR minimum

[0145] Among them, α is the adaptive factor, ABR current is the current bit rate, ABR minimum Finally, use a low-latency, high-reliability transmission protocol (such as WebRTC) to transmit the encoded video stream I encoded Real-time transmission to AR display devices ensures the continuity and smoothness of video streaming.

[0146] Step 6. Data recording and intelligent analysis

[0147] All inspection data, video streams, and virtual information are stored in a local server or cloud database for easy access and analysis. The system uses historical defect data and environmental factors to train a random forest regression model or a support vector regression model to predict the remaining useful life (RUL) of the wire rope:

[0148] RUL=Model(F defect (t),environment(t))

[0149] Among them, F defect(t) contains the current defect characteristics (location, size, type, and severity), environment(t) includes working environment factors, and t is time. The real-time prediction component provides a wire rope RUL prediction based on the current defect characteristics and environmental factors. The system also performs time series analysis to analyze the changing trends in defect quantity and severity over time. It uses models such as ARIMA and LSTM to predict potential failure risks, providing a scientific basis for maintenance decisions.

Claims

1. An elevator wire rope defect detection device combining AR coding and magnetic leakage, characterized in that: The invention comprises a magnetic flux leakage detection sensor array device (1) and a wire rope motion robot (2); the magnetic flux leakage detection sensor array device (1) is installed on the wire rope motion robot (2) and is used to obtain the magnetic flux leakage signal of the wire rope in real time; The wire rope motion robot (2) is used to carry a high-precision industrial camera and a magnetic flux leakage detection sensor array, and moves along the wire rope to achieve full coverage detection of the wire rope.

2. The elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to claim 1, characterized in that: The high-precision industrial camera is used to capture real-time video frames of the wire rope, and extract the contour information of the wire rope through Gaussian blur filtering and Canny edge detection algorithm to assist in the preliminary positioning of defects; The provided video stream serves as the underlying image for AR technology, and the detected defect information is superimposed on the real-time video through virtual markers, which is intuitively displayed to the user; Combined with camera calibration parameters, the defect coordinates in the 2D image are converted into 3D space coordinates to achieve accurate defect positioning; The video data and the magnetic flux leakage signal are fused through a Kalman filter or a particle filter.

3. The elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to claim 1, characterized in that: The magnetic flux leakage detection sensor array is used to apply a magnetic field on the surface of the wire rope through an excitation module, detect magnetic field distortion signals caused by defects, and reflect the damage inside and on the surface of the wire rope.

4. The elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to claim 1, characterized in that: The magnetic flux leakage detection sensor array device (1) comprises a detection device mounting bracket (101), a clamping device driving motor (102), a clamping device driving gear (103), a clamping device driving rack (104), a clamping device guide block (105), a clamping arm (106), a clamping arm (107), a magnetic flux leakage sensor array mounting bracket (108), a magnetic flux leakage sensor array mounting bracket (109), an excitation module (110), a magnetic flux leakage sensor array (111), and a high-precision industrial camera (112); The clamping device driving motor 1 (102) is installed on the detection device mounting bracket 1 (101), the clamping device driving gear (103) is fixedly connected to the clamping device driving motor 1 (102), the clamping device guide block (105) is fixedly installed on the detection device mounting bracket 1 (101), the clamping device driving rack (104) is movably connected to the clamping device guide block (105); the clamping device driving gear (103) cooperates with the clamping device driving rack (104); the clamping arm 1 (106) and the clamping arm 2 (107) are movably connected to the clamping device driving rack (10); The clamping arm 1 (106) and the clamping arm 2 (107) are two arcuate parallel clamping claws.

5. The elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to claim 4, characterized in that: The first magnetic flux leakage sensor array mounting bracket (108) is fixedly mounted on the end of the first clamping arm (106), and the second magnetic flux leakage sensor array mounting bracket (109) is fixedly mounted on the end of the second clamping arm (107); one excitation module (110) is respectively mounted on both ends of the first magnetic flux leakage sensor array mounting bracket (108), and a group of the magnetic flux leakage sensor arrays (111) is mounted on the middle portion of the first magnetic flux leakage sensor array mounting bracket (108); one excitation module (110) is respectively mounted on both ends of the second magnetic flux leakage sensor array mounting bracket (109), and a group of the magnetic flux leakage sensor arrays (111) is mounted on the second magnetic flux leakage sensor array mounting bracket (109).

6. The elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to claim 1, characterized in that: The wire rope motion robot (2) comprises a second mounting bracket (201), a first driving wheel (202), a second driving wheel (203), a first guide device (204), a pressure wheel (205), a control module (206), a tension wheel (207), a second guide device (208), a first robot mounting plate (209), a first mounting bearing (210), a second mounting bearing (211), a second robot mounting plate (212), a first driving motor (213), and a second driving motor (214); The second mounting bracket (201) is fixedly mounted on the first robot mounting plate (209), and the first detection device mounting bracket (101) is fixedly mounted on the front end of the second mounting bracket (201); the first drive motor (213) and the second drive motor (214) are fixedly mounted on the outside of the first robot mounting plate (209), the first drive wheel (202) is fixedly connected to the first drive motor (213) via a connecting shaft, and is movably connected to the second robot mounting plate (212) via a connecting shaft and the first mounting bearing (210); the second drive wheel (203) is fixedly connected to the second drive motor (214) via a connecting shaft, and is movably connected to the second robot mounting plate (212) via a connecting shaft and the second mounting bearing (211); the first guide device (204) is movably mounted on one end of the first robot mounting plate (209), and the second guide device (208) is movably mounted on the other end of the first robot mounting plate (209); The pressure wheel (205) is movably mounted on the rear end of the second mounting bracket (201); the control module (206) is mounted on the lower side of the first robot mounting plate (209), and the control module (206) includes an inertial measurement unit (IMU), a data processing unit, and a wireless transmission unit. The inertial measurement unit is used to measure the three-dimensional attitude angle and acceleration / angular velocity of the robot device in real time through a gyroscope and an accelerometer, and is used to dynamically sense the motion state of the robot on the wire rope, as well as sensor data alignment and compensation and motion control feedback; The data processing unit is used to fuse heterogeneous data from the magnetic flux leakage sensor array, camera and IMU, as well as perform defect detection and quantitative analysis, and make real-time decisions and early warnings; The wireless transmission unit is used for real-time data stream transmission, detection data return and storage, and remote command reception; The control module (206) is used to obtain the posture and position information of the robot in real time, assist in the spatial alignment and posture adjustment of sensor data, and to fuse video data and leakage magnetic signals, estimate the position and posture of the wire rope, perform defect detection and positioning, generate virtual markers, and perform video encoding and real-time signal transmission; the tensioning wheel (207) is movably connected to the robot mounting plate 1 (209) at a lower position between the drive motor 1 (213) and the drive motor 2 (214); the robot mounting plate 2 (212) is fixedly connected to the robot mounting plate 1 (209).

7. The elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to claim 6, characterized in that: The driving wheel 1 (202) is located at the front end of the robot device, and the driving wheel 2 (203) is located at the rear side of the driving wheel 1 (202). The driving wheel 1 (202) and the driving wheel 2 (203) are arranged axially in parallel. The wheel surfaces of the driving wheel 1 (202) and the driving wheel 2 (203) are both provided with grooves. The steel wire rope is in the groove and is driven to rotate by a motor to drive the robot device to walk on the steel wire rope; the guide device 1 (204) is located at the front end of the robot device, and the steel wire rope passes through the guide device 1 (204) to guide the robot device to walk on the steel wire rope; the pressure wheel (205) is located on one side of the driving wheel 1 (202) to press the steel wire rope into the groove of the driving wheel 1 (202); The tensioning wheel (207) is arranged in the middle part of the driving wheel 1 (202) and the driving wheel 2 (203). When the wire rope passes through the grooves of the driving wheel 1 (202) and the driving wheel 2 (203), it passes around the tensioning wheel (207) to tighten the wire rope, so that the driving wheel 1 (202) and the driving wheel 2 (203) drive the robot device to crawl on the wire rope through the friction force with the wire rope; the guide device 2 (208) is located at the end of the robot device, and the wire rope passes through the guide device 2 (208) to play a guiding role.

8. A method for detecting defects in elevator wire ropes based on the elevator wire rope defect detection device combining AR coding and magnetic flux leakage according to any one of claims 1 to 7, characterized in that: The following steps are involved: Step 1: First, a high-precision industrial camera (112), a magnetic flux leakage detection sensor array (1), and an inertial measurement unit (IMU) are installed on a wire rope motion robot (2). The magnetic flux leakage detection sensor array (1) is evenly arranged along the wire rope to ensure that the sensor can fully detect surface and internal defects of the wire rope. At the same time, the IMU is connected to obtain the posture and position information of the robot on the wire rope in real time, thereby assisting the spatial alignment and posture adjustment of the sensor data. Next, a calibration process of the equipment is performed, and the high-precision industrial camera (112) is calibrated using a calibration plate to obtain the internal parameter matrix K and the external parameter matrix R, t of the high-precision industrial camera (112) to ensure that the image is consistent with the actual space coordinate system; The calibration of a high-precision industrial camera (112) includes determining the focal length, the optical center position and the distortion parameters. For a magnetic flux leakage detection sensor array device (1), calibration is performed to determine the transformation matrix A and the offset b to ensure the accuracy of the magnetic flux leakage signal. The calibration process collects the magnetic flux leakage signal on a sample with known defect position and size, and performs linear or nonlinear fitting to determine the correction parameters. Step 2: Remove the robot mounting plate 2 (212), pass the elevator wire rope through the leakage magnetic sensor array mounting bracket 1 (108), the middle of the leakage magnetic sensor array mounting bracket 2 (109), pass through the guide device 1 (204), pass through the driving wheel 1 (202) and between the driving wheel 2 (203) and the tensioning wheel (207), and pass through the (208); the pressure wheel (205) presses the wire rope tightly into the groove of the driving wheel 1 (202); then install the robot mounting plate 2 (212); The robot device is powered on, and the high-precision industrial camera (112) is powered on to capture the image of the wire rope in real time, and perform image preprocessing and edge detection; after the image preprocessing and edge detection reduce noise by Gaussian blur, the Canny edge detection algorithm is used to extract the wire rope contour to provide a visual reference for defect positioning; The driving motor 1 (213) drives the driving wheel 1 (202) to rotate, and the driving motor 2 (214) drives the driving wheel 2 (203) to rotate, thereby driving the robot device to walk on the wire rope; During detection, the clamping device driving motor 1 (102) is energized, and the clamping device driving gear (103) drives the clamping device driving rack (104) matched therewith to move downward along the clamping device guide block (105), thereby driving the clamping arm 1 (106) and the clamping arm 2 (107) to rotate, so that the excitation module (110) and the leakage magnetic sensor array (111) installed on the leakage magnetic sensor array mounting bracket 1 (108) and the leakage magnetic sensor array mounting bracket 2 (109) clamp the wire rope, so that the excitation module (110) and the leakage magnetic sensor array (111) form a uniform small gap with the surface of the wire rope; then the excitation module (110) is energized to form an excitation magnetic field on the surface of the wire rope, and the leakage magnetic sensor array (111) collects signals; Step 3: After the system initialization is completed, real-time data acquisition begins. The high-precision industrial camera (112) captures the real-time video frame It of the wire rope and applies a Gaussian blur filter to reduce image noise: I blur =G σ *I t Among them, G σ is the Gaussian kernel, * represents the convolution operation, σ is the standard deviation of the Gaussian kernel, and then the Canny operator is used for edge detection: I edges =G anny (I blur ) Edge detection helps extract the contour information of the wire rope and assists in the location and identification of defects; At the same time, the magnetic flux leakage detection sensor array device (1) acquires the original magnetic flux leakage signal M in real time. mea (x,y), and the real magnetic flux leakage signal is obtained through the correction formula: M true (x,y)=A·M mea (x,y)+b The data fusion part uses a Kalman filter or a particle filter to combine the video data and the magnetic flux leakage signal to estimate the precise position and attitude of the wire rope: x k =Ax k-1 +Bu k +ω k Among them, x k Contains the position and attitude information of the wire rope, A is the state transfer matrix, B is the control matrix, u k is the control input, ω k is the process noise; Where θ is a preset threshold, which is determined based on the actual application scenario and experimental results. Then, Fourier transform filtering is applied to remove high-frequency noise: F{M abnormal (x,y)}=F(u,v) F filtered (u,v)=F(u,v)·H(ω) M filtered (x,y)=F -1 {F filtered (u,v)} Where F and F-1 represent Fourier transform and inverse Fourier transform respectively, and H(ω) is a low-pass filter used to remove high-frequency noise; The location of the defect is achieved by converting the 2D image coordinates (u,v) into 3D space coordinates (X,Y,Z): Among them, Z is provided by the depth sensor to ensure the accuracy of the defect location. The defect size estimation includes the calculation of length, depth and image size. The length L is calculated by integrating the amplitude variation range of the leakage magnetic signal: Where k1 is the proportional constant, x1 and x2 are the starting and ending positions of the defect, and the depth D is estimated by the peak intensity Mpeak of the magnetic flux leakage signal: D=k2·M peak Where k2 is a proportional constant, and the image size Areal is obtained by identifying the pixel area Apixel of the defect area in the image and converting it into the actual area: A real =A pixel Pixel density Distance factor Step 4: The detected defect information is intuitively displayed to the user through augmented reality technology; First, generate a virtual marker based on the 3D defect location and adjust the size of the virtual marker based on the actual size: Mark size = actual size × L scale factor × D scale factor The L scale factor and the sub-D scale factor are set according to the display ratio and resolution of the AR display device to ensure that the size of the virtual marker is consistent with the actual defect ratio. The depth information Z is predicted by the deep neural network fdepth: Z=f depth (I blur ) Finally, the virtual marker is superimposed into the video frame: I final (x,y) = I blur (x,y) + α·mark(x,y) Among them, α is the overlay transparency, which ensures the natural fusion effect of virtual markers and actual images; Step 5: Enhanced AR Video Stream I final (x,y) is encoded using HEVC (H.265): I encoded =HEVC_encode(I final ) Adaptive bitrate (ABR) technology is used to dynamically adjust the resolution and bit rate of video encoding: Bitrate = α·ABR current +(1-α)·ABR minimum Among them, α is the adaptive factor, ABR current is the current bit rate, ABR minimum The lowest bit rate is set, and finally, the encoded video stream I is transmitted using a low-latency, high-reliability transmission protocol. encoded Real-time transmission to AR display devices to ensure the continuity and smoothness of video streams; Step 6. All inspection data, video streams, and virtual information are stored on a local server or cloud database for subsequent access and analysis. The system uses historical defect data and environmental factors to train a random forest regression model or a support vector regression model to predict the remaining service life of the wire rope: RUL=Model(F defect (t),environment(t)) Among them, F defect (t) contains the characteristics of the current defect, environment(t) contains the working environment factors, t is the time, and the real-time prediction part provides the RUL prediction of the wire rope based on the current defect characteristics and environmental factors.

Images