A Machine Vision-Based Method and System for Defect Identification of Wire Ropes in Port Cranes

Abstract

This invention discloses a machine vision-based method and system for identifying defects in wire ropes of port cranes, belonging to the field of visual defect recognition technology. The method includes: real-time acquisition of wire rope images; real-time acquisition of the wire rope's motion state data, including acceleration and angular velocity, via an inertial measurement unit; prediction of the wire rope's current motion state using a Kalman filter algorithm based on the previously acquired motion state data; construction of a fuzzy kernel function based on the predicted current motion state, and sharpening of the currently acquired wire rope image using an inverse filtering algorithm; and defect identification using a target detection model on the sharpened image. This invention enables rapid and accurate detection of surface defects in wire ropes under dynamic fuzzy and complex environments.

Description

Technical Field

[0001] This invention belongs to the technical field of visual defect recognition, specifically relating to a method and system for identifying defects in steel wire ropes of port cranes based on machine vision. Background Technology

[0002] Cranes are critical loading and unloading equipment in ports and docks, and wire ropes, as important load-bearing components, are subjected to alternating loads, harsh natural environments, and frequent friction with other components over long periods. This makes them prone to defects such as wear, fatigue breakage, corrosion, deformation, and overload. The safety condition of the wire rope directly affects the stable operation of the crane system. A malfunction could lead to serious safety accidents, causing cargo damage, personal injury, and production stoppage.

[0003] The commonly used methods for defect identification are:

[0004] (1) Traditional visual methods: based on threshold segmentation (such as the Otsu algorithm) and morphological processing. The Otsu algorithm automatically calculates the threshold based on the image's gray-level histogram, and classifies the pixels in the image into different categories according to their gray-level values, in order to separate the target from the background and attempt to segment the defective region of the wire rope. Morphological processing, on the other hand, processes the image after threshold segmentation through operations such as erosion, dilation, opening, and closing, aiming to extract image features, eliminate noise, smooth boundaries, and segment regions of interest.

[0005] However, when using this method for crane wire rope inspection, the high-speed swing of the wire rope causes dynamic blurring in the image, resulting in unclear defect features and complex grayscale distribution. This makes it difficult for the Otsu algorithm based on a fixed threshold to accurately segment the defect region, easily leading to missegmentation or omission. Morphological processing is ineffective in handling noise and blurred edges in blurred images, easily destroying the true shape and features of the defect. At the same time, environmental interference such as changes in lighting and dust pollution can alter the image grayscale values ​​and texture, making traditional vision methods poorly adaptable and limiting their ability to process defect features at different scales.

[0006] Deep learning models include Faster R-CNN and U-Net. Faster R-CNN, based on a Region Proposal Network (RPN), first extracts image features through a convolutional neural network. The RPN generates candidate regions for classification and regression, followed by region-of-interest pooling, classifiers, and regressors to achieve accurate target classification and location regression. U-Net is a semantic segmentation model. Its U-shaped structure extracts high-level semantic features by shrinking the path, restoring resolution by expanding the path, and fusing features to classify image pixels. However, when processing dynamic detection of crane wire ropes, Faster R-CNN suffers from high computational cost and time consumption in generating candidate regions and subsequent classification and regression operations, resulting in slow detection speeds and difficulty meeting real-time requirements. U-Net's complex network structure, with multiple convolution, pooling, upsampling, and deconvolution operations, requires significant computational resources, leading to poor real-time performance and insufficient ability to detect minute defects.

[0007] Laser scanning technology: This technology uses a laser beam to scan the surface of a steel wire rope. By measuring information such as the time or phase difference of the laser beam's reflection, three-dimensional information about the surface of the steel wire rope can be obtained, thereby detecting whether the steel wire rope has defects such as wear or deformation.

[0008] However, when applied to scenarios involving the large-scale movement of cranes, the equipment costs are high, including laser scanners, data acquisition systems, and data processing software, making large-scale applications economically unsustainable. The equipment also has high installation and calibration requirements, which are time-consuming and labor-intensive to install and calibrate within the complex structure of cranes, and vibration and displacement during operation can reduce detection accuracy. Furthermore, it is susceptible to environmental factors such as strong light, dust, rain, and fog, which can lead to increased measurement errors and a shortened effective detection distance. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a machine vision-based method and system for identifying defects in wire ropes of port cranes, which can achieve rapid and accurate detection of surface defects in wire ropes in dynamic, fuzzy, and complex environments.

[0010] This invention provides the following technical solution:

[0011] Firstly, a machine vision-based method for identifying defects in wire ropes of port cranes is provided, including:

[0012] Real-time acquisition of wire rope images;

[0013] The motion state data of the wire rope, including its acceleration and angular velocity, is collected in real time by an inertial measurement unit. Based on the motion state data of the wire rope collected at the previous moment, the current motion state of the wire rope is predicted using a Kalman filter algorithm.

[0014] Based on the predicted current motion state of the wire rope, a fuzzy kernel function is constructed, and an inverse filtering algorithm is used to sharpen the real-time acquired wire rope image.

[0015] Defect identification is performed on sharpened images using a target detection model.

[0016] Optionally, the step of constructing a fuzzy kernel function based on the predicted current motion state of the wire rope and using an inverse filtering algorithm to sharpen the currently acquired wire rope image specifically includes:

[0017] Based on the predicted velocity at the current moment, a fuzzy kernel function is constructed according to the velocity components of the wire rope in the set X and set Y directions;

[0018]

[0019] Where K(x,y) is the blur level of the image at point (x,y), σ is a set coefficient related to velocity, and v x and v y These are the velocity components of the wire rope in the set X and set Y directions at the current moment;

[0020] After normalizing the fuzzy kernel function K(x,y), a Fourier transform is performed to obtain its frequency domain representation H(u,v), and the real-time acquired wire rope image is converted into the frequency domain form F(u,v).

[0021] In the frequency domain, based on the frequency domain form F(u,v) of the wire rope image and the frequency domain form H(u,v) of the blur kernel function, the frequency domain representation F(u,v) of the wire rope image is restored to the frequency domain representation G(u,v) of the clear image by using an inverse filtering algorithm or a Wiener filtering algorithm.

[0022] The frequency domain representation G(u,v) of the clear image is transformed back to the spatial domain by inverse Fourier transform, thereby achieving the sharpening process of the acquired wire rope image.

[0023] Optionally, in the frequency domain, the frequency domain representation F(u,v) of the wire rope image is restored to the frequency domain representation G(u,v) of the clear image using an inverse filtering algorithm, specifically as follows:

[0024]

[0025] Where ∈ represents the set minimum value.

[0026] Optionally, in the frequency domain, the frequency domain representation F(u,v) of the wire rope image is restored to the frequency domain representation G(u,v) of the clear image using the Wiener filtering algorithm, specifically as follows:

[0027]

[0028] Among them, H * (u,v) is the conjugate complex number of H(u,v), S n (u,v) is the noise power spectrum, S f (u,v) is the power spectrum of the acquired wire rope image.

[0029] Optionally, the defect identification of the sharpened image is performed using a target detection model, wherein the target detection model is an improved YOLOv8 model;

[0030] The improved YOLOv8 model uses MobileNetV3 as its backbone network and embeds the CBAM attention mechanism within the backbone network; the detection head of the improved YOLOv8 model includes an angle detection head for predicting the rotation angle of the wire rope.

[0031] Optionally, the improved YOLOv8 model, after training, is used for defect identification, specifically including:

[0032] Feature extraction is performed on the sharpened image, including the outline, shape, texture features, and local orientation information of the spiral texture of the steel wire rope;

[0033] The extracted features are input into the improved YOLOv8 model to predict the target bounding box and the rotation angle of the wire rope.

[0034] Based on the predicted target bounding box and rotation angle, the YOLOv8 model is improved to rotate the target bounding box in the post-processing stage to obtain the rotated box.

[0035] After adjusting the nonmaximum suppression of the rotating frame, the improved YOLOv8 model outputs the defect identification results of the wire rope and realizes the defect location based on the position of the rotating frame.

[0036] Optionally, based on the predicted target bounding box and rotation angle, the improved YOLOv8 model rotates the target bounding box during the post-processing stage to obtain the vertex coordinates (x′, y′) of the rotated bounding box:

[0037]

[0038] Where (x,y) are the vertex coordinates of the predicted target bounding box, x c and y c θ represents the center coordinates of the predicted target bounding box, and θ is the predicted rotation angle of the wire rope.

[0039] Optionally, the loss function L during training of the improved YOLOv8 model is:

[0040] L = L ce +L MSE +L rotation

[0041] Among them, L ce For the cross-entropy loss of defect classification, L MSE L is the mean squared error loss for position regression. rotation The mean square error loss for angle prediction;

[0042]

[0043] Where N is the number of samples, C is the number of categories, and y i,c p represents the true label (0 or 1) of the i-th sample in class c. i,c y represents the probability of the i-th sample in class c predicted by the model. i Let i be the true value of the i-th sample. The value of the i-th sample predicted by the model. Let be the rotation angle predicted by the model for the i-th sample. Let be the actual rotation angle of the i-th sample.

[0044] Optionally, the improved YOLOv8 model introduces an adversarial training strategy during training. Specifically, during training, image generation technology is used to add simulated noise such as haze and raindrops to the original wire rope image, and the image with noise is input into the improved YOLOv8 model along with the original image as training data.

[0045] Secondly, a machine vision-based defect identification system for port crane wire ropes is provided, including:

[0046] The acquisition module is used to acquire images of the wire rope in real time.

[0047] The state estimation module is used to collect the motion state data of the wire rope in real time through the inertial measurement unit, including the acceleration and angular velocity of the wire rope; based on the motion state data of the wire rope collected at the previous moment, the Kalman filter algorithm is used to predict the current motion state of the wire rope.

[0048] The sharpening module is used to construct a fuzzy kernel function based on the predicted current motion state of the wire rope, and to use an inverse filtering algorithm to sharpen the currently acquired wire rope image.

[0049] The defect identification module is used to identify defects in the sharpened image using a target detection model.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] (1) This invention innovatively integrates IMU and visual data. The IMU is used to acquire the motion state information of the wire rope in real time. Combined with visual image data, when the wire rope swings at high speed, the motion trajectory prediction information fed back by the IMU is used to dynamically compensate the blurred image, thereby improving the accuracy and reliability of defect identification.

[0052] (2) The present invention performs lightweight design of YOLOv8 model, replaces backbone network with MobileNetV3 to reduce number of parameters and improve inference speed; embeds CBAM attention mechanism to enhance feature response capability to different types of defect areas; and also uses angle prediction head to realize rotating box detection to adapt to the spiral texture feature of wire rope and improve detection performance and efficiency. Attached Figure Description

[0053] Figure 1 This is a flowchart of the steps of the machine vision-based method for identifying defects in steel wire ropes of port cranes according to the present invention.

[0054] Figure 2 This is a flowchart of the dynamic blur compensation process for images according to the present invention;

[0055] Figure 3 This is a structural block diagram of the improved YOLOv8 model of the present invention. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0057] It should be noted that the term "comprising" and any variations thereof in the specification and claims of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0058] Example 1

[0059] like Figure 1 and Figure 2 As shown, a machine vision-based method for identifying defects in wire ropes of port cranes is provided, including the following steps:

[0060] S1: Real-time acquisition of wire rope images.

[0061] Images of the wire rope are captured using a high-speed industrial camera with a frame rate of up to 2000fps. This allows for rapid shooting even when the wire rope is swinging at high speed, reducing motion blur. For example, when the wire rope is swinging at 5m / s, two images can be captured within 1ms, which is a significant advantage over ordinary 30fps cameras.

[0062] S2: The inertial measurement unit collects the motion state data of the wire rope in real time, including the acceleration and angular velocity of the wire rope; based on the motion state data of the wire rope collected at the previous moment, the Kalman filter algorithm is used to predict the current motion state of the wire rope.

[0063] An inertial measurement unit (IMU) consists of an accelerometer and a gyroscope, which measures the acceleration and angular velocity of the wire rope in three axes in real time. Kalman filtering is an algorithm that uses the state equations of a linear system to optimally estimate the system state using system input and output observation data. Its state update equation is:

[0064] in, is the state prediction value based on time step k-1; A is the state transition matrix; B is the optimal estimate at time k-1; B is the control matrix; u k It controls the input.

[0065] The processed data is used to calculate the position and velocity of the wire rope and predict its trajectory.

[0066] S3: Based on the predicted current motion state of the wire rope, a fuzzy kernel function is constructed, and an inverse filtering algorithm is used to sharpen the real-time acquired wire rope image.

[0067] Step S3 specifically includes:

[0068] S31: Based on the predicted velocity at the current moment, construct a fuzzy kernel function according to the velocity components of the wire rope in the set X and set Y directions;

[0069]

[0070] Where K(x,y) is the blur level of the image at point (x,y), σ is a set coefficient related to velocity, and v x and v y These are the velocity components of the wire rope in the set X and set Y directions at the current moment.

[0071] The larger the σ value, the larger the range of the blur kernel function, and the blurrier the image.

[0072] S32: After normalizing the fuzzy kernel function K(x,y), perform a Fourier transform to obtain its frequency domain representation H(u,v), and convert the real-time acquired wire rope image into the frequency domain form F(u,v).

[0073] The method for converting real-time acquired wire rope images into frequency domain form F(u,v) can refer to existing technologies. The fuzzy kernel function K(x,y) is normalized to satisfy ∑ x ∑ y K(x,y)=1.

[0074] S33: In the frequency domain, based on the frequency domain form F(u,v) of the wire rope image and the frequency domain form H(u,v) of the fuzzy kernel function, the frequency domain representation F(u,v) of the wire rope image is restored to the frequency domain representation G(u,v) of the clear image by using an inverse filtering algorithm or a Wiener filtering algorithm.

[0075] Specifically, in the frequency domain, the inverse filtering algorithm is used to restore the frequency domain representation F(u,v) of the wire rope image to the frequency domain representation G(u,v) of the clear image, as follows:

[0076]

[0077] Wherein, ∈ represents the set minimum value. Setting a minimum value can avoid division by zero errors, and the value of the minimum value is optionally 1e-8.

[0078] In the frequency domain, the Wiener filtering algorithm is used to restore the frequency domain representation F(u,v) of the wire rope image to the frequency domain representation G(u,v) of the clear image, specifically:

[0079]

[0080] Among them, H * (u,v) is the conjugate complex number of H(u,v), S n (u,v) is the noise power spectrum, S f (u,v) is the power spectrum of the acquired wire rope image.

[0081] By using the Wiener filtering algorithm, the frequency domain representation F(u,v) of the wire rope image can be restored to the frequency domain representation G(u,v) of the clear image, which can reduce the impact of noise.

[0082] The acquisition of the noise power spectrum and the original image power spectrum can refer to existing technologies. In this application, the acquisition of the noise power spectrum typically involves: acquiring multiple image samples containing noise, preprocessing these samples (e.g., removing the mean, normalizing, etc.), and then performing a Fourier transform to convert the image from the spatial domain to the frequency domain. In the frequency domain, the image power spectrum is calculated, and the power spectra of multiple samples are averaged to reduce the influence of random errors, thus obtaining the noise power spectrum. The acquisition of the original image power spectrum involves: for the original image (i.e., an image unaffected by noise or blur), a Fourier transform is directly performed to convert the image from the spatial domain to the frequency domain. In the frequency domain, the image power spectrum is calculated to obtain the original image power spectrum.

[0083] S34: The frequency domain representation G(u,v) of the clear image is transformed back to the spatial domain through inverse Fourier transform to achieve the sharpening process of the acquired wire rope image.

[0084] The Inverse Fourier Transform (IFT) can be referenced from existing technologies.

[0085] S4: Use the target detection model to identify defects in the sharpened image.

[0086] Specifically, such as Figure 3 As shown, a target detection model is used to identify defects in the sharpened image. The target detection model is an improved YOLOv8 model. The improved YOLOv8 model uses MobileNetV3 as the backbone network and embeds the CBAM attention mechanism in the backbone network. The detection head of the improved YOLOv8 model includes an angle detection head, which is used to predict the rotation angle of the wire rope.

[0087] The improved YOLOv8 model's input layer can receive preprocessed images of steel wire ropes (640×640×3 pixels). Preprocessing includes dynamic blur compensation and image enhancement after multispectral fusion.

[0088] The improved YOLOv8 model uses MobileNetV3 as its backbone, which consists of multiple inverted residual blocks. Each block contains the following components: 1×1 expanded convolutions (expanding the number of channels by a ratio controlled by the expansion ratio), and depthwise separable convolutions (3×3 convolutions to extract spatial features and reduce computation). A Squeeze-and-Excitation (SE) module enhances important channel features. A CBAM module is embedded after each inverted residual block, sequentially performing channel and spatial attention calculations. Replacing the YOLOv8 backbone with MobileNetV3 reduces the number of parameters by 42% and improves inference speed by 3 times. CBAM performs attention calculations on the feature maps in both channel and spatial dimensions. The ChannelAttention class obtains global information in the channel dimension through global average pooling and global max pooling, generating channel attention weights through two convolutional layers. The SpatialAttention class calculates the average and maximum values ​​of the feature maps in the channel dimension, concatenates them, and generates spatial attention weights through a 7x7 convolution. The CBAM class multiplies the feature map by channel and spatial attention weights sequentially, enhancing the feature response capability to different types of defect regions. It also processes feature maps of different scales, improving the detection capability for both small and large defects.

[0089] CBAM performs attention calculations on feature maps in both channel and spatial dimensions. The ChannelAttention class obtains global information in the channel dimension through global average pooling and global max pooling, and generates channel attention weights through two convolutional layers. Assume the input feature map is F∈R. C×H×W The result after global average pooling The calculation formula is:

[0090]

[0091] The result of global max pooling The calculation formula is:

[0092] Then, after passing through two convolutional layers, Conv1 and Conv2, we obtain M. c (F):

[0093]

[0094] Where σ is the Sigmoid function.

[0095] The SpatialAttention class averages and maximizes the feature maps along the channel dimension, concatenates them, and then performs a 7×7 convolution to generate spatial attention weights. The result after averaging along the channel dimension... The calculation formula is: The result after taking the maximum value in the channel dimension. The calculation formula is: Will and After splicing, M is obtained by 7×7 convolution. s (F): The CBAM class multiplies the feature map by channel and spatial attention weights sequentially, enhancing its ability to respond to features of different types of defect regions. It also processes feature maps of different scales, improving the detection capability for both small and large defects. The final output feature map F′ is:

[0096] F′=F×M c (F)×M s (F)

[0097] The improved YOLOv8 model's neck network can utilize an improved PANet, which enhances the detection of both small and large defects through multi-scale feature fusion. Structure: Feature Pyramid (FPN): Delivers high-level semantic features from top to bottom. Path Aggregation Network (PAN): Delivers low-level detailed features from bottom to top. CBAM Embedding: A CBAM module is added after the feature fusion layer to further optimize the feature response.

[0098] The improved YOLOv8 model's head includes an angle detection head, which is an existing branch of the YOLOv8 model. Bounding box prediction outputs the center coordinates and class probability. The new branch is the angle prediction head, with a network layer consisting of a 1×1 I convolutional layer and a fully connected layer, outputting the rotation angle θ.

[0099] To ensure that the network can accurately learn the features related to the target rotation angle, a 1×1 convolutional layer can be used to adjust the number of channels in order to better integrate feature information, and then a multilayer perceptron (MLP) structure is used to output the final rotation angle prediction value.

[0100] The loss function L during training of the improved YOLOv8 model is:

[0101] L = L ce +L MSE +L rotation

[0102] Among them, L ce For the cross-entropy loss of defect classification, L MSE For the mean squared error loss of position regression, L rotation The mean square error loss for angle prediction;

[0103]

[0104] Where N is the number of samples, C is the number of categories, and y i,c p represents the true label (0 or 1) of the i-th sample in class c. i,c y represents the probability of the i-th sample in class c predicted by the model. i Let i be the true value of the i-th sample. The value of the i-th sample predicted by the model. Let be the rotation angle predicted by the model for the i-th sample. Let be the actual rotation angle of the i-th sample.

[0105] The improved YOLOv8 model introduces an adversarial training strategy during training. Specifically, during training, image generation techniques (such as generators in GANs) are used to add simulated noise such as haze and raindrops to the original wire rope image. The image with noise is then fed into the improved YOLOv8 model along with the original image as training data.

[0106] Adversarial training can be viewed as a game between a generator G and a discriminator D. The generator's goal is to generate realistic noisy images that cause the discriminator to misjudge. Its loss function is L. G for:

[0107]

[0108] The goal of the discriminator is to accurately distinguish between real images and generated noisy images, and its loss function L... D for:

[0109]

[0110] Where, p data (x) is the distribution of the real image, p z (z) represents the noise distribution.

[0111] Step S4 utilizes the trained, improved YOLOv8 model for defect identification, specifically including:

[0112] S41: Extract features from the sharpened image, including the outline, shape, texture features, and local orientation information of the spiral texture of the wire rope.

[0113] The outline and shape of the wire rope are extracted using an edge detection algorithm, texture features are extracted using a local binary mode, and the local orientation information of the spiral texture is obtained by calculating the gradient direction of pixels in the image.

[0114] S42: Input the extracted features into the improved YOLOv8 model to predict the target bounding box and the wire rope rotation angle.

[0115] S43: Based on the predicted target bounding box and rotation angle, the YOLOv8 model is improved to rotate the target bounding box in the post-processing stage to obtain the rotated box.

[0116] Specifically, the detection box is rotated based on the predicted rotation angle. This can be achieved using a two-dimensional rotation matrix, where the vertex coordinates (x′, y′) of the rotated box are:

[0117]

[0118] Where (x,y) are the vertex coordinates of the predicted target bounding box, x c and y c θ represents the center coordinates of the predicted target bounding box, and θ is the predicted rotation angle of the wire rope.

[0119] The four vertices of the detection frame are rotated according to the formula described above to obtain a rotated detection frame that conforms to the actual shape. In practical applications, whether for online monitoring of wire ropes or offline data analysis, the results based on the rotating frame detection can more accurately locate and describe helical texture defects, providing more reliable data support for subsequent maintenance decisions.

[0120] S44: After adjusting the non-maximum suppression of the rotating frame, the improved YOLOv8 model outputs the defect identification results of the wire rope and realizes the defect location based on the position of the rotating frame.

[0121] Non-maximum suppression (NMS) can be used as a reference for existing techniques. NMS is a commonly used post-processing technique in object detection, which can help the model select the most accurate result from multiple detection results.

[0122] Example 2

[0123] A machine vision-based defect identification system for steel wire ropes of port cranes is provided, comprising:

[0124] The acquisition module is used to acquire images of the wire rope in real time.

[0125] The state estimation module is used to collect the motion state data of the wire rope in real time through the inertial measurement unit, including the acceleration and angular velocity of the wire rope; based on the motion state data of the wire rope collected at the previous moment, the Kalman filter algorithm is used to predict the current motion state of the wire rope.

[0126] The sharpening module is used to construct a fuzzy kernel function based on the predicted current motion state of the wire rope, and to use an inverse filtering algorithm to sharpen the currently acquired wire rope image.

[0127] The defect identification module is used to identify defects in the sharpened image using a target detection model.

[0128] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The systems disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments; relevant parts can be found in the method section.

[0130] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0131] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for identifying defects in wire ropes of port cranes based on machine vision, characterized in that, include: Real-time acquisition of wire rope images; The motion data of the wire rope, including its acceleration and angular velocity, are collected in real time by an inertial measurement unit. Based on the motion state data of the wire rope collected at the previous moment, the Kalman filter algorithm is used to predict the current motion state of the wire rope. Based on the predicted current motion state of the wire rope, a fuzzy kernel function is constructed, and the real-time acquired wire rope image is sharpened using an inverse filtering algorithm or a Wiener filtering algorithm; the fuzzy kernel function is: Where K(x,y) is the blur level of the image at point (x,y), σ is a set coefficient related to velocity, and v x and v y These are the velocity components of the wire rope in the set X and set Y directions at the current moment; Defect identification is performed on the sharpened image using a target detection model; the target detection model is an improved YOLOv8 model; the improved YOLOv8 model uses MobileNetV3 as the backbone network and embeds the CBAM attention mechanism in the backbone network; the detection head of the improved YOLOv8 model includes an angle detection head for predicting the rotation angle of the wire rope.

2. The method for identifying defects in port crane wire ropes based on machine vision according to claim 1, characterized in that, The fuzzy kernel function is constructed based on the predicted current velocity, according to the velocity components of the wire rope in the set X and set Y directions; The process of using inverse filtering or Wiener filtering algorithms to sharpen the currently acquired wire rope images specifically includes: After normalizing the fuzzy kernel function K(x,y), a Fourier transform is performed to obtain its frequency domain representation H(u,v), and the real-time acquired wire rope image is converted into the frequency domain form F(u,v). In the frequency domain, based on the frequency domain form F(u,v) of the wire rope image and the frequency domain form H(u,v) of the blur kernel function, the frequency domain representation F(u,v) of the wire rope image is restored to the frequency domain representation G(u,v) of the clear image by using an inverse filtering algorithm or a Wiener filtering algorithm. The frequency domain representation G(u,v) of the clear image is transformed back to the spatial domain by inverse Fourier transform, thereby achieving the sharpening process of the acquired wire rope image.

3. The method for identifying defects in port crane wire ropes based on machine vision according to claim 2, characterized in that, In the frequency domain, the inverse filtering algorithm is used to restore the frequency domain representation F(u,v) of the wire rope image to the frequency domain representation G(u,v) of the clear image, specifically: Where ∈ represents the set minimum value.

4. The method for identifying defects in port crane wire ropes based on machine vision according to claim 2, characterized in that, In the frequency domain, the Wiener filtering algorithm is used to restore the frequency domain representation F(u,v) of the wire rope image to the frequency domain representation G(u,v) of the clear image, specifically: Among them, H * (u,v) is the conjugate complex number of H(u,v), S n (u,v) is the noise power spectrum, S f (u,v) is the power spectrum of the acquired wire rope image.

5. The method for identifying defects in port crane wire ropes based on machine vision according to claim 1, characterized in that, Defect identification is performed using the improved YOLOv8 model after training, specifically including: Feature extraction is performed on the sharpened image, including the outline, shape, texture features, and local orientation information of the spiral texture of the wire rope; The extracted features are input into the improved YOLOv8 model to predict the target bounding box and the rotation angle of the wire rope. Based on the predicted target bounding box and rotation angle, the YOLOv8 model is improved to rotate the target bounding box in the post-processing stage to obtain the rotated box. After adjusting the nonmaximum suppression of the rotating frame, the improved YOLOv8 model outputs the defect identification results of the wire rope and realizes the defect location based on the position of the rotating frame.

6. The method for identifying defects in port crane wire ropes based on machine vision according to claim 5, characterized in that, Based on the predicted target bounding box and rotation angle, the improved YOLOv8 model rotates the target bounding box during the post-processing stage to obtain the vertex coordinates (x′, y′) of the rotated bounding box: Where (x,y) are the vertex coordinates of the predicted target bounding box, x c and y c θ represents the center coordinates of the predicted target bounding box, and θ is the predicted rotation angle of the wire rope.

7. The method for identifying defects in port crane wire ropes based on machine vision according to claim 1, characterized in that, The loss function L during training of the improved YOLOv8 model is: L=L ce +L MSE +L rotation Among them, L ce For the cross-entropy loss of defect classification, L MSE L is the mean squared error loss for position regression. rotation The mean square error loss for angle prediction; Where N is the number of samples, C is the number of categories, and y i,c p represents the true label (0 or 1) of the i-th sample in class c. i,c y represents the probability of the i-th sample in class c predicted by the model. i Let i be the true value of the i-th sample. The value predicted by the model for the i-th sample. Let be the rotation angle predicted by the model for the i-th sample. Let be the actual rotation angle of the i-th sample.

8. The method for identifying defects in port crane wire ropes based on machine vision according to claim 1, characterized in that, The improved YOLOv8 model incorporates an adversarial training strategy during training. Specifically, during training, image generation technology is used to add simulated noise such as haze and raindrops to the original wire rope image, and the image with noise is input into the improved YOLOv8 model along with the original image as training data.

9. A machine vision-based defect identification system for steel wire ropes of port cranes, characterized in that, include: The acquisition module is used to acquire images of the wire rope in real time. The state estimation module is used to collect the motion state data of the wire rope in real time through the inertial measurement unit, including the acceleration and angular velocity of the wire rope; Based on the motion state data of the wire rope collected at the previous moment, the Kalman filter algorithm is used to predict the current motion state of the wire rope. The blurring module is used to construct a blur kernel function based on the predicted current motion state of the wire rope, and to perform blurring processing on the currently acquired wire rope image using an inverse filtering algorithm or a Wiener filtering algorithm; the blur kernel function is: Where K(x,y) is the blur level of the image at point (x,y), σ is a set coefficient related to velocity, and v x and v y These are the velocity components of the wire rope in the set X and set Y directions at the current moment; The defect identification module is used to identify defects in the sharpened image using a target detection model. The target detection model is an improved YOLOv8 model. The improved YOLOv8 model uses MobileNetV3 as the backbone network and embeds the CBAM attention mechanism in the backbone network. The detection head of the improved YOLOv8 model includes an angle detection head for predicting the rotation angle of the wire rope.

Images