Deep learning-based traction type elevator steel wire rope defect detection method

Abstract

The invention provides a traction type elevator steel wire rope defect detection method based on deep learning, and belongs to the technical field of elevator steel wire rope fault detection. Comprising the following steps: S1, performing multidirectional acquisition by adopting a plurality of image acquisition devices to obtain a real defect image of the elevator steel wire rope; s2, an improved UCG-GAN model is constructed; s3, generating a first steel wire rope defect image by using the real defect image and a generator G of the improved UCG-GAN model; parameters of a generator G and a discriminator D are updated through back propagation of the generated first steel wire rope defect image, and the generator G and the discriminator D are trained through iteration; establishing a steel wire rope defect sample library by using the trained model; s4, establishing and initializing an improved OfficientNet network model, and carrying out training by adopting a steel wire rope defect sample library; and S5, performing defect identification on the real defect images of the elevator steel wire rope acquired by the plurality of image acquisition devices in real time by using the trained improved OfficientNet network model, and outputting a defect identification result. The defect image is generated by using the improved UCG-GAN model, the problem of insufficient diversity of the defect image is improved, and the recognition effect on different defects is improved in combination with the improved OfficientNet network model.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator wire rope fault detection, and in particular to a traction elevator wire rope defect detection method based on deep learning. Background Art

[0002] As one of the core components of the modern urban transportation system, the safety of traction elevators is directly related to the safety of life and property of the general public. As an important load-bearing element for elevator operation, the quality and condition of the wire rope are crucial to the safety and operational stability of the elevator. With the continuous increase in urban population density and the number of high-rise buildings, the frequency of elevator use is also increasing, which puts higher requirements on the safety and reliability of the elevator system. However, under long-term operation and high-intensity use, the wire rope is easily affected by problems such as wear, corrosion, and breakage, which may cause accidents in the elevator, resulting in serious casualties and property losses. Therefore, regular and accurate defect detection of wire ropes has become one of the technical problems that the elevator industry urgently needs to solve.

[0003] According to different detection principles, elevator wire rope defect detection can be roughly divided into visual inspection and electromagnetic inspection. The visual inspection method requires inspectors to work in a dim environment in the elevator shaft, which has low work efficiency and is prone to fatigue. The field of vision is poor, and it is very dependent on the experience of the personnel, and the inspection time is also long. The electromagnetic inspection method is to ensure that the elevator is stopped and safety protection is done well, and prepare electromagnetic detection equipment, sensors and related tools; then, install the electromagnetic sensor on the wire rope to ensure that the sensor is in good contact with the wire rope; then, start the electromagnetic detection equipment and slowly move the sensor along the wire rope to scan the entire length of the wire rope; during the scanning process, the equipment detects changes in electromagnetic signals to identify defects inside the wire rope, such as broken wires, wear and corrosion, etc. This method requires a lot of equipment support, the detection results are easily affected by the environment, and the accuracy is limited.

[0004] Since the elevator wire rope is exposed outside, it can be easily photographed by a camera, and visual inspection methods can be used for identification. At the same time, if visual inspection is performed during the normal operation of the elevator, it can replace manual identification, greatly reducing inspection time and labor costs. Due to the low frequency of defects in elevator wire ropes, fewer images of wire rope defects are obtained, resulting in insufficient diversity of training samples. Therefore, this application is based on artificial intelligence and big data processing methods, based on machine vision, and integrated with computer science and image processing technology to propose a traction elevator wire rope defect detection method based on deep learning, which is used to generate wire rope defect images for training, improve the image recognition accuracy of wire rope defects, and improve the efficiency of automated detection, which is very necessary. Summary of the invention

[0005] In view of this, the present invention proposes a traction elevator wire rope defect detection method based on deep learning, which combines an improved UCG-GAN network to generate wire rope defect images, improves the defect of insufficient existing defect training samples, and further identifies and detects wire rope defects through an improved EfficientNet network model.

[0006] The present invention provides a method for detecting defects in a traction elevator wire rope based on deep learning, comprising the following steps:

[0007] S1: Use several image acquisition devices to perform multi-directional acquisition to obtain the real defect image of the elevator wire rope;

[0008] S2: construct an improved UCG-GAN model, wherein the improved UCG-GAN model includes an independent generator G and a discriminator D;

[0009] S3: Generate a first wire rope defect image using a real defect image and the generator G of the improved UCG-GAN model; Use the generated first wire rope defect image to back-propagate and update the parameters of the generator G and the discriminator D, and train the generator G and the discriminator D through iteration; Generate a second wire rope defect image using the trained generator G and the discriminator D, and establish a wire rope defect sample library based on the second wire rope defect image;

[0010] S4: Establish and initialize an improved EfficientNet network model, use the wire rope defect sample library for training, update the parameters of the improved EfficientNet network model, and obtain the trained improved EfficientNet network model;

[0011] S5: Using the trained improved EfficientNet network model, defect recognition is performed on real defect images of elevator wire ropes collected in real time by several image acquisition devices, and the defect recognition results are output.

[0012] On the basis of the above technical solution, preferably, the content of step S1 is: configure more than two image acquisition devices, set the more than two image acquisition devices on the end face of the elevator room floor close to the car side, and evenly distribute them circumferentially; the more than two image acquisition devices are respectively aimed at the surface of different positions of the same wire rope to obtain the real defect image of the elevator wire rope.

[0013] Preferably, the generator G of the improved UCG-GAN model includes five encoder blocks, one intermediate block and five decoder blocks arranged in sequence, and the five encoder blocks are numbered 4, 3, 2, 1, and 0 in sequence; the five decoders are numbered 1, 2, 3, 4, and 5 in sequence; four jump connections are arranged in sequence between the encoder block and the decoder block, and r∈{1, 2, 3, 4} is set, one end of each jump connection is connected to the output end of the encoder block numbered r, and the other end of each jump connection is connected to the output end of the decoder block numbered r; a CA attention mechanism module is inserted into each jump connection, and the CA attention mechanism module takes the encoder numbered r and the decoder numbered r as input, and the output of the CA attention mechanism module is connected in series with the output of the decoder numbered r as the input of the decoder numbered r+1; the input of the encoder block at the head end is the real defect image x k , the last decoder block outputs the first wire rope defect image

[0014] Further preferably, each encoder block and decoder block includes three convolutional layers, using leaky ReLU and batch normalization, a downsampling layer is arranged at the end of each encoder block, and the output of the downsampling layer serves as the output of the encoder port; an upsampling layer is arranged at the end of each decoder block, and the output of the upsampling layer can serve as the output of the decoder block.

[0015] Further preferably, the discriminator D of the improved UCG-GAN model includes a convolutional layer, five residual blocks and a fully connected layer arranged in sequence; after processing by the convolutional layer and each residual block, the fully connected layer outputs the discriminator score.

[0016] More preferably, the step S3 uses the generated first wire rope defect image to back propagate and update the parameters of the generator G and the discriminator D, and iteratively trains the generator G and the discriminator D, and sets k real defect images x k The average value of the discriminator output score is D(x), k first wire rope defect images The average of the discriminator output scores is Using hinge adversarial loss, the iterative training goal of the discriminator D is to minimize the discriminator loss function λ D , the iterative training goal of the generator G is to minimize the generator loss function λ GD : in is the real defect image x k The expected value of The first wire rope defect image The expected value of max(·) is the ReLU activation function, which ensures that the loss function result is non-negative. By defining the loss function λD and λ GD ,The discriminator D and the generator G continuously compete with each other during the training process, and finally reach a Nash equilibrium so that the generator generates the second wire rope defect image that is closest to the real defect image.

[0017] Preferably, the content of step S4 is: the constructed improved EfficientNet network model includes a first convolution module, seven graph convolution layer modules ATConv, a second convolution module, an average global pooling and a fully connected layer arranged in sequence; each graph convolution layer module ATConv includes a first ordinary convolution layer, an asymmetric convolution module, a Triplet attention mechanism, a second ordinary convolution layer and a Dropout layer arranged in sequence, and a jump connection is also arranged between the input end of the first ordinary convolution layer and the output end of the Dropout layer; the loss function of the graph convolution layer module ATConv is a FILOSS loss function obtained by fusing the FocalLoss loss function with the IOULOSS loss function; the improved EfficientNet network model obtains the second wire rope defect image in the sample library as input, identifies and marks the defective part of the wire rope for output.

[0018] Further preferably, the asymmetric convolution module adds a horizontal asymmetric convolution kernel and a vertical asymmetric convolution kernel on the basis of the standard square convolution, and the input image is convolved with three different convolution kernels to extract different branch features: Among them, M ：，：，K ∈R U×V×C Represents the feature map of the Kth channel with input size U×V, K=1, 2, ..., C, U≠V; represents the jth convolution kernel of the Kth channel with an input size of H×W, H≠W; O ：，：，j is the output feature map corresponding to the jth convolution kernel, * represents the dot multiplication operation;

[0019] The additivity of convolution is used to fuse different branches to obtain the fused feature output. The dimension of the fused feature is consistent with the dimension of the input feature, satisfying: I is the input feature map matrix; K 1 and K 2 Represent different convolution kernels respectively; ★ represents convolution operation; Indicates the addition of the corresponding positions of the convolution kernel.

[0020] Further preferably, the Triplet attention mechanism includes the following contents: the first branch is a channel attention calculation branch, the input feature passes through Z-Pool, then passes through 7×7 convolution, and generates spatial attention weights through Sigmoid activation function; the second branch is a channel C and space W dimension interactive capture branch, the input feature is first transformed into H×C×W dimensional feature through permutation function, then Z-Pool is performed on H dimension, then processed through 7×7 convolution and Sigmoid activation function, and then converted into C×H×W dimensional feature through permutation function; the third branch is a channel C and space H dimension interactive capture branch, the input feature is first transformed into W×H×C dimensional feature through permutation function, then Z-Pool is performed on W dimension, then processed through 7×7 convolution and Sigmoid activation function, and then converted into C×H×W dimensional feature through permutation function; the output result of the Triplet attention mechanism is obtained by adding and averaging the output features of the three branches.

[0021] Further preferably, the expression of FILOSS loss function is: FILOSS = α × FocalLoss + (1-α) × IOULOSS α is the weight; FocalLoss loss function is defined as: FocalLoss = -α t (1-p t ) γ log(p t ), where p t is the probability value predicted by the model; α t is the adjustment factor; γ is a hyperparameter; the IOULOSS loss function is defined as follows: Among them, A is the intersection of the predicted box and the true box, and B is the union of the predicted box and the true box.

[0022] The present invention provides a traction elevator wire rope defect detection method based on deep learning, which has the following beneficial effects compared with the prior art:

[0023] (1) In order to improve the problem of insufficient training samples that affects the training of the detection model, the present invention proposes an improved UCG-GAN network and uses it to generate wire rope defect samples. Specifically, four jump connections are introduced into the generator, and a CA attention mechanism module is inserted into the jump connection. In order to stabilize the training process in the discriminator, a hinge adversarial loss is introduced and included in the classifier, which improves the quality and authenticity of the produced wire rope defect images and provides a training set basis for the training of subsequent models.

[0024] (2) In the process of wire rope defect detection, the present invention proposes an improved EfficientNet network model. Asymmetric convolution modules, Triplet attention mechanism and FILOSS loss function are introduced into each graph convolution layer module ATConv of the EfficientNet network model, which can simultaneously consider the losses of both positioning and classification, improve the performance of the target detection model for optimized training of different samples, and enable the network model to take into account defects of larger and smaller sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is a flowchart of a method for detecting defects in traction elevator wire ropes based on deep learning according to the present invention;

[0027] Figure 2 It is a schematic diagram of the layout of several image acquisition devices of a traction elevator wire rope defect detection method based on deep learning of the present invention;

[0028] Figure 3 The structure of a generative adversarial network (GAN) of a traction elevator wire rope defect detection method based on deep learning in the present invention;

[0029] Figure 4 The structure of the generator G of the improved UCG-GAN model of the traction elevator wire rope defect detection method based on deep learning of the present invention;

[0030] Figure 5 The structure of the CA attention mechanism module of the traction elevator wire rope defect detection method based on deep learning of the present invention;

[0031] Figure 6 The structure of the discriminator D of the CG-GAN model of a traction elevator wire rope defect detection method based on deep learning of the present invention;

[0032] Figure 7 It is a structural diagram of an improved EfficientNet network model of a traction elevator wire rope defect detection method based on deep learning in the present invention;

[0033] Figure 8It is a structural schematic diagram of a graph convolutional layer module ATConv of a traction elevator wire rope defect detection method based on deep learning in the present invention;

[0034] Fig. 9 A schematic diagram of an asymmetric convolution processing process of a traction elevator wire rope defect detection method based on deep learning according to the present invention;

[0035] Fig.10 It is a structural diagram of an asymmetric convolution module of a traction elevator wire rope defect detection method based on deep learning of the present invention;

[0036] Fig.11 It is a structural schematic diagram of a Triplet attention mechanism of a traction elevator wire rope defect detection method based on deep learning in the present invention;

[0037] Fig.12 This is a schematic diagram of actual detection results of a traction elevator wire rope defect detection method based on deep learning in the present invention.

[0038] Figure numerals: 1. elevator machine room floor; 2. image acquisition device; 3. steel wire rope; 4. elevator car. DETAILED DESCRIPTION

[0039] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In the defect detection and identification of traction elevator wire ropes, a large number of defect images are required for model training. However, due to the relatively small number of wire rope defects, the number of defect images is small and difficult to collect, resulting in insufficient sample diversity of training data. Figure 1 As shown, the present invention provides a method for detecting defects in traction elevator wire ropes based on deep learning, comprising the following steps:

[0041] S1: Use several image acquisition devices to perform multi-directional acquisition to obtain the real defect image of the elevator wire rope.

[0042] Reference Figure 2, the content of step S1 is: configure more than two image acquisition devices 2, set more than two image acquisition devices on the end surface of the elevator machine room floor 1 close to the car 4, and evenly distribute them in the circumferential direction; more than two image acquisition devices 2 are respectively aimed at the surface of the same wire rope 3 at different positions to obtain the real defect image of the elevator wire rope 3. As a preferred embodiment, the image acquisition devices 2 are usually set at equal angles in the circumferential direction of the wire rope 3, generally 2-4 are used, and 3 are recommended in this embodiment. The elevator machine room floor 1 close to the car 4 side can capture the surface image of the wire rope 3 that has contacted the pulley to the maximum extent.

[0043] Image acquisition device 2 uses IOI Victorem 4K SDI series high-speed camera, which has 4K resolution and can clearly capture various defects of wire ropes. It uses Sony Pregius CMOS chip, which is smaller, more sensitive and more cost-effective than the previous Flare series. According to the elevator running speed of 2m / s, the sampling frequency of the camera was finally selected to be set to 100fps.

[0044] S2: Construct an improved UCG-GAN model, wherein the improved UCG-GAN model includes an independent generator G and a discriminator D.

[0045] The present invention proposes an improved UGG-GAN to generate wire rope defect images. Generative adversarial network GAN can generate realistic defect images through adversarial training of generator and discriminator, thereby enriching the training data set. Figure 3 As shown in the figure, the calculation process and structure of GAN are shown. It is composed of a generator G and a discriminator D. The generator G of the GAN network generates a wire rope defect image from random noise Z, while the discriminator D is used to distinguish the generated image, that is, the first wire rope defect image and the real defect image. Through this adversarial mechanism, the generator G continuously improves its output and generates samples that are highly similar to the real wire rope defect image X.

[0046] The generator G uses a combination of U-Net and ResNet, searches for relevant information through the CA attention mechanism module, retains more details, and improves the clarity and authenticity of the generated image. The discriminator D enhances the stability and effect of adversarial training through hinge adversarial loss and GDS Losss-based classifier. These improvements enable UCG-GAN to generate high-quality images with a small number of samples, meeting the needs of practical applications.

[0047] Reference Figure 4The structure of the generator G of the improved UCG-GAN model includes 11 residual modules; specifically, it includes five encoder blocks, one intermediate block and five decoder blocks arranged in sequence, and the five encoder blocks are numbered 4, 3, 2, 1, and 0 in sequence; the five decoders are numbered 1, 2, 3, 4, and 5 in sequence; four jump connections are arranged in sequence between the encoder block and the decoder block, and r∈{1, 2, 3, 4} is set, one end of each jump connection is connected to the output end of the encoder block numbered r, and the other end of each jump connection is connected to the output end of the decoder block numbered r; a CA attention mechanism module is inserted into each jump connection, and the CA attention mechanism module takes the encoder numbered r and the decoder numbered r as input, and the output of the CA attention mechanism module is connected in series with the output of the decoder numbered r as the input of the decoder numbered r+1; the input of the encoder block at the head end is the real defect image x k , the last decoder block outputs the first wire rope defect image Structure reference of CA attention mechanism module Figure 5 .

[0048] The CA attention mechanism enables the model to focus on important features more accurately by fusing global and local spatial information, thereby improving feature representation capabilities. Figure 5 The size of the input tensor is C×H×W, where C represents the number of channels, H represents the height, and W represents the width. GAP refers to Global Average Pooling. The input tensor undergoes a Global Average Pooling (GAP) operation to reduce the spatial dimensions (height and width) of the tensor to 1. 1×1×C represents the size of the tensor after GAP. κ=ψ(C), which refers to the adaptive selection of the kernel size based on the channel dimension. This is a dynamic process, and the network determines the appropriate kernel size or the number of key features to focus on based on the input data. K=5 output channels, which indicates that the network focuses on 5 key features or components. σ represents the activation function sigmoid.

[0049] Further preferably, each encoder block and decoder block includes three convolutional layers, using leaky ReLU and batch normalization, a downsampling layer is arranged at the end of each encoder block, and the output of the downsampling layer serves as the output of the encoder port; an upsampling layer is arranged at the end of each decoder block, and the output of the upsampling layer can serve as the output of the decoder block.

[0050] The discriminator D of the improved UCG-GAN model includes a convolutional layer, five residual blocks and a fully connected layer in sequence; after being processed by the convolutional layer and each residual block, the fully connected layer outputs the discriminator score. Figure 6 , the convolutional layer is Figure 6The light-colored part of the image, the five residual layers are stacked in dark blue, and the fully connected layer is in orange.

[0051] S3: Generate the first wire rope defect image using the real defect image and the generator G of the improved UCG-GAN model; use the generated first wire rope defect image to backpropagate and update the parameters of the generator G and the discriminator D, and train the generator G and the discriminator D through iteration; use the trained generator G and discriminator D to generate the second wire rope defect image, and establish a wire rope defect sample library based on the second wire rope defect image.

[0052] Specifically, the first wire rope defect image generated in step S3 is used to back propagate and update the parameters of the generator G and the discriminator D, and the generator G and the discriminator D are trained iteratively. k The average value of the discriminator output score is D(x), k first wire rope defect images The average of the discriminator output scores is Using hinge adversarial loss, the iterative training goal of the discriminator D is to minimize the discriminator loss function λ D , the iterative training goal of the generator G is to minimize the generator loss function λ GD : in is the real defect image x k The expected value of The first wire rope defect image The expected value of max(·) is the ReLU activation function, which ensures that the loss function result is non-negative. By defining the loss function λ D and λ GD ,The discriminator D and the generator G continuously compete with each other during the training process, and finally reach a Nash equilibrium so that the generator generates the second wire rope defect image that is closest to the real defect image.

[0053] The present invention further adds a classifier to the discriminator D, and uses GDS Loss to classify the real wire rope defect image and the generated wire rope defect image. GDS loss is to add a square to the original Generalized Dice Loss, which avoids the loss function being a negative number, ensures that the category of the generated image is consistent with the conditional image, and improves the quality and authenticity of the generated wire rope defect image. The classification loss formula is: Where P X and T X are respectively the predicted area of ​​category X deteriorating the real area, w X is the weight of category X.

[0054] S4: Establish and initialize an improved EfficientNet network model, use the wire rope defect sample library for training, update the parameters of the improved EfficientNet network model, and obtain the trained improved EfficientNet network model.

[0055] like Figure 7 and Figure 8 As shown, the content of step S4 is: the constructed improved EfficientNet network model includes a first convolution module, seven graph convolution layer modules ATConv, a second convolution module, an average global pooling and a fully connected layer arranged in sequence; each graph convolution layer module ATConv includes a first ordinary convolution layer, an asymmetric convolution module, a Triplet attention mechanism, a second ordinary convolution layer and a Dropout layer arranged in sequence, and a jump connection is also arranged between the input end of the first ordinary convolution layer and the output end of the Dropout layer; the loss function of the graph convolution layer module ATConv is a FILOSS loss function obtained by fusing the FocalLoss loss function with the IOULOSS loss function; the improved EfficientNet network model obtains the second wire rope defect image in the sample library as input, identifies and marks the defective part of the wire rope for output.

[0056] Figure 7 In the figure, the 1 or 6 after ATConv represents the multiplication factor n, that is, the first 1x1 convolution layer in ATConv will expand the number of channels of the input feature matrix to n times, where k3x3 or k5x5 represents the convolution kernel size used in the asymmetric convolution in ATConv.

[0057] The graph convolution layer module ATConv is improved on the basis of MBConv in the EfficientDet network model. It replaces the Depwise convolution in the original MBConv module with an asymmetric convolution, and the original SE attention mechanism with a Triplet attention mechanism. Finally, the original FocalLoss loss function is fused with the IOULOSS loss function to obtain a new FILOSS loss function, and finally forms the ATConv module:

[0058] After being replaced with asymmetric convolution, the ATConv module can more effectively capture feature information in different directions and improve the model's ability to represent complex image structures;

[0059] After replacing it with the Triplet attention mechanism, the model can pay more attention to important areas in the feature map, thereby improving the model's ability to recognize targets;

[0060] After integrating the loss functions FocalLoss and IOULOSS, the new FILOSS can enable the model to perform better in processing unbalanced samples and target positioning accuracy, and finally make the model have better robustness and accuracy in detection tasks.

[0061] Reference Fig. 9 , Fig. 9 (a) and (b) are respectively the schematic diagrams of the feature map before and after horizontal flipping, and the convolution processing using a horizontal asymmetric convolution kernel. Fig. 9 (c) and (d) are respectively the convolution processing of the feature map before and after horizontal flipping using a standard square convolution kernel. Fig. 9 It can be seen that after the input feature map is horizontally flipped, the features extracted by the standard square convolution have a significant change, while the horizontal asymmetric convolution can still accurately extract features during the sliding convolution process. This shows that asymmetric convolution is more robust in terms of rotational distortion characteristics. The structural diagram of the asymmetric convolution module is shown in Fig.10 shown.

[0062] Fig.10 In the figure, the ACB_N part represents the input of this asymmetric convolution module; Zero Padding represents the zero padding operation, which is used to keep the spatial dimensions of the input and output tensors consistent.

[0063] BN (Batch Normalization): Batch normalization is used to accelerate training and stabilize the training process of neural networks.

[0064] Leaky ReLU is a leaky ReLU activation function that allows some negative values ​​to pass through to prevent neurons from "dying".

[0065] ResUnit (Residual Unit): The residual unit is used to build a residual network. It adds the input directly to the output through a skip connection, which helps alleviate the gradient vanishing problem and improves the effect of training deep networks.

[0066] The asymmetric convolution module adds horizontal asymmetric convolution kernels and vertical asymmetric convolution kernels to the standard square convolution. The input image is convolved with three different convolution kernels to extract different branch features: Among them, M ：，：，K ∈R U×V×C Represents the feature map of the Kth channel with input size U×V, K=1, 2, ..., C, U≠V; represents the jth convolution kernel of the Kth channel with an input size of H×W, H≠W; O ：，：，j is the output feature map corresponding to the jth convolution kernel, * represents the dot multiplication operation;

[0067] The additivity of convolution is used to fuse different branches to obtain the fused feature output. The dimension of the fused feature is consistent with the dimension of the input feature, satisfying: I is the input feature map matrix; K 1 and K 2 Represent different convolution kernels respectively; ★ represents convolution operation; Indicates the addition of the corresponding positions of the convolution kernel.

[0068] The Triplet attention mechanism includes the following contents: the first branch is the channel attention calculation branch. The input features are Z-Pooled, then 7×7 convolutions, and the spatial attention weights are generated by the Sigmoid activation function. The second branch is the channel C and space W dimension interactive capture branch. The input features are first transformed into H×C×W dimensional features through the permutation function, then Z-Pooled in the H dimension, and then processed by 7×7 convolutions and Sigmoid activation functions, and then transformed into C×H×W dimensional features through the permutation function. The third branch is the channel C and space H dimension interactive capture branch. The input features are first transformed into W×H×C dimensional features through the permutation function, then Z-Pooled in the W dimension, then processed by 7×7 convolutions and Sigmoid activation functions, and then transformed into C×H×W dimensional features through the permutation function. The output result of the Triplet attention mechanism is obtained by adding and averaging the output features of the three branches.

[0069] like Fig.11 As shown in the figure, the Triplet attention mechanism consists of three parallel branches. Two of the branches are responsible for capturing the interactions between different dimensions: one focuses on the interaction between the channel dimension (C) and the spatial dimension (H or W), and the other is used to construct spatial attention. The outputs of all three branches are combined by averaging their respective weights and then aggregated. This innovative design with lateral interactions solves the challenge of separating channel attention and spatial attention in traditional computational models, and it captures the interaction between spatial and channel dimensions in the same framework. In essence, it captures information interactions in three dimensions at the same time: (C, H), (C, W), and (H, W), representing the interactions between the channel, height, and width dimensions of the input data, respectively.

[0070] The expression of FILOSS loss function is: FILOSS = α × FocalLoss + (1-α) × IOULOSS, α is the weight; FocalLoss loss function is defined as: FocalLoss = -α t (1-p t ) γ log(pt ) where p t is the probability value predicted by the model; α t is the adjustment factor; γ is a hyperparameter; the IOULOSS loss function is defined as follows: A is the intersection of the predicted box and the true box, and B is the union of the predicted box and the true box. By combining FocalLoss with IOULOSS, we can comprehensively consider the losses of classification and positioning, and improve the performance of the target detection model in processing difficult and easy samples and optimizing training effects.

[0071] S5: Using the trained improved EfficientNet network model, defect recognition is performed on real defect images of elevator wire ropes collected in real time by several image acquisition devices, and the defect recognition results are output.

[0072] In order to solve the problem of insufficient wire rope defect images affecting the training of the detection model, the present invention proposes to improve the UCG-GAN network and use it for wire rope defect sample generation. The specific experimental results are as follows: the U-Net structure is introduced in the generator G, and 4 jump structures are added between the encoder block and the decoder block, and then the CA attention mechanism module is inserted into the jump structure; in the discriminator D, in order to stabilize the training process, the hinge adversarial loss is introduced and included in the classifier, and a new classification loss function GDSloss is proposed to avoid the loss function result being a negative number, thereby improving the quality and authenticity of the generated wire rope defect image. In addition, in the wire rope defect detection stage, the present invention proposes an improved EfficientNet network model, replacing the Depwise convolution in the original MBConv module with an asymmetric convolution, which has an asymmetric structure and has the characteristics of scale-invariant feature transformation. The original SE attention mechanism is replaced by the Triplet attention mechanism, which can capture the interaction between the spatial dimension and the channel dimension at the same time. Finally, the original FocalLoss loss function is fused with the IOULOSS loss function to obtain the new FILOSS loss function, which takes into account both classification and positioning losses. It can improve the performance of the target detection model in processing difficult and easy samples and optimizing training effects, and finally form the ATConv module. The final calculation results are as follows: Fig.12 As shown in the figure, it can be found that the improved EfficientNet model can not only accurately identify larger defects such as wear and breakage of elevator wire ropes, but also accurately identify smaller defects such as breakage defects, and the confidence level can reach above 0.9.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting defects in traction elevator wire ropes based on deep learning. It is characterized in that The steps include: S1: Use several image acquisition devices to perform multi-directional acquisition to obtain the real defect image of the elevator wire rope; S2: construct an improved UCG-GAN model, wherein the improved UCG-GAN model includes an independent generator G and a discriminator D; S3: Generate a first wire rope defect image using a real defect image and the generator G of the improved UCG-GAN model; Use the generated first wire rope defect image to back-propagate and update the parameters of the generator G and the discriminator D, and train the generator G and the discriminator D through iteration; Generate a second wire rope defect image using the trained generator G and the discriminator D, and establish a wire rope defect sample library based on the second wire rope defect image; S4: Establish and initialize an improved EfficientNet network model, use the wire rope defect sample library for training, update the parameters of the improved EfficientNet network model, and obtain the trained improved EfficientNet network model; S5: Using the trained improved EfficientNet network model, defect recognition is performed on real defect images of elevator wire ropes collected in real time by several image acquisition devices, and the defect recognition results are output.

2. According to claim 1, a method for detecting defects in traction elevator wire ropes based on deep learning, It is characterized in that The content of step S1 is: configure more than two image acquisition devices, set the more than two image acquisition devices on the end surface of the elevator room floor close to the car side, and evenly distribute them in the circumferential direction; the more than two image acquisition devices are respectively aimed at the surface of different positions of the same wire rope to obtain the real defect image of the elevator wire rope.

3. According to claim 2, a method for detecting defects in traction elevator wire ropes based on deep learning, It is characterized in that The generator G of the improved UCG-GAN model includes five encoder blocks, one intermediate block and five decoder blocks arranged in sequence, and the five encoder blocks are numbered 4, 3, 2, 1, and 0 in sequence; the five decoders are numbered 1, 2, 3, 4, and 5 in sequence; four jump connections are arranged in sequence between the encoder block and the decoder block, and r∈{1, 2, 3, 4} is set, one end of each jump connection is connected to the output end of the encoder block numbered r, and the other end of each jump connection is connected to the output end of the decoder block numbered r; a CA attention mechanism module is inserted into each jump connection, and the CA attention mechanism module takes the encoder numbered r and the decoder numbered r as input, and the output of the CA attention mechanism module is connected in series with the output of the decoder numbered r as the input of the decoder numbered r+1; the input of the encoder block at the head end is the real defect image x k , the last decoder block outputs the first wire rope defect image 4. According to claim 3, a method for detecting defects in traction elevator wire ropes based on deep learning, It is characterized in that Each encoder block and decoder block includes three convolutional layers, using leaky ReLU and batch normalization. A downsampling layer is set at the end of each encoder block, and the output of the downsampling layer is used as the output of the encoder port; an upsampling layer is set at the end of each decoder block, and the output of the upsampling layer can be used as the output of the decoder block.

5. According to claim 3, a method for detecting defects in traction elevator wire ropes based on deep learning, It is characterized in that The discriminator D of the improved UCG-GAN model includes a convolutional layer, five residual blocks and a fully connected layer arranged in sequence; after being processed by the convolutional layer and each residual block, the fully connected layer outputs a discriminator score.

6. A traction elevator wire rope defect detection method based on deep learning according to claim 5, It is characterized in that The first wire rope defect image generated in step S3 is used to back propagate and update the parameters of the generator G and the discriminator D, and the generator G and the discriminator D are trained iteratively. k The average value of the discriminator output score is D(x), k first wire rope defect images The average of the discriminator output scores is Using hinge adversarial loss, the iterative training goal of the discriminator D is to minimize the discriminator loss function λ D , the iterative training goal of the generator G is to minimize the generator loss function λ GD : in is the real defect image x k The expected value of The first wire rope defect image The expected value of max(·) is the ReLU activation function, which ensures that the loss function result is non-negative. By defining the loss function λ D and λ GD ,The discriminator D and the generator G continuously compete with each other during the training process, and finally reach a Nash equilibrium so that the generator generates the second wire rope defect image that is closest to the real defect image.

7. The method for detecting defects in traction elevator wire ropes based on deep learning according to claim 2, It is characterized in that The content of step S4 is: the constructed improved EfficientNet network model includes a first convolution module, seven graph convolution layer modules ATConv, a second convolution module, an average global pooling and a fully connected layer arranged in sequence; each graph convolution layer module ATConv includes a first ordinary convolution layer, an asymmetric convolution module, a Triplet attention mechanism, a second ordinary convolution layer and a Dropout layer arranged in sequence, and a jump connection is also arranged between the input end of the first ordinary convolution layer and the output end of the Dropout layer; the loss function of the graph convolution layer module ATConv is a FILOSS loss function obtained by fusing the FocalLoss loss function with the IOULOSS loss function; the improved EfficientNet network model obtains the second wire rope defect image in the sample library as input, identifies and marks the defective part of the wire rope for output.

8. The method for detecting defects in traction elevator wire ropes based on deep learning according to claim 7, It is characterized in that The asymmetric convolution module adds horizontal asymmetric convolution kernels and vertical asymmetric convolution kernels on the basis of standard square convolution. The input image is convolved with three different convolution kernels to extract different branch features: where M:,:,K∈R U×V×C Represents the feature map of the Kth channel with input size U×V, K=1, 2, ..., C, U≠V; represents the jth convolution kernel of the Kth channel with input size H×W, H≠W; O： ， : , j is the output feature map corresponding to the jth convolution kernel, * represents the dot multiplication operation; The additivity of convolution is used to fuse different branches to obtain the fused feature output. The dimension of the fused feature is consistent with the dimension of the input feature, satisfying: I is the input feature map matrix; K 1 and K 2 Represent different convolution kernels respectively; * represents convolution operation; Indicates the addition of the corresponding positions of the convolution kernel.

9. A method for detecting defects in traction elevator wire ropes based on deep learning according to claim 8, It is characterized in that The Triplet attention mechanism includes the following contents: the first branch is the channel attention calculation branch, the input feature passes through Z-Pool, then passes through 7×7 convolution, and passes through the Sigmoid activation function to generate spatial attention weights; the second branch is the channel C and space W dimension interaction capture branch, the input feature is first transformed into H×C×W dimensional features through the permutation function, then Z-Pool is performed on the H dimension, then processed by 7×7 convolution and Sigmoid activation function, and then converted into C×H×W dimensional features through the permutation function; the third The branch is a channel C and space H dimension interactive capture branch. The input feature is first transformed into a W×H×C dimensional feature through the permutation function, then Z-Pool is performed on the W dimension, and then processed by a 7×7 convolution and a Sigmoid activation function, and then transformed into a C×H×W dimensional feature through the permutation function. The output result of the Triplet attention mechanism is obtained by adding and averaging the output features of the three branches.

10. The method for detecting defects in traction elevator wire ropes based on deep learning according to claim 7, It is characterized in that The expression of FILOSS loss function is: FILOSS = α × FocalLoss + (1-α) × IOULOSS, α is the weight; The FocalLoss loss function is defined as: FocalLoss = -α t (1-p t )γlog(p t ), where p t is the probability value predicted by the model; α t is the adjustment factor; γ is a hyperparameter; The IOULOSS loss function is defined as follows: Where A is the intersection of the predicted box and the true box, and B is the union of the predicted box and the true box.

Images