Method and system for detecting surface defects of elevator steel belt

Abstract

The invention discloses an elevator steel belt surface defect detection method and system, and the method comprises the steps: enabling a plurality of surface images to form a complex data set, and enabling the complex data set to interact with a YOLOv8n model; performing model optimization on the YOLOv8n model based on the complex data set, and outputting an elevator steel belt surface fault detection model; a to-be-detected image of the elevator steel belt is collected, surface fault information of the elevator steel belt is output according to the to-be-detected image of the elevator steel belt and the elevator steel belt surface fault detection model, the surface fault information of the elevator steel belt comprises abrasion exposure, mouse biting damage or transverse cracks, at the moment, a YOLOv8n model is trained based on the complex data set, and the elevator steel belt surface fault detection model is obtained. According to the method, the YOLOv8n model is selected, so that the YOLOv8n model is optimized, the elevator steel belt surface fault detection model is output, surface fault information of the elevator steel belt is output according to the to-be-detected image of the elevator steel belt and the elevator steel belt surface fault detection model, and intelligent fault detection of the to-be-detected image of the elevator steel belt is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of detecting surface defects of elevator steel strips, and in particular to a method and system for detecting surface defects of elevator steel strips. Background Art

[0002] With the development of science and technology, elevator steel belts are used in elevator scenes. Elevator steel belts, also known as elevator car guide steel belts, are components that connect the elevator car and the drive motor. The main function of the elevator steel belt is to guide the car from the motor to the corresponding floor during elevator operation and to ensure the safety and stability of the elevator. Elevator steel belts need to be used for a long time. In the existing technology, there are some defects on the surface of the elevator steel belts. Obvious defects can be observed by the human eye, and intelligent fault detection based on the image to be detected of the elevator steel belt is impossible. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a method and system for detecting surface defects of elevator steel strips. A YOLOv8n model is trained based on a complex data set to optimize the YOLOv8n model, thereby outputting an elevator steel strip surface fault detection model, so as to output the surface fault information of the elevator steel strip according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, thereby realizing intelligent fault detection of the image to be detected of the elevator steel strip and ensuring the accuracy of the surface fault information of the elevator steel strip.

[0004] In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting surface defects of an elevator steel belt, which is applied to an elevator steel belt; The method for detecting surface defects of an elevator steel belt comprises: Acquire a surface image of an elevator steel belt, where the surface image of the elevator steel belt is collected based on multiple scenes and multiple angles; Form multiple surface images into a complex dataset and interact the complex dataset with the YOLOv8n model; Based on complex data sets, the YOLOv8n model is optimized and the elevator steel belt surface fault detection model is output; The image to be detected of the elevator steel belt is collected, and the surface fault information of the elevator steel belt is output according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

[0005] Optionally, the acquiring of the surface image of the elevator steel belt, wherein the surface image of the elevator steel belt is collected based on multiple scenes and multiple angles, includes: Collect the positioning of the elevator steel belt, and trigger the shooting of the elevator steel belt based on the positioning of the elevator steel belt; Based on the shooting of the elevator steel belt, the surface image of the elevator steel belt is output, wherein the orientation of the elevator steel belt and the scene where the elevator steel belt is located are marked, multiple coordination is performed according to the orientation of the elevator steel belt and the scene where the elevator steel belt is located, and the corresponding surface image is output according to the multiple coordination of the orientation of the elevator steel belt and the scene where the elevator steel belt is located.

[0006] Optionally, forming a complex data set from the plurality of surface images and interacting the complex data set with the YOLOv8n model includes: Collect multiple surface images and form the multiple surface images into a complex data set; GAN based on unsupervised learning performs data enhancement processing on complex data sets and outputs processed data sets; The processed data set is divided into training set, validation set and test set according to the proportion. The training set, validation set and test set are used in the test scenario, training scenario and validation scenario of the model respectively. Collect YOLOv8n model; Freeze the convolutional modules in the YOLOv8n model and replace them with the deformable volume DCNv2 module to optimize the YOLOv8n model for the first time. Interact the processed dataset with the YOLOv8n model.

[0007] Optionally, the forming of a complex data set from the plurality of surface images and interacting the complex data set with the YOLOv8n model further includes: In the deformable convolution DCNv2, by introducing offsets and deformable convolution kernels, the YOLOv8n model learns the offsets and weights of each sampling point, and uses R-CNN to perform knowledge distillation on Faster R-CNN, so that the deformable convolution can learn better sampling points, as well as better offsets and weights of the sampling points. Among them, y(p) represents the pixel value at position p in the output feature map; represents the weight, which is used to perform weighted summation on the interpolation; x represents the pixel value in the input feature map; represents the position of the sampling point in the input feature map; p is the position in the output feature map, is the offset of the sampling point relative to p, is the offset learned from the deformable convolution; Represents the weight of the offset learned by deformable convolution.

[0008] Optionally, the method of optimizing the YOLOv8n model based on the complex data set and outputting the elevator steel belt surface fault detection model includes: The LSKA attention mechanism is introduced to improve the SPPF module in the YOLOv8 model and construct a new SPPF-LSKA module; In the new SPPF-LSKA module, the representation ability of the model is further enhanced through the local attention layer, kernelized attention layer and self-attention fusion; Through the deep dilated convolution (DW-Conv) module, we can focus on local information to locate the local area; The Gaussian kernel function is used to calculate the correlation score between each pair of local regions. At the same time, the output of the kernelized attention is element-wise multiplied with the input, and then a 1×1 convolution layer is used to fuse the local attention and global attention features to output the elevator steel belt surface fault detection model.

[0009] Optionally, the method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model further includes: Local attention is calculated as follows: in, is the feature representation of region i, N is the number of local regions in the image, and K(i,j) is the kernel function used to measure the correlation between region i and region j; The calculation formula of the relevance score K(i,j) is: in, and is the feature representation of local regions i and j, represents the norm of a vector, It is the parameter of the Gaussian kernel function, which is used to control the bandwidth of the kernel function.

[0010] Optionally, the method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model further includes: The attention weight between each pair of local regions is obtained by exponentially and normalizing the correlation score , which is used to represent weighted local features to better capture the nonlinear relationship between different regions in the image. Its calculation formula is: Where N represents the number of local regions in the image, and K(i,j) is the correlation score between local regions i and j calculated by the kernel function.

[0011] Optionally, the method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model further includes: The process of self-attention fusion can be expressed by the following formula: in, Expressing local concern, Expressing global concern, Represents the fusion parameter, which is used to control the weight of the two.

[0012] Optionally, the image to be detected of the elevator steel belt is collected, and the surface fault information of the elevator steel belt is output according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt, the surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks, including: Collect the image of the elevator steel belt to be tested; The surface fault information of the elevator steel belt is output according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

[0013] Optionally, the elevator steel strip surface defect detection system is applied to the above-mentioned elevator steel strip surface defect detection method, and the elevator steel strip surface defect detection system includes: An acquisition module, used to acquire a surface image of an elevator steel belt, wherein the surface image of the elevator steel belt is acquired based on multiple scenes and multiple angles; An interaction module, which is used to form a complex dataset from multiple surface images and interact the complex dataset with the YOLOv8n model; The optimization module is used to optimize the YOLOv8n model based on complex data sets and output the elevator steel belt surface fault detection model; The surface fault module is used to collect the image to be detected of the elevator steel belt, and output the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

[0014] In an embodiment of the present invention, a surface image of an elevator steel strip is obtained by a method in an embodiment of the present invention, and the surface image of the elevator steel strip is collected based on multiple scenes and multiple angles; the multiple surface images are formed into a complex data set, and the complex data set is interacted with the YOLOv8n model; the YOLOv8n model is optimized based on the complex data set, and a surface fault detection model of the elevator steel strip is output; an image to be detected of the elevator steel strip is collected, and the surface fault information of the elevator steel strip is output according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, the surface fault information of the elevator steel strip includes wear and tear, mouse bite damage or transverse cracks. At this time, the YOLOv8n model is trained based on the complex data set so as to optimize the YOLOv8n model, thereby outputting the surface fault detection model of the elevator steel strip, so as to output the surface fault information of the elevator steel strip according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, thereby realizing intelligent fault detection of the image to be detected of the elevator steel strip and ensuring the accuracy of the surface fault information of the elevator steel strip. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 is a schematic flow chart of a method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 2 is a schematic flow chart of S11 in the method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 3 is a schematic flow chart of S12 in the method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 4 is a schematic flow chart of S13 in the method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 5 is a schematic flow chart of S14 in the method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 6 is a schematic diagram of a model in a method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 7 It is a test curve intention in the method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Figure 8is a schematic diagram of fault information in a method for detecting surface defects of an elevator steel strip in an embodiment of the present invention; Fig. 9 Schematic diagram of the structure of a detection system for surface defects of an elevator steel strip in an embodiment of the present invention; Fig.10 The figure is a hardware diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1

[0019] See also Figures 1 to 10 , a method for detecting surface defects of an elevator steel belt, applied to the elevator steel belt; the method for detecting surface defects of the elevator steel belt comprises: Step S11: acquiring a surface image of an elevator steel belt, wherein the surface image of the elevator steel belt is collected based on multiple scenes and multiple angles; Step S12: forming a complex data set from the plurality of surface images, and interacting the complex data set with the YOLOv8n model; Step S13: Optimizing the YOLOv8n model based on the complex data set, and outputting an elevator steel belt surface fault detection model; Step S14: Collect the image to be detected of the elevator steel belt, and output the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

[0020] In an embodiment of the present invention, a surface image of an elevator steel strip is obtained by a method in an embodiment of the present invention, and the surface image of the elevator steel strip is collected based on multiple scenes and multiple angles; the multiple surface images are formed into a complex data set, and the complex data set is interacted with the YOLOv8n model; the YOLOv8n model is optimized based on the complex data set, and a surface fault detection model of the elevator steel strip is output; an image to be detected of the elevator steel strip is collected, and the surface fault information of the elevator steel strip is output according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, the surface fault information of the elevator steel strip includes wear and tear, mouse bite damage or transverse cracks. At this time, the YOLOv8n model is trained based on the complex data set so as to optimize the YOLOv8n model, thereby outputting the surface fault detection model of the elevator steel strip, so as to output the surface fault information of the elevator steel strip according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, thereby realizing intelligent fault detection of the image to be detected of the elevator steel strip and ensuring the accuracy of the surface fault information of the elevator steel strip.

[0021] In step S11, a surface image of an elevator steel belt is acquired, where the surface image of the elevator steel belt is collected based on multiple scenes and multiple angles; In the specific implementation process of the present invention, the specific steps may be: S111: collecting the location of the elevator steel belt, and triggering the shooting of the elevator steel belt based on the location of the elevator steel belt; S112: Outputting a surface image of the elevator steel belt based on the shooting of the elevator steel belt, wherein the orientation of the elevator steel belt and the scene where the elevator steel belt is located are marked, multiple coordination is performed according to the orientation of the elevator steel belt and the scene where the elevator steel belt is located, and outputting a corresponding surface image according to the multiple coordination of the orientation of the elevator steel belt and the scene where the elevator steel belt is located.

[0022] In an embodiment of the present application, the shooting of the elevator steel belt is triggered based on the positioning of the elevator steel belt, so that the surface image of the elevator steel belt is output based on the shooting of the elevator steel belt, thereby controlling the surface image of the elevator steel belt to facilitate intelligent detection of the surface of the elevator steel belt.

[0023] At this time, the orientation of the elevator steel belt and the scene where the elevator steel belt is located are marked, multiple coordination is performed according to the orientation of the elevator steel belt and the scene where the elevator steel belt is located, and the corresponding surface image is output according to the multiple coordination of the orientation of the elevator steel belt and the scene where the elevator steel belt is located.

[0024] Optionally, the obtained surface image is adaptively scaled to 640px×640px×3, and labeled by an open source image annotation tool Labelimg to create a YOLO format dataset and corresponding labels.

[0025] In step S12, the plurality of surface images are formed into a complex data set, and the complex data set is interacted with the YOLOv8n model; In the specific implementation process of the present invention, the specific steps may be: S121: collecting multiple surface images and forming a complex data set from the multiple surface images; S122: Perform data enhancement processing on complex data sets based on unsupervised learning GAN and output the processed data sets; S123: Divide the processed data set into a training set, a validation set, and a test set according to a proportion, and the training set, the validation set, and the test set are respectively applied to the test scenario, the training scenario, and the validation scenario of the model; S124: Collect YOLOv8n model; S125: Freeze the convolution module in the YOLOv8n model and replace the convolution module with the deformable volume DCNv2 module to optimize the YOLOv8n model for the first time; S126: Interact the processed dataset with the YOLOv8n model.

[0026] In an embodiment of the present application, multiple surface images are collected and formed into a complex data set to facilitate processing of the complex data set. At this time, GAN based on unsupervised learning performs data enhancement processing on the complex data set and outputs the processed data set. At the same time, the GAN data enhancement method includes contrast enhancement, color enhancement, sharpening enhancement and image denoising.

[0027] Furthermore, the processed dataset is divided into training set, validation set and test set in proportion, and the training set, validation set and test set are respectively applied to the test scenario, training scenario and validation scenario of the model; the YOLOv8n model is acquired; the convolution module in the YOLOv8n model is frozen, and the convolution module is replaced by the deformable volume DCNv2 module to initially optimize the YOLOv8n model; the processed dataset is interacted with the YOLOv8n model to facilitate in-depth training of the processed dataset and the YOLOv8n model, thereby optimizing the YOLOv8n model.

[0028] Furthermore, the forming of a complex data set from the plurality of surface images and interacting the complex data set with the YOLOv8n model further includes: In the deformable convolution DCNv2, by introducing offsets and deformable convolution kernels, the YOLOv8n model learns the offsets and weights of each sampling point, and uses R-CNN to perform knowledge distillation on Faster R-CNN, so that the deformable convolution can learn better sampling points, as well as better offsets and weights of the sampling points. Among them, y(p) represents the pixel value at position p in the output feature map; represents the weight, which is used to perform weighted summation on the interpolation; x represents the pixel value in the input feature map; represents the position of the sampling point in the input feature map; p is the position in the output feature map, is the offset of the sampling point relative to p, is the offset learned from the deformable convolution; Represents the weight of the offset learned by deformable convolution.

[0029] In step S13, the YOLOv8n model is optimized based on the complex data set, and the elevator steel belt surface fault detection model is output; In the specific implementation process of the present invention, the specific steps may be: S131: Introduce LSKA attention mechanism to improve the SPPF module in the YOLOv8 model and construct a new SPPF-LSKA module; S132: In the new SPPF-LSKA module, the representation ability of the model is further enhanced through the local attention layer, kernelized attention layer and self-attention fusion; S133: Focus on local information to locate local areas through the deep dilated convolution (DW-Conv) module; S134: The correlation score between each pair of local regions is calculated by using the Gaussian kernel function. At the same time, the output of the kernelized attention is element-wise multiplied with the input, and then the local attention and global attention features are fused through a 1×1 convolution layer to output the elevator steel belt surface fault detection model.

[0030] In the embodiment of the present application, the LSKA attention mechanism is introduced to improve the SPPF module in the YOLOv8 model, and a new SPPF-LSKA module is constructed; in the new SPPF-LSKA module, the representation ability of the model is further enhanced through the local attention layer, the kernelized attention layer and the self-attention fusion; the deep dilated convolution (DW-Conv) module is used to realize the attention to local information to locate the local area, thereby ensuring the local area. Optionally, the representation ability of the model is further enhanced through the local attention layer, the kernelized attention layer and the self-attention fusion.

[0031] Furthermore, the method of optimizing the YOLOv8n model based on the complex data set and outputting the elevator steel belt surface fault detection model also includes: Local attention is calculated as follows: in, is the feature representation of region i, N is the number of local regions in the image, and K(i,j) is the kernel function used to measure the correlation between region i and region j; The calculation formula of the relevance score K(i,j) is: in, and is the feature representation of local regions i and j, represents the norm of a vector, It is the parameter of the Gaussian kernel function, which is used to control the bandwidth of the kernel function.

[0032] In addition, the method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model also includes: The attention weight between each pair of local regions is obtained by exponentially and normalizing the correlation score , which is used to represent weighted local features to better capture the nonlinear relationship between different regions in the image. Its calculation formula is: Where N represents the number of local regions in the image, and K(i,j) is the correlation score between local regions i and j calculated by the kernel function.

[0033] At the same time, by performing element-wise multiplication operation on the output and input of kernelized attention, and then fusing the local attention and global attention features through a 1×1 convolutional layer, the process of self-attention fusion can be expressed by the following formula: in, Expressing local concern, Expressing global concern, Represents the fusion parameter, which is used to control the weight of the two.

[0034] In addition, the positioning branch uses DCNV2 and interactive features to generate DCNV2 offset and mask, while the classification branch uses interactive features for dynamic feature selection. The structure of the improved YOLOv8n network model includes Backbone structure, Neck structure and Head structure.

[0035] In addition, in order to solve the feature conflict problem in the classification and positioning of typical failure mode detection of elevator steel belts and further improve the key performance of the model, this paper introduces DSDH into the head network structure of the YOLOv8n algorithm.

[0036] The main idea flow is as follows: Firstly, the input image is processed by two convolutions and group normalization to form the feature map of the elevator steel belt failure mode, and the two features are concatenated.

[0037] Then, the merged feature map is segmented: one part generates masks and offsets as the basis for dynamic filtering in the subsequent DCNv2 module, and the other part passes through two task decomposition

[23] (TD) modules to generate multiple sub-task features, such as Figure 8 As shown in the figure, one subtask feature is processed by DCNv2, while the other subtask feature is multiplied with other feature maps after passing through ReLU and Sigmoid activation functions, thereby fusing information from different modalities and improving the discriminability of decision making.

[0038] The calculation formulas for the subtask features of each classification or positioning task are as follows: Where: Represents the kth element of the subtask interaction feature; Represents cross-layer interaction features; is the kth element of the learned attention layer; w is the attention vector used to capture the dependencies between layers: Where: w is a fully connected layer through two layers and The calculated attention vector; is the sigmoid activation function; It is through The average pooling result is Finally, from each The classification or positioning results are predicted.

[0039] Finally, DCNv2 will adaptively adjust its filter position according to the input information, effectively capture the complex feature changes, and send the processed features to an independent convolutional layer for regression prediction, generating 3 different regression outputs. At the same time, the feature map activated by ReLU and Sigmoid is multiplied with the feature map generated by the TD module to generate 3 different classification results.

[0040] In summary, DSDH fuses interactive features, strengthens the feature learning ability of the model on the basis of ensuring a lightweight model, and better realizes the collaborative work of the positioning and classification tasks of typical failure modes of elevator steel belts.

[0041] S14: collecting the image to be detected of the elevator steel belt, and outputting the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt, wherein the surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse crack; In the specific implementation process of the present invention, the specific steps may be: S141: collecting the image of the elevator steel belt to be inspected; S142: outputting surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt, where the surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks; In an embodiment of the present application, a YOLOv8n model is trained based on a complex data set to optimize the YOLOv8n model, thereby outputting an elevator steel belt surface fault detection model, so as to output the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt, thereby realizing intelligent fault detection of the image to be detected of the elevator steel belt and ensuring the accuracy of the surface fault information of the elevator steel belt.

[0042] In the Backbone structure, the first layer of DCNv2 module performs convolution operation on the input image, the convolution kernel size is 3*3, the number of output channels is 64, and the feature map is reduced to 1 / 2 of the initial image; The second layer DCNv2 module performs convolution operation on the input feature map. The convolution kernel size is 3*3, the number of output channels becomes 128, and the feature map is reduced to 1 / 4 of the initial image again; The third layer C2f-DCNv2 module performs feature extraction and learning on the input feature map three times, and the number of output channels is 128; The fourth layer DCNv2 module performs convolution on the input feature map again. The convolution kernel size is 3*3, the number of output channels becomes 256, and the feature map is reduced to 1 / 8 of the initial image again. The fifth layer C2f-DCNv2 module performs 6 repeated feature extraction and learning on the input feature map, and the number of output channels is 256; The sixth layer DCNv2 module continues to perform convolution operations on the input feature map. The convolution kernel size is 3*3, the number of output channels becomes 512, and the feature map is reduced to 1 / 16 of the initial image again; The seventh layer C2f-DCNv2 module performs 6 repeated feature extraction and learning on the input feature map, and the number of output channels is 512; The eighth layer DCNv2 module performs convolution on the feature map again. The convolution kernel size is 3*3, the number of output channels becomes 1024, and the feature map is reduced to 1 / 32 of the initial image again. The ninth layer C2f-DCNv2 module performs feature extraction and learning on the input feature map three times, and the number of output channels is 1024; The tenth-layer SPPF-LSKA module performs a large separable kernel attention and spatial pyramid pooling fusion operation on the input feature map. The pooling kernel size is 5, which is used to extract multi-scale feature representations, thereby capturing the detailed information of the target object at different scales.

[0043] The eleventh layer, nn.Upsample, uses the nearest neighbor interpolation algorithm as the upsampling algorithm with a scaling factor of 2. The output feature map size is twice the size of the input feature map, and the number of channels remains unchanged. The twelfth Concat layer, the feature fusion layer, concatenates and fuses the P4 feature map of the Backbone structure with other feature maps, with a concatenation dimension of 1; The thirteenth layer C2f-DCNv2 module performs feature extraction and learning on the input feature map three times, and the number of output channels is 512; The fourteenth layer, nn.Upsample, uses the nearest neighbor interpolation algorithm as the upsampling algorithm with a scaling factor of 2. The output feature map size is twice the size of the input feature map, and the number of channels remains unchanged. The fifteenth Concat layer is a feature fusion layer that concatenates and fuses the P3 feature map of the Backbone structure with other feature maps, with a concatenation dimension of 1. The sixteenth layer C2f-DCNv2 module performs feature extraction and learning on the input feature map three times, with the number of output channels being 512, and the feature map being reduced to 1 / 8 of the initial image; The seventeenth layer DCNv2 module performs convolution operation on the input feature map again, the convolution kernel size is 3*3, and the number of output channels becomes 256; The eighteenth Concat layer, the feature fusion layer, concatenates and fuses the P4 feature map of the Head structure with other feature maps, and the concatenation dimension is 1; The nineteenth layer C2f-DCNv2 module performs feature extraction and learning on the input feature map three times, with the number of output channels being 512, and the feature map being reduced to 1 / 16 of the initial image; The twentieth layer of DCNv2 module performs convolution operation on the input feature map again, the convolution kernel size is 3*3, and the number of output channels becomes 512; The twenty-first Concat layer is a feature fusion layer that concatenates and fuses the P5 feature map of the Head structure with other feature maps, with a concatenation dimension of 1. The 22nd layer C2f-DCNv2 module performs feature extraction and learning on the input feature map three times, with the number of output channels being 1024, and the feature map being reduced to 1 / 32 of the initial image; The twenty-third layer Detect detection head is used for target detection and generates the final output.

[0044] In addition, by using shared convolution, feature maps from different branches are fused to form richer interactive features. At the same time, task decomposition is performed through the TD (Task Decomposition) module, and the interactive features are further processed to adapt to different tasks. The task features of each classification or positioning task are calculated separately: Where: Represents the kth element of the subtask interaction feature; Represents cross-layer interaction features; is the kth element of the learned attention layer; w is the attention vector used to capture the dependencies between layers: Where: w is a fully connected layer through two layers and The calculated attention vector; is the sigmoid activation function; It is through The average pooling result is Finally, from each The classification or positioning results are predicted.

[0045] In an embodiment of the present invention, a surface image of an elevator steel strip is obtained by a method in an embodiment of the present invention, and the surface image of the elevator steel strip is collected based on multiple scenes and multiple angles; the multiple surface images are formed into a complex data set, and the complex data set is interacted with the YOLOv8n model; the YOLOv8n model is optimized based on the complex data set, and a surface fault detection model of the elevator steel strip is output; an image to be detected of the elevator steel strip is collected, and the surface fault information of the elevator steel strip is output according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, the surface fault information of the elevator steel strip includes wear and tear, mouse bite damage or transverse cracks. At this time, the YOLOv8n model is trained based on the complex data set so as to optimize the YOLOv8n model, thereby outputting the surface fault detection model of the elevator steel strip, so as to output the surface fault information of the elevator steel strip according to the image to be detected of the elevator steel strip and the surface fault detection model of the elevator steel strip, thereby realizing intelligent fault detection of the image to be detected of the elevator steel strip and ensuring the accuracy of the surface fault information of the elevator steel strip.

[0046] Example 2 See also Fig. 9 , Fig. 9 It is a schematic diagram of the structural composition of a system for detecting surface defects of an elevator steel belt in an embodiment of the present invention.

[0047] like Fig. 9 As shown, a system for detecting surface defects of an elevator steel strip, the system for detecting surface defects of an elevator steel strip comprising: An acquisition module 21 is used to acquire a surface image of an elevator steel belt, where the surface image of the elevator steel belt is acquired based on multiple scenes and multiple angles; An interaction module 22, for forming a complex data set from a plurality of surface images, and interacting the complex data set with a YOLOv8n model; An optimization module 23 is used to optimize the YOLOv8n model based on a complex data set and output an elevator steel belt surface fault detection model; The surface fault module 24 is used to collect the image to be detected of the elevator steel belt, and output the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

[0048] Example 3 See also Fig.10 , refer to the following Fig.10 The electronic device 40 according to this embodiment of the present invention will be described. Fig.10 The electronic device 40 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0049] like Fig.10 As shown, the electronic device 40 is in the form of a general computing device. The components of the electronic device 40 may include but are not limited to: at least one processing unit 41, at least one storage unit 42, and a bus 43 connecting different system components (including the storage unit 42 and the processing unit 41).

[0050] The storage unit stores program codes, which can be executed by the processing unit 41, so that the processing unit 41 executes the steps according to various exemplary embodiments of the present invention described in the above “Embodiment Method” section of this specification.

[0051] The storage unit 42 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 421 and / or a cache memory unit 422 , and may further include a read-only memory unit (ROM) 423 .

[0052] The storage unit 42 may also include a program / utility 424 having a set (at least one) of program modules 425, such program modules 425 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0053] Bus 43 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0054] The electronic device 40 may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 40, and / or any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 44. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 45. Fig.10 As shown, the network adapter 45 communicates with other modules of the electronic device 40 via the bus 43. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup planning systems.

[0055] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0056] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. In addition, it stores computer program instructions, and when the computer program instructions are executed by a computer, the computer executes the above method.

[0057] In addition, the above is a detailed introduction to the detection method and system for the surface defects of the elevator steel belt provided in the embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in the field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting surface defects of elevator steel strips, characterized in that: Applied to elevator steel belts; The method for detecting surface defects of an elevator steel belt comprises: Acquire a surface image of an elevator steel belt, where the surface image of the elevator steel belt is collected based on multiple scenes and multiple angles; Form multiple surface images into a complex dataset and interact the complex dataset with the YOLOv8n model; Based on complex data sets, the YOLOv8n model is optimized and the elevator steel belt surface fault detection model is output; The image to be detected of the elevator steel belt is collected, and the surface fault information of the elevator steel belt is output according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

2. The method for detecting surface defects of elevator steel strip according to claim 1, characterized in that: The acquiring of the surface image of the elevator steel belt, where the surface image of the elevator steel belt is collected based on multiple scenes and multiple angles, includes: Collect the positioning of the elevator steel belt, and trigger the shooting of the elevator steel belt based on the positioning of the elevator steel belt; Based on the shooting of the elevator steel belt, the surface image of the elevator steel belt is output, wherein the orientation of the elevator steel belt and the scene where the elevator steel belt is located are marked, multiple coordination is performed according to the orientation of the elevator steel belt and the scene where the elevator steel belt is located, and the corresponding surface image is output according to the multiple coordination of the orientation of the elevator steel belt and the scene where the elevator steel belt is located.

3. The method for detecting surface defects of elevator steel strip according to claim 2, characterized in that: The method forms a complex data set with multiple surface images and interacts the complex data set with the YOLOv8n model, including: Collect multiple surface images and form the multiple surface images into a complex data set; GAN based on unsupervised learning performs data enhancement processing on complex data sets and outputs processed data sets; The processed data set is divided into training set, validation set and test set according to the proportion. The training set, validation set and test set are used in the test scenario, training scenario and validation scenario of the model respectively. Collect YOLOv8n model; Freeze the convolutional modules in the YOLOv8n model and replace them with the deformable volume DCNv2 module to optimize the YOLOv8n model for the first time. Interact the processed dataset with the YOLOv8n model.

4. The method for detecting surface defects of elevator steel strip according to claim 3, characterized in that: The method of forming a complex data set from multiple surface images and interacting the complex data set with the YOLOv8n model further includes: In the deformable convolution DCNv2, by introducing offsets and deformable convolution kernels, the YOLOv8n model learns the offsets and weights of each sampling point, and uses R-CNN to perform knowledge distillation on Faster R-CNN, so that the deformable convolution can learn better sampling points, as well as better offsets and weights of the sampling points. Among them, y(p) represents the prime value of position p in the output feature map; represents the weight, which is used to perform weighted summation on the interpolation; x represents the pixel value in the input feature map; represents the position of the sampling point in the input feature map; p is the position in the output feature map, is the offset of the sampling point relative to p, is the offset learned from the deformable convolution; Represents the weight of the offset learned by deformable convolution.

5. The method for detecting surface defects of elevator steel strip according to claim 3, characterized in that: The method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model includes: The LSKA attention mechanism is introduced to improve the SPPF module in the YOLOv8 model and construct a new SPPF-LSKA module; In the new SPPF-LSKA module, the representation ability of the model is further enhanced through the local attention layer, kernelized attention layer and self-attention fusion; Through the deep dilated convolution (DW-Conv) module, we can focus on local information to locate the local area; The Gaussian kernel function is used to calculate the correlation score between each pair of local regions. At the same time, the output of the kernelized attention is element-wise multiplied with the input, and then a 1×1 convolution layer is used to fuse the local attention and global attention features to output the elevator steel belt surface fault detection model.

6. The method for detecting surface defects of elevator steel strip according to claim 5, characterized in that: The method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model also includes: Local attention is calculated as follows: in, is the feature representation of region i, N is the number of local regions in the image, and K(i,j) is the kernel function used to measure the correlation between region i and region j; The calculation formula of the relevance score K(i,j) is: in, and is the feature representation of local regions i and j, represents the norm of a vector, It is the parameter of the Gaussian kernel function, which is used to control the bandwidth of the kernel function.

7. The method for detecting surface defects of elevator steel strip according to claim 6, characterized in that: The method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model also includes: The attention weight between each pair of local regions is obtained by exponentially and normalizing the correlation score , which is used to represent weighted local features to better capture the nonlinear relationship between different regions in the image. The calculation formula is: Where N represents the number of local regions in the image, and K(i,j) is the correlation score between local regions i and j calculated by the kernel function.

8. The method for detecting surface defects of elevator steel strip according to claim 7, characterized in that: The method of optimizing the YOLOv8n model based on a complex data set and outputting an elevator steel belt surface fault detection model also includes: The process of self-attention fusion can be expressed by the following formula: in, Expressing local concern, Expressing global concern, Represents the fusion parameter, which is used to control the weight of the two.

9. The method for detecting surface defects of elevator steel strip according to claim 8, characterized in that: The method collects the image to be detected of the elevator steel belt, and outputs the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt, wherein the surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks, including: Collect the image of the elevator steel belt to be tested; The surface fault information of the elevator steel belt is output according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

10. A system for detecting surface defects of elevator steel strips, characterized in that: The surface defect detection system of the elevator steel strip is applied to the surface defect detection method of the elevator steel strip as claimed in any one of claims 1 to 9, and the surface defect detection system of the elevator steel strip comprises: An acquisition module, used to acquire a surface image of an elevator steel belt, wherein the surface image of the elevator steel belt is acquired based on multiple scenes and multiple angles; An interaction module, which is used to form a complex dataset from multiple surface images and interact the complex dataset with the YOLOv8n model; The optimization module is used to optimize the YOLOv8n model based on complex data sets and output the elevator steel belt surface fault detection model; The surface fault module is used to collect the image to be detected of the elevator steel belt, and output the surface fault information of the elevator steel belt according to the image to be detected of the elevator steel belt and the surface fault detection model of the elevator steel belt. The surface fault information of the elevator steel belt includes wear and tear, rat bite damage or transverse cracks.

Images