A multi-camera cable defect detection method and system based on deep learning

Through the combination of multi-camera shooting and custom pre-selected frame models, the limitation of viewing angles and deformation problems in cable defect detection are solved, and high-precision defect detection is achieved, which is suitable for cable quality inspection in various scenarios.

CN118657782BActive Publication Date: 2024-11-29HANGZHOU ZHIJUAN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411149330.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-29
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

The existing cable defect detection methods have problems with limited viewing angle and image deformation, making it difficult to achieve high-precision defect detection, especially when detecting small defects, it is prone to missed or missed detection.

Method used

Multiple cameras are used to capture cable images from different angles, eliminate deformation through image flattening processing, and build an object detection model based on custom pre-selected boxes to improve detection performance through an optimized training process.

Benefits of technology

It significantly improves the accuracy and accuracy of detection, can operate stably under different environments and conditions, reduces the error of manual detection, and meets the high-precision detection needs in industrial production.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657782B_ABST
    Figure CN118657782B_ABST
Patent Text Reader

Abstract

The present invention discloses a multi-camera cable defect detection method and system based on deep learning, including: constructing an initial cable image set; flattening the image and extracting the cable area to obtain a cable area image, and constructing a first cable image set; splicing multiple cable area images taken of the same cable area to obtain a complete cable image of the cable area; annotating defects on all complete cable images to obtain a defect annotation data set; constructing a target detection model based on a custom pre-selected box, and training the target detection model using the defect annotation data set to obtain a trained target detection model; using the trained target detection model to perform defect detection on the cable image to be detected to obtain a defect detection result. The present invention improves the comprehensiveness and accuracy of cable defect detection through an improved deep learning model, meets the real-time detection needs in industrial production, and has a wide range of application value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing and deep learning, and specifically relates to a multi-camera cable defect detection method and system based on deep learning. Background Art

[0002] In modern industrial production, cable quality inspection is a vital link. Existing cable defect detection methods mainly rely on a single camera for image acquisition, and then detect defects through image processing algorithms. However, this method has some significant shortcomings. First, a single camera can only obtain cable surface images from one angle, and it is impossible to fully capture all the details of the cable, especially defects hidden in the shadows or detection blind spots caused by changes in lighting. Secondly, the cable may bend and twist during the shooting process, causing the captured image to deform, thus affecting the subsequent defect detection accuracy. In addition, traditional image processing algorithms often have poor results in detecting small defects, and are prone to missed detection or false detection, which cannot meet the needs of high-precision detection.

[0003] In recent years, deep learning technology has made remarkable progress in the field of computer vision, especially in target detection. However, despite this, existing cable defect detection methods still face many challenges in practical applications. For example, how to better combine multi-camera image acquisition technology to improve the comprehensiveness and accuracy of detection is still a technical problem that needs to be solved urgently. At the same time, in the existing technology, defect detection based on deep learning models is difficult to achieve high-precision detection effects at the same time when facing images with different defect sizes. For example, for larger defects in the image, there may be situations where they cannot be fully identified; and for smaller defects, detection may be missed. Therefore, it is urgent to optimize the existing technology to meet the needs of high-precision detection in industrial production. Summary of the invention

[0004] In order to solve the problems in the prior art, the present invention provides a multi-camera cable defect detection method and system based on deep learning. Cable images are captured from different angles by multiple cameras, image flattening technology is used to eliminate deformation, and detection performance is improved through custom pre-selection boxes and optimized training processes.

[0005] The technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention discloses a multi-camera cable defect detection method based on deep learning, comprising the following steps:

[0007] 1) Use multiple cameras to shoot defective cables from different angles to obtain cable defect images and construct an initial cable image set;

[0008] 2) Smoothing all images in the initial cable image set, and extracting cable regions from the smoothed images to obtain cable region images, thereby constructing a first cable image set;

[0009] 3) stitching together multiple images of the same cable section in the first cable image set to obtain a complete cable image of the cable section;

[0010] 4) Annotate all complete cable images for defects and obtain a defect annotation dataset;

[0011] 5) Construct a target detection model based on a custom pre-selected box, input the defect annotation dataset into the target detection model for training, and obtain a trained target detection model;

[0012] 6) Using the trained target detection model to perform defect detection on the cable image to be detected, and obtaining a defect detection result; wherein the defect detection result includes the type of defect and the location of the defect.

[0013] In a second aspect, the present invention discloses a multi-camera cable defect detection system for implementing the method, comprising:

[0014] An initial cable image set construction module is used to photograph defective cables from different angles using multiple cameras to obtain cable defect images and construct an initial cable image set; a first cable image set construction module is used to flatten all images in the initial cable image set, extract cable regions from the flattened images, obtain cable region images, and construct a first cable image set;

[0015] An image stitching module, which is used to stitch multiple images of the same cable section in the first cable image set to obtain a complete cable image of the cable section;

[0016] A data set acquisition module, which is used to annotate all complete cable images for defects and obtain a defect annotated data set;

[0017] Model building module, which is used to build a target detection model based on custom pre-selected boxes;

[0018] A model training module, which is used to train the target detection model through a defect annotation data set to obtain a trained target detection model;

[0019] The defect detection module is used to use the trained target detection model to perform defect detection on the cable image to be detected to obtain a defect detection result; wherein the defect detection result includes the type of defect and the location of the defect.

[0020] In a third aspect, the present invention discloses a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, they are used to implement the multi-camera cable defect detection method based on deep learning.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. Aiming at the problems of limited viewing angle and image deformation in the existing cable defect detection methods, the present invention proposes a cable defect detection method and device based on multiple cameras and custom pre-selection boxes. The method and device are simple and efficient, and the output detection results meet the real-time detection requirements in industrial production.

[0023] 2. The present invention uses multiple cameras to simultaneously capture cable images from different angles, thereby solving the problem of missed defect detection caused by a single viewing angle and significantly improving detection accuracy.

[0024] 3. The present invention adopts image flattening processing technology to flatten the bent and deformed cable images, eliminating the deformation problem in the image and providing high-quality image data for subsequent splicing and defect detection.

[0025] 4. The present invention customizes the pre-selection box of the target detection model and combines it with an optimized training process, so that the model can better adapt to defects of different sizes and types, improves the detection performance and accuracy of the model, realizes efficient defect detection, and is suitable for cable defect detection in various scenarios.

[0026] 5. The present invention provides a reliable cable quality detection method that can operate stably under different environments and conditions, reduces manual detection errors, improves the overall reliability of the system, and has wide applicability and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flowchart of a multi-camera cable defect detection method based on deep learning provided by an embodiment of the present invention;

[0028] Figure 2 is a structural schematic diagram of a multi-camera cable defect detection device based on deep learning provided by an embodiment of the present invention;

[0029] Figure 3 is a diagram of camera arrangement when acquiring cable defect images according to the present invention;

[0030] Figure 4 are the original image and result image of defect detection of the present invention;

[0031] Figure 5 It is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0033] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is described in detail below in conjunction with specific embodiments. Specific embodiments are described below to simplify the present invention. However, it should be recognized that the present invention is not limited to the illustrated embodiments, and various modifications of the present invention are possible without departing from the basic principles, and these equivalent forms also fall within the scope defined by the claims attached to the present application.

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1 An embodiment of the present invention provides a multi-camera cable defect detection method based on deep learning, which mainly includes the following steps:

[0036] S1: Acquire cable defect images and construct an initial cable image set.

[0037] S2: Smoothing the images in the initial cable image set by an image processing method, and extracting cable regions from the smoothed images to obtain cable region images, thereby constructing a first cable image set.

[0038] S3: stitching together multiple images of the same cable area in the first cable image set to obtain a complete cable image of the cable area.

[0039] S4: Annotate all complete cable images for defects, create a defect annotation dataset, and divide it into a training set and a test set in proportion.

[0040] S5: construct a target detection model based on the custom pre-selected box, input the defect annotated data set into the target detection model for training, and obtain a trained target detection model.

[0041] S6: Use the trained target detection model to perform defect detection on the cable image to be detected to obtain a defect detection result; wherein the defect detection result includes the type of defect and the location of the defect.

[0042] See also Figure 4 , Figure 4 The picture on the left is the image of the cable to be tested. Figure 4 The figure on the right is the defect detection result.

[0043] The specific implementation method is introduced below:

[0044] 1. Constructing the initial cable image set

[0045] like Figure 3 As shown, three line scan cameras are used, which are evenly spaced in a circle, to collect the initial cable image set. The camera arrangement enables the cable images to be taken from different angles at the same time, providing a comprehensive perspective. The cable is illuminated with an oblique light source to ensure that the details of the cable surface in the image are clear and reduce the impact of light shadows and reflections on image quality.

[0046] 2. Constructing the first cable image set

[0047] All images in the initial cable image set are flattened, and cable regions are extracted from the flattened images to obtain cable region images, thereby constructing a first cable image set. The specific steps include:

[0048] 2.1) For each cable defect image Perform binarization to obtain the threshold image . Use an adaptive thresholding method (such as Otsu's method) to convert the grayscale image Convert to threshold image :

[0049] ;

[0050] in, is the threshold calculated by Otsu's method; For the The midpoint of the cable defect image The pixel value of For the The midpoint of the threshold image The pixel value of

[0051] 2.2) In the threshold image In the first row of , find the position of the first non-black pixel and record it as the reference position :

[0052] ;

[0053] 2.3) Find the threshold image The position of the first non-black pixel in each row except the first row , and calculate the offset ;

[0054] ;

[0055] ;

[0056] 2.4) According to the offset Threshold image The pixels of each row except the first row are translated to make the cable part in the cable defect image flat. The flattened image is obtained ,

[0057] ;

[0058] in, The image after flattening midpoint The pixel value of

[0059] 2.5) For the smoothed image Extract the cable area and the left boundary of the extracted cable area and right border They are:

[0060] ;

[0061] ;

[0062] Get cable area image ,

[0063] ;

[0064] in, Cable area image midpoint The pixel value of .

[0065] (III) Image of cable splicing area

[0066] Cable Area Image The width is , height is ; Wherein, the heights of the multiple cable area images taken of the same cable area are equal, and the multiple cable area images taken of the same cable area are spliced ​​along the width direction to obtain a complete cable image of the cable area. The specific method includes:

[0067] 3.1) Define the image stitching area. Assume that the three cable defect images of the same cable area are flattened and the images after the cable area is extracted are , and , whose widths are , and , the height is . Define the final stitched image Width for:

[0068] ;

[0069] image The height is .

[0070] 2) Image , and Stitch along the horizontal axis (width direction), stitching images Pixel value Obtained by the following formula:

[0071] ;

[0072] By using the above formula, the three images are spliced ​​into a complete cable image of the cable area.

[0073] 4. Defect annotation and dataset creation

[0074] Defects are annotated on the complete cable image of the cable section, and a defect annotation dataset is created, which is divided into a training set and a test set in proportion. The specific steps include:

[0075] 1) Defect annotation: input the complete cable image of the cable area obtained by splicing into the image annotation tool, and manually or semi-automatically annotate the defects in each image. The annotation information should include the type, location (in the form of a rectangular box) and size of the defect; the types of defects include holes, impurities, wrinkles, bubbles, etc. These defect types may be specific to a certain type of cable.

[0076] 2) Generate an annotation file and save the annotation information as an annotation file. The annotation file format can be XML or JSON. The annotation file should contain the image file name, image size and defect annotation information.

[0077] 3) Dataset division: divide the labeled image dataset into training set and test set according to the ratio. The division ratio can be 80% for training and 20% for testing. The specific steps are as follows: list all the labeled image files and shuffle them. According to the set ratio, the first 80% of the image files are used as the training set, and the remaining 20% ​​of the image files are used as the test set.

[0078] 4) Generate the file structure of the dataset, and generate the file structure of the training set and the test set according to the division results. The file structure should contain the image files of the training set and the test set and the corresponding annotation files.

[0079] 5. Custom configuration.

[0080] In step five, the target detection model is obtained by replacing the original pre-selected box in the first target detection model with a custom pre-selected box. The first target detection model includes a pre-selected box-based target detection model such as a YOLO series model, RetinaNet, and FasterR-CNN. The YOLO series model can be a YOLOv8 model, a YOLOv7 model, a YOLOv6 model, a YOLOv5 model, a YOLOv2 model, or a YOLOv3 model. The custom pre-selected box is obtained by a k-means clustering algorithm, and the number of the custom pre-selected boxes is equal to the number of the original pre-selected boxes in the first target detection model.

[0081] In a specific embodiment of the present invention, the first target detection model is a YOLOv5 model. By customizing the pre-selection box of the YOLOv5 model, the model's detection capability for defects of different sizes is optimized. The specific steps include:

[0082] 5.1) Customized pre-selection box. In order to optimize the model's ability to detect defects of different sizes, the pre-selection box of the YOLOv5 model is customized. The customized pre-selection box has the same function as the original pre-selection box in the model. The model with customized pre-selection box can better adapt to defects of different sizes and types, and the detection results will be more accurate. The pre-selection box is a predefined bounding box used to detect objects of different scales. The steps to customize the pre-selection box include:

[0083] The width and height of each defect in the defect annotation data set are obtained, and the width and height of all defects are used as inputs of the k-means clustering algorithm to obtain k cluster centers. The values ​​corresponding to the k cluster centers are used as the width and height of k custom pre-selected boxes, where k is the number of original pre-selected boxes in the first object detection model.

[0084] So, the custom pre-selected box is:

[0085] ;

[0086] in, For custom pre-selected boxes; The width of the kth custom pre-selected box; The height of the kth custom pre-selected box.

[0087] 5.2) Configure the YOLOv5 model and add the custom pre-selection box to the YOLOv5 model so that the model can better detect defects of different sizes in the dataset. The specific configuration method is as follows:

[0088] a) Modify the configuration file of the YOLOv5 model and replace the default pre-selected box with a custom pre-selected box. Specifically:

[0089] Multiply the width and height of the custom pre-selected box to obtain a product result, and then sort all the custom pre-selected boxes from small to large based on the product result;

[0090] Multiply the width and height of the original pre-selected box to obtain a product result, and then sort all the original pre-selected boxes from small to large based on the product result;

[0091] b) Use the sorted k custom pre-selected boxes to replace the sorted k original pre-selected boxes in sequence, that is, use the custom pre-selected boxes in each detection layer of the model to optimize the target detection effect.

[0092] (6) Model training.

[0093] The defect annotated dataset is input into the target detection model for training to obtain a trained target detection model. The specific steps include configuring the target detection model, training with the training set, iteratively updating the model weights, and minimizing the loss function.

[0094] In this embodiment, a multi-camera cable defect detection system based on deep learning is also provided, including:

[0095] An initial cable image set construction module is used to photograph defective cables from different angles using multiple cameras to obtain cable defect images and construct an initial cable image set; a first cable image set construction module is used to flatten all images in the initial cable image set, extract cable regions from the flattened images, obtain cable region images, and construct a first cable image set;

[0096] An image stitching module, which is used to stitch multiple images of the same cable section in the first cable image set to obtain a complete cable image of the cable section;

[0097] A data set acquisition module, which is used to annotate all complete cable images for defects and obtain a defect annotated data set;

[0098] Model building module, which is used to build a target detection model based on custom pre-selected boxes;

[0099] A model training module, which is used to train the target detection model through a defect annotation data set to obtain a trained target detection model;

[0100] The defect detection module is used to use the trained target detection model to perform defect detection on the cable image to be detected to obtain a defect detection result; wherein the defect detection result includes the type of defect and the location of the defect.

[0101] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0102] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.

[0103] like Figure 5 As shown, an embodiment of the present invention further provides an electronic device, including a memory and a processor;

[0104] The memory is used to store computer programs;

[0105] The processor is used to implement the above-mentioned multi-camera cable defect detection method based on deep learning when executing the computer program.

[0106] An embodiment of the present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the above-mentioned multi-camera cable defect detection method based on deep learning is implemented.

[0107] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc., equipped on the device.

[0108] Furthermore, the computer-readable storage medium may include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0109] Obviously, the above-described embodiments and drawings are only some examples of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that, although the work done in this development process may be complicated and lengthy, for those of ordinary skill in the art, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application. Without departing from the concept of the present application, several variations and improvements can also be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A multi-camera cable defect detection method based on deep learning, characterized in that: The steps include: 1) Use multiple cameras to shoot defective cables from different angles to obtain cable defect images and construct an initial cable image set; 2) Smoothing all images in the initial cable image set, and extracting cable regions from the smoothed images to obtain cable region images, thereby constructing a first cable image set; The step 2) comprises: 2.1) Cable defect images in the initial cable image set Perform binarization to obtain the respective threshold images , ; in, is the threshold value; For the The midpoint of the cable defect image The pixel value of For the The midpoint of the threshold image The pixel value of 2.2) In the threshold image In the first row of , find the position of the first non-black pixel and record it as the reference position , ; 2.3) Find the threshold image The position of the first non-black pixel in each row except the first row , and calculate the offset , ; ; 2.4) According to the offset Threshold image The pixels of each row except the first row are translated to make the cable part in the cable defect image flat, and the flattened image is obtained. , ; in, The image after flattening midpoint The pixel value of 2.5) For the smoothed image Extract the cable area and the left boundary of the extracted cable area and right border They are: ; ; Get cable area image , ; in, Cable area image midpoint The pixel value of 3) stitching together multiple images of the same cable section in the first cable image set to obtain a complete cable image of the cable section; The step 3) comprises: Cable Area Image The width is , height is ; The heights of the multiple cable area images taken of the same cable area are equal, and the multiple cable area images taken of the same cable area are spliced ​​along the width direction to obtain a complete cable image of the cable area; 4) Annotate all complete cable images for defects and obtain a defect annotation dataset; 5) Construct a target detection model based on a custom pre-selected box, input the defect annotation dataset into the target detection model for training, and obtain a trained target detection model; 6) Using the trained target detection model to perform defect detection on the cable image to be detected, and obtaining a defect detection result; wherein the defect detection result includes the type of defect and the location of the defect.

2. The multi-camera cable defect detection method based on deep learning according to claim 1 is characterized in that: In step 1), the cable images are captured from different angles by using multiple cameras, including: Three line scan cameras are circumferentially arranged around the cable, and the three line scan cameras are evenly distributed; and the three line scan cameras are used to capture images of the cable.

3. The multi-camera cable defect detection method based on deep learning according to claim 1, characterized in that: In step 4), the defect annotation includes annotating the defect type, defect location and defect size, wherein the defect location is represented in the form of a rectangular frame; The types of defects include holes, inclusions, wrinkles or bubbles.

4. The multi-camera cable defect detection method based on deep learning according to claim 1, characterized in that: In step 5), the target detection model is obtained by replacing the original pre-selected boxes in the first target detection model with the custom pre-selected boxes, the first target detection model includes a YOLO series model, RetinaNet or Faster R-CNN, and the YOLO series model is a YOLOv8 model, a YOLOv7 model, a YOLOv6 model, a YOLOv5 model, a YOLOv2 model or a YOLOv3 model; the custom pre-selected boxes are obtained by a k-means clustering algorithm, and the number of the custom pre-selected boxes is equal to the number of the original pre-selected boxes in the first target detection model.

5. The multi-camera cable defect detection method based on deep learning according to claim 4 is characterized in that: The method for obtaining the custom pre-selection box includes: The width and height of each defect in the defect annotation data set are obtained, and the width and height of all defects are used as inputs of the k-means clustering algorithm to obtain k cluster centers. The values ​​corresponding to the k cluster centers are used as the width and height of k custom pre-selected boxes, where k is the number of original pre-selected boxes in the first object detection model. So, the custom pre-selected box is: ; in, For custom pre-selected boxes; The width of the kth custom pre-selected box; The height of the kth custom pre-selected box.

6. The multi-camera cable defect detection method based on deep learning according to claim 5 is characterized in that: The object detection model based on the custom pre-selected box is constructed, including: Multiply the width and height of the custom pre-selected box to obtain a product result, and then sort all the custom pre-selected boxes from small to large based on the product result; Multiply the width and height of the original pre-selected box to obtain a product result, and then sort all the original pre-selected boxes from small to large based on the product result; The sorted k custom pre-selected boxes are used to replace the sorted k original pre-selected boxes in sequence.

7. A multi-camera cable defect detection system implementing the method of claim 1, characterized in that: include: An initial cable image set construction module is used to photograph defective cables from different angles using multiple cameras to obtain cable defect images and construct an initial cable image set; A first cable image set construction module, which is used to flatten all images in the initial cable image set, extract cable regions from the flattened images, obtain cable region images, and construct a first cable image set; An image stitching module, which is used to stitch multiple images of the same cable section in the first cable image set to obtain a complete cable image of the cable section; A data set acquisition module, which is used to annotate all complete cable images for defects and obtain a defect annotated data set; Model building module, which is used to build a target detection model based on custom pre-selected boxes; A model training module, which is used to train the target detection model through a defect annotation data set to obtain a trained target detection model; The defect detection module is used to use the trained target detection model to perform defect detection on the cable image to be detected to obtain a defect detection result; wherein the defect detection result includes the type of defect and the location of the defect.

8. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions, which, when called and executed by a processor, are used to implement the multi-camera cable defect detection method based on deep learning as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Data processing method and related device

    CN116958582A

  • Cable opening surface defect detection method and device

    CN118134830A