Recycled book surface defect detection method based on DD-Net architecture

The DD-Net architecture with deformable convolution and dense feature pyramid fusion addresses the challenges of detecting complex defects in recycled books, achieving high accuracy and efficiency in surface defect detection.

CN119067956APending Publication Date: 2024-12-03Shanxi Taihang Laboratory Co., Ltd.

Patent Information

Application Number
CN202411260002.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing surface defect detection methods struggle to effectively handle complex backgrounds and multiple defect types in recycled and recirculated books, particularly due to limitations in feature extraction and geometric deformation handling.

Method used

A deep learning-based method utilizing the DD-Net architecture, incorporating a deformable convolution operator module and densely connected pyramid feature fusion, to adapt to various shape defects and enhance feature extraction and detection accuracy.

Benefits of technology

The method achieves rapid, automated, and real-time detection of surface defects in recycled and recirculated books, with improved accuracy and robustness, demonstrated by leading COCO mean average precision (mAP) results compared to baseline models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067956A_ABST
    Figure CN119067956A_ABST
Patent Text Reader

Abstract

The invention discloses a recycling and recirculating book surface defect detection method based on a DD-Net framework, and relates to the technical field of object surface defects. Comprising the following steps: S1, data acquisition; s2, a data preprocessing step; s3, a frame building step; and S4, implementing a detection step. When the method is used for detecting the surface defects of the recycled and recirculated books, complex geometric deformation in the image can be effectively recognized, and various types of defects can be effectively detected by connecting the information of the uppermost layer with the information of the lowermost layer and extracting multi-scale features with rich semantic information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of object surface defects, and in particular to a method for detecting surface defects of recycled and recirculated books based on a DD-Net architecture. Background Art

[0002] In modern industrial production and quality control, defect detection technology plays a vital role. Mainstream technologies include magnetic particle testing, penetration testing, eddy current testing, ultrasonic testing and machine vision. Among them, surface defect detection based on machine vision is becoming more and more the core of promoting production automation and smart product pricing strategy due to its non-contact, non-destructive, wide spectral response range, ability to work in harsh environments, and high production efficiency.

[0003] A. Traditional surface defect detection methods

[0004] Traditional methods have long been an integral part of surface defect detection, relying mainly on image processing techniques and machine learning algorithms to analyze and identify defects through manually designed feature extraction. These methods perform well when dealing with simple backgrounds and single defect types, but may encounter difficulties with complex backgrounds or multiple defect types.

[0005] 1) Transform domain based methods

[0006] Signal processing techniques such as Fourier transform, Gabor transform and wavelet transform are used to convert images from the spatial domain to the frequency domain to highlight defect features and facilitate subsequent analysis. For example, Chetverikov et al. proposed a detection method based on texture direction variation, specifically targeting mutation defects on the surface of textiles. Hou et al. extracted the frequency domain information of the image through Gabor wavelet transform and combined it with support vector machine (SVM) for classification, which is particularly effective for images with rich textures.

[0007] 2) Methods based on statistical analysis

[0008] Statistical feature extraction methods, such as grayscale variation difference and grayscale histogram, identify anomalies by analyzing the distribution pattern of image pixels. In addition, color features, including color space transformation and color histogram, play a vital role in characterizing defects in color images.

[0009] 3) Machine learning-based methods

[0010] Traditional machine learning algorithms, such as SVM and random forest, effectively classify defects by mapping features into decision space.

[0011] B. Surface defect detection method based on deep learning

[0012] In recent years, deep learning, especially convolutional neural network (CNN), has shown extraordinary potential in surface defect detection due to its excellent feature learning ability and robustness to complex backgrounds. Unlike traditional methods, deep learning methods can automatically extract features without human intervention, greatly improving the detection accuracy and generalization ability of complex patterns and various defects.

[0013] 1) Two-stage network

[0014] Bin Wan et al. introduced a semantically aware multi-layer feature interaction network that combines hierarchical features with semantically aware modules to meet the efficiency requirements of industrial defect detection. Dahai Liao et al. used semantic segmentation and decision subnetworks to solve the efficiency and accuracy challenges of Si3N4 bearing cylindrical roller detection. Hui Zhang et al. developed a multi-model CNN system that can identify various railway surface defects. Guang Wan et al. built an efficient tile defect detection system with an enhanced backbone network and a small-scale detection layer.

[0015] 2) Single-stage network

[0016] Xiaoxuan Guo et al. improved the accuracy of wheelset defect detection through the innovative receptive field and adjustable weight mechanism of the URS model. MIBin Roslan et al. proposed a real-time detection method for plastic surface defects and trained it on a customized dataset. Yunpeng Wu et al. developed a railway surface defect detection model that combines surface and edge features. Wenhao Li et al. combined spatial pyramid pooling with exponential moving average technology to design an automatic defect detection system for wire and arc additive manufacturing. Xuyi Yu et al. proposed a progressively refined redistributed pyramid network that performed well on various datasets.

[0017] Therefore, it is an urgent problem for those skilled in the art to propose a surface defect detection method for recycled and recirculated books based on the DD-Net architecture to solve the difficulties existing in the prior art. Summary of the invention

[0018] In view of this, the present invention provides a surface defect detection method for recycled and recirculated books based on the DD-Net architecture, aiming to explore more effective surface defect feature extraction technology for second-hand items, and design a high-performance detection architecture under the strict COCO mAP indicator, which will provide technical support for quality control and intelligent pricing strategies in the recycled and recirculated book market.

[0019] In order to achieve the above object, the present invention adopts the following technical solution: a method for detecting surface defects of recycled and recirculated books based on DD-Net architecture, comprising the following steps:

[0020] S1, data acquisition step: acquiring surface image data of recycled and recirculated books, and constructing a surface image database of recycled and recirculated books;

[0021] S2, data preprocessing step: based on the recycled and recirculated book surface image dataset, the defects are labeled according to the defect type to obtain a labeled recycled and recirculated book surface image dataset;

[0022] S3, framework building steps: Based on the labeled recycled and recirculated book surface image dataset obtained in S2, a DD-Net recycled and recirculated book surface defect recognition model is constructed;

[0023] S4. Implement the detection step: Based on the DD-Net recycled and recirculated book surface defect recognition model achieved in S3, surface defect detection is performed on the recycled and recirculated books to be detected, and defect detection results of the recycled and recirculated books to be detected are obtained.

[0024] In the above method, optionally, the defect types in S2 include: incomplete, stained, cracked, repaired, stamped and ink-stained.

[0025] The above method, optionally, the DD-Net recycled and recirculated book surface defect recognition model in S3 includes: a feature extraction network part, a deformable convolution operator module, a densely connected pyramid feature fusion module and a decoupled detection head;

[0026] Among them, the feature extraction network is used for multi-level feature extraction, and the deformable convolution operator module is used to adapt to various shape defect features.

[0027] The above method is optional, and the specific contents of S3 include:

[0028] S301, using the Darknet backbone network and deformable convolution module to extract multi-resolution feature maps from the labeled recycled and recirculated book surface image dataset obtained in S2;

[0029] S302, inputting the multi-resolution feature map obtained in S301 into a dense path aggregation feature pyramid network to obtain a multi-level feature map for each image, and connecting the generated multi-level feature maps in series through dense connections to obtain a multi-scale feature map;

[0030] S303, while the network is in the forward propagation process, learning the effective features in the training samples, matching and calculating with the ground truth values; using the back propagation algorithm to minimize the loss between the output feature map and the ground truth value, and optimizing the network parameters;

[0031] S304, multi-scale feature maps are fused through dense skip connections; a decoupled detection head is used to perform target localization and classification of defects, and finally an accurate prediction result of the defect target is generated.

[0032] In the above method, optionally, the specific contents in S301 include:

[0033] Feature extraction is performed through the deformable convolution operator module. An offset domain is added to the deformable convolution operator module to learn the size of the offset. The offset domain is connected to the network as a convolution layer. The calculation formula of the deformable convolution is as follows:

[0034]

[0035] Here, for each position p0 on the output feature map y, the deformable convolution operation uses the input feature map x combined with an offset Δp n to perform feature extraction.

[0036] In the above method, optionally, in S302, the generated multi-level feature maps are connected in series by dense connection. For each feature map, it is downsampled to the scale of other feature maps with the same spatial size, and then they are merged by tensor splicing, and the generated feature map is sent to the detection head.

[0037] The above method can optionally use average precision and average precision mean as evaluation indicators to perform surface defect detection on the recovered and recirculated books to be detected, and obtain defect detection results of the recovered and recirculated books to be detected.

[0038] It can be seen from the above technical solution that, compared with the prior art, the surface defect detection method of recycled and recirculated books based on the DD-Net architecture of the present invention has the following beneficial effects:

[0039] This paper proposes a deep learning-based method to address the challenges of detecting surface defects in recycled and recirculated books, and achieves rapid, automated, and real-time detection of surface defects in second-hand items.

[0040] Facing the new task of detecting surface defects in recycled and recirculated books, a dataset of surface defects in recycled and recirculated books containing 4781 images was constructed. This dataset covers a variety of book types, such as ancient books and recycled textbooks, and annotates 6 surface defect features, such as stains, insect bites, and damage.

[0041] The deformable convolution and densely connected feature pyramid modules are introduced to address the irregular changes in image shape and scale, respectively, providing effective solutions to practical problems.

[0042] In terms of COCO mean average precision (mAP) indicator, the model has achieved leading results by comparing with the baseline model YOLOv8 and performing ablation experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A flow chart of a method for detecting surface defects of recycled and recirculated books based on the DD-Net architecture provided by the present invention;

[0045] Figure 2 A schematic diagram of feature calculation of ordinary convolution and deformable convolution provided by the present invention;

[0046] Figure 3 A detailed schematic diagram of a dense feature pyramid network structure including a dense connection strategy provided by the present invention;

[0047] Figure 4 This is a specific operation flow chart of a dense feature pyramid network using N5 as an example provided in this embodiment. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0050] Reference Figure 1 As shown, the present invention discloses a method for detecting surface defects of recycled and recirculated books based on a DD-Net architecture, comprising the following steps:

[0051] S1, data acquisition step: acquiring surface image data of recycled and recirculated books, and constructing a surface image database of recycled and recirculated books;

[0052] S2, data preprocessing step: based on the recycled and recirculated book surface image dataset, the defects are labeled according to the defect type to obtain a labeled recycled and recirculated book surface image dataset;

[0053] S3, framework building steps: Based on the labeled recycled and recirculated book surface image dataset obtained in S2, a DD-Net recycled and recirculated book surface defect recognition model is constructed;

[0054] S4. Implement the detection step: Based on the DD-Net recycled and recirculated book surface defect recognition model achieved in S3, surface defect detection is performed on the recycled and recirculated books to be detected, and defect detection results of the recycled and recirculated books to be detected are obtained.

[0055] Furthermore, the defect types in S2 include: incomplete, stained, cracked, repaired, stamped and ink-stained.

[0056] Furthermore, the DD-Net model for surface defect recognition of recycled and recirculated books in S3 includes: a feature extraction network part, a deformable convolution operator module, a densely connected pyramid feature fusion module, and a decoupled detection head;

[0057] Among them, the feature extraction network is used for multi-level feature extraction, and the deformable convolution operator module is used to adapt to various shape defect features.

[0058] Specifically, the framework of DD-Net mainly includes four key components: first, the feature extraction network, which is used for multi-level feature extraction; second, the deformable convolution operator module, which is used to adapt to various shape defect features; third, the densely connected pyramid feature fusion module; fourth, the decoupled detection head.

[0059] 1) First, a Darknet backbone network is used to extract multi-resolution feature maps from the input recycled and recirculated book images. In order to better adapt to surface defects of different shapes, a deformable convolution operator is embedded in the backbone network to dynamically adjust the receptive field of the convolution, thereby improving the network's adaptability to defects of different shapes.

[0060] 2) Next, we applied the Dense Path Aggregation Feature Pyramid Network (Des-FPN), which generates multi-level feature maps for each input image and connects these feature maps in series through dense connections. This method helps alleviate the information loss during the propagation of shallow features and strengthens the semantic information of multi-resolution feature maps, thereby improving the detection performance of defects of different sizes.

[0061] 3) During the forward propagation of the network, we learn the effective features in the training samples and match them with the ground truth. The network parameters are optimized by calculating the loss between the output feature map and the ground truth and minimizing this loss using the back propagation algorithm.

[0062] 4) Finally, we use a decoupled detection head to perform the defect localization and classification tasks. The decoupled detection head consists of two branches: the classification branch is responsible for determining the category of the target, while the localization branch is responsible for predicting the location of the target. Multi-scale features are fused through dense skip connections to ultimately generate accurate prediction results for defect targets.

[0063] Furthermore, the specific contents of S3 include:

[0064] S301, using the Darknet backbone network and deformable convolution module to extract multi-resolution feature maps from the labeled recycled and recirculated book surface image dataset obtained in S2;

[0065] S302, inputting the multi-resolution feature map obtained in S301 into a dense path aggregation feature pyramid network to obtain a multi-level feature map for each image, and connecting the generated multi-level feature maps in series through dense connections to obtain a multi-scale feature map;

[0066] S303, while the network is in the forward propagation process, learning the effective features in the training samples, matching and calculating with the ground truth values; using the back propagation algorithm to minimize the loss between the output feature map and the ground truth value, and optimizing the network parameters;

[0067] S304, multi-scale feature maps are fused through dense skip connections; a decoupled detection head is used to perform target localization and classification of defects, and finally an accurate prediction result of the defect target is generated.

[0068] Specifically, in the detection of surface defects in recycled and recirculated books, recognizing complex geometric deformations in images is a major challenge. Traditional methods such as SIFT, ORB, DPM, and HOG are effective, but they are limited by artificially designed features and have difficulty in handling complex geometric deformations. Although convolutional neural networks (CNNs) have powerful feature extraction capabilities, their fixed geometric structure limits the modeling of geometric deformations.

[0069] To address this challenge, deformable convolution is used to replace part of the ordinary convolution in the surface defect detection of recycled and recirculated books. For small surface defects such as tiny dirt spots, holes, wrinkles or cracks, the traditional convolution kernel may lose key feature information, resulting in incorrect classification results. The deformable convolution technology can better capture these subtle features and improve the accuracy and reliability of surface defect detection.

[0070] Furthermore, the specific contents of S301 include:

[0071] Feature extraction is performed through the deformable convolution operator module. An offset domain is added to the deformable convolution operator module to learn the size of the offset. The offset domain is connected to the network as a convolution layer. The calculation formula of the deformable convolution is as follows:

[0072]

[0073] Here, for each position p0 on the output feature map y, the deformable convolution operation uses the input feature map x combined with an offset Δp n to perform feature extraction.

[0074] Specifically, the deformable convolution learns the size of the offset through an additional offset field, which is connected to the network as a convolution layer. The number of channels in the offset field is usually twice the size of the convolution kernel. For example, for a 3×3 convolution kernel, the number of channels is 18. The calculation formula of the deformable convolution is as follows:

[0075] Here, for each position p0 on the output feature map y, the deformable convolution operation uses the input feature map x combined with an offset Δp n To extract features. Offset Δp n The role of is to dynamically adjust the sampling position, thereby achieving more flexible receptive field control. This flexibility enables deformable convolution to better adapt to the shape and size changes of the target, thereby improving the feature extraction capability when dealing with complex and irregular targets.

[0076] To illustrate the advantages of deformable convolution over traditional convolution, take a 3×3 convolution kernel as an example. Figure 2 The comparison shows the difference between them. In traditional convolution, the sampling position on the target is fixed. In contrast, deformable convolution can dynamically adjust its receptive field during the calculation process, so it has better robustness and can adapt to the feature extraction needs of complex and irregular targets. This adaptability makes deformable convolution play a key role in surface defect detection of recycled and recirculated books, and can effectively identify and process defect features of different shapes and sizes.

[0077] Furthermore, in S302, the generated multi-level feature maps are connected in series through dense connections. For each feature map, it is downsampled to the scale of other feature maps with the same spatial size, and then they are merged through tensor splicing, and the generated feature map is sent to the detection head.

[0078] Specifically, throughout the convolutional neural network, the feature map gradually becomes more abstract from the bottom layer to the top layer, and the semantic information increases accordingly. Integrating the information from the highest layer back to the bottom layer can provide richer multi-scale features and enhance the detection of defects of various scales. Given that defects vary in scale and shape, the use of multi-scale features can improve the robustness and accuracy of defect detection. When detecting surface defects in recycled and recirculated books, connecting the information from the top layer with the information from the bottom layer helps to extract multi-scale features with rich semantic information, thereby more effectively detecting various types of defects. However, there are limitations to the path aggregation feature pyramid network (PAFPN) used by YOLOv8. The path from the bottom layer to the top layer in PAFPN is long, which may limit the advantages of feature reuse and multi-scale feature fusion. In addition, there are many small and similar objects in the images of surface defects of recycled and recirculated books. The coarse information from the lower layers in PAFPN may not be sufficient to effectively distinguish these objects.

[0079] To address these issues, we propose a dense feature pyramid network (DFPN) with a dense connection strategy. This method densely connects the feature maps of each scale with other feature layers from top to bottom to generate richer feature maps for the subsequent detection network. Figure 3 The structure of this densely connected pyramid feature fusion network is shown. The network enhances the transfer of image features and has better generalization ability, allowing the network to better identify different defects in old book images. Since these defects vary significantly in size and shape, the network can extract more detailed features for better classification.

[0080] In the Feature Pyramid Network (FPN), feature maps with the same spatial dimensions (i.e., width and height) generated by a specific level or module in the network are classified as feature layers of the same stage. Each feature layer corresponds to a specific stage in the network. Therefore, we use C*(C1~C5) to represent the feature maps of different sizes generated by the backbone network (CSPDarkNet-53); P*(P1~P5) to represent the feature layers integrated through operations such as lateral connections and upsampling; and N*(N1~N5) to represent the enhanced feature layer layers obtained by dense connection through the proposed DFPN.

[0081] Based on C1 to C5, DFPN first performs a 1×1 convolution operation on Ci to ensure the consistency of the number of channels in the fused feature layer, and then fuses it with the upsampled feature map of Ci+1 through element-wise addition to generate a new feature map Pi. Then, after downsampling the Pi-1 layer features, a tensor connection is made with the Pi layer feature map to obtain a preliminary fused feature map Ni. Finally, in order to further improve the expressiveness of the features, DFPN fuses the generated Ni feature map with all previous feature maps N1~Ni-1 through dense connections. Specifically, for each feature map N1~Ni-1, use the scale of other feature maps that are downsampled to the same spatial size as Ni, and then merge them through tensor concatenation to generate a richer feature map and send it to the detection head. Taking the N5 layer as an example, the specific operation process is as follows: Figure 4 shown.

[0082] Furthermore, the average precision and the mean of the average precision are used as evaluation indicators to perform surface defect detection on the recovered and recirculated books to obtain defect detection results of the recovered and recirculated books to be detected.

[0083] Specifically, in the task of surface defect detection on second-hand data, the metrics used include precision, recall, and IoU (Intersection over Union). These metrics can quantify the accuracy and coverage of the proposed model in detecting specific defects. When the IoU between the predicted bounding box and the real defect is greater than 0.5 and the confidence is higher than the threshold thconf, the detected object is judged to belong to a certain category of defects.

[0084] Among them, IoU represents the intersection-over-union ratio of the predicted bounding box and the true bounding box, that is:

[0085]

[0086] The parameter thconf is the threshold of the confidence score used to determine whether the category in the bounding box belongs to a certain defect.

[0087] Precision refers to the proportion of samples predicted to be a certain defect that are actually the defect.

[0088]

[0089] Among them, TP (True Positive) represents the number of samples correctly predicted as a defect, and FP (False Positive) represents the number of samples incorrectly predicted as a defect.

[0090] Recall refers to the proportion of samples that are correctly predicted to be a defect among all samples that are actually a defect.

[0091]

[0092] Among them, FN (False Negative) represents the number of samples that are actually a defect but are not correctly predicted.

[0093] Since precision and recall are affected by conf , and show opposite trends of change. There is a trade-off relationship between them, which is usually called negative correlation. Therefore, this study refers to the evaluation indicators of the COCO object detection task and uses the average precision (AP) to comprehensively evaluate the precision and recall of DD-NET. The average precision (AP) is defined by calculating the area under the precision-recall curve (PR curve). Finally, by averaging the AP values ​​of all categories, we can get the mean average precision (mAP). mAP is an important evaluation indicator that provides a comprehensive metric to evaluate the performance of DD-NET on multiple defect categories.

[0094] This study built and trained a baseline model (YOLOV8) and a model with specific components for comparison and analysis. On the task of ancient book surface defect detection, our model achieved an mAP of 0.467, which is 0.142 higher than the baseline model. In addition, the model has high inference efficiency. On a single GPU, our model inference time reaches 2.3ms per image.

[0095] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0096] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination of both.

[0097] In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0098] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting surface defects of recycled and recirculated books based on DD-Net architecture, characterized in that: The following steps are involved: S1, data acquisition step: acquiring surface image data of recycled and recirculated books, and constructing a surface image database of recycled and recirculated books; S2, data preprocessing step: based on the recycled and recirculated book surface image dataset, the defects are labeled according to the defect type to obtain a labeled recycled and recirculated book surface image dataset; S3, framework building steps: Based on the labeled recycled and recirculated book surface image dataset obtained in S2, a DD-Net recycled and recirculated book surface defect recognition model is constructed; S4. Implement the detection step: Based on the DD-Net recycled and recirculated book surface defect recognition model achieved in S3, surface defect detection is performed on the recycled and recirculated books to be detected, and defect detection results of the recycled and recirculated books to be detected are obtained.

2. The method for detecting surface defects of recycled and recirculated books based on DD-Net architecture according to claim 1 is characterized in that: The defect types in S2 include: incomplete, stained, cracked, repaired, stamped and ink-stained.

3. The method for detecting surface defects of recycled and recirculated books based on DD-Net architecture according to claim 1 is characterized in that: The DD-Net model for surface defect recognition of recycled and recirculated books in S3 includes: feature extraction network part, deformable convolution operator module, densely connected pyramid feature fusion module and decoupled detection head; Among them, the feature extraction network is used for multi-level feature extraction, and the deformable convolution operator module is used to adapt to various shape defect features.

4. The method for detecting surface defects of recycled and recirculated books based on DD-Net architecture according to claim 3 is characterized in that: S3 specific content includes: S301, using the Darknet backbone network and deformable convolution module to extract multi-resolution feature maps from the labeled recycled and recirculated book surface image dataset obtained in S2; S302, inputting the multi-resolution feature map obtained in S301 into a dense path aggregation feature pyramid network to obtain a multi-level feature map for each image, and connecting the generated multi-level feature maps in series through dense connections to obtain a multi-scale feature map; S303, while the network is in the forward propagation process, learning the effective features in the training samples, matching and calculating with the ground truth values; using the back propagation algorithm to minimize the loss between the output feature map and the ground truth value, and optimizing the network parameters; S304, multi-scale feature maps are fused through dense skip connections; a decoupled detection head is used to perform target localization and classification of defects, and finally an accurate prediction result of the defect target is generated.

5. The method for detecting surface defects of recycled and recirculated books based on DD-Net architecture according to claim 3 is characterized in that: The specific contents of S301 include: Feature extraction is performed through the deformable convolution operator module. An offset domain is added to the deformable convolution operator module to learn the size of the offset. The offset domain is connected to the network as a convolution layer. The calculation formula of the deformable convolution is as follows: Here, for each position p0 on the output feature map y, the deformable convolution operation uses the input feature map x combined with an offset Δp n to perform feature extraction.

6. The method for detecting surface defects of recycled and recirculated books based on DD-Net architecture according to claim 5 is characterized in that: In S302, the generated multi-level feature maps are connected in series through dense connections. For each feature map, it is downsampled to the scale of other feature maps with the same spatial size, and then they are merged through tensor splicing, and the generated feature map is sent to the detection head.

7. The method for detecting surface defects of recycled and recirculated books based on DD-Net architecture according to claim 1 is characterized in that: Taking the average precision and the mean of the average precision as evaluation indicators, surface defect detection is carried out on the recovered and recirculated books to obtain the defect detection results of the recovered and recirculated books to be detected.

Citation Information

Patent Citations

  • Glass defect detection method and device based on deformable convolution and feature fusion

    CN114170230A

  • Automatic Interpretation Method for Winter Wheat Based on Deformable Fully Convolutional Neural Network

    US20210272266A1

Cited By

  • Quality inspection method for plastic lace

    CN119850534A