Rope and knot detection method and system based on deep learning

Abstract

The invention relates to the technical field of image processing, and provides a rope and knot detection method and system based on deep learning, and the method comprises the steps: obtaining a region image of a rope and a knot, analyzing the material of the knot in the region image, generating a corresponding image compensation parameter, compensating the region image, and obtaining a compensated region image; and inputting the compensated regional image into a preset multi-material detection model for defect detection to obtain a first detection result, inputting the compensated regional image into a preset single-material detection model for defect detection to obtain a second detection result, and carrying out conjoint analysis on the second detection result and the first detection result to generate a detection result. Through automatic analysis and comparison of a deep learning model in combination with image compensation and a multiple detection model, it is ensured that rope and knot defects can be effectively recognized, and the problems that in the face of ropes of different materials, knots of different structures and complex background interference, an existing scheme is poor in detection stability and low in accuracy are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a rope and knot detection method and system based on deep learning. Background Art

[0002] As consumers' demands for product quality continue to rise, production efficiency and product quality control have become increasingly important, especially in the packaging industry. In the production of printed tote bags and other packaging products, rope knots are a crucial functional component, widely used for bag closures and handles. The quality of these knots directly impacts the safety and aesthetics of packaging. Therefore, efficiently and accurately testing knot quality has become a major challenge in the industry.

[0003] Among the relevant technical means, the detection of ropes and knots usually relies on manual visual inspection or traditional rule-based machine vision methods, which can detect the number of ropes and knots and solve the problems of insufficient ropes, insufficient knots, and excessive knots in printed handbags.

[0004] While manual inspection and traditional machine vision methods can detect the number of ropes and knots in the aforementioned technical solution, in practice, manual inspection is not only inefficient but also prone to missed detections or misjudgments. Traditional machine vision methods are also prone to misidentification when the bag is wrinkled or has interference from printed patterns. Therefore, when faced with a variety of rope materials, knot structures, and complex background interference, existing methods often fail to provide stable and accurate detection results, resulting in poor detection stability and low accuracy. Summary of the Invention

[0005] In order to improve the problems of poor detection stability and low accuracy in existing solutions when facing ropes of different materials, knots of different structures and complex background interference, the present application provides a rope and knot detection method and system based on deep learning.

[0006] The present invention provides a rope and knot detection method based on deep learning, which is applied to rope and knot detection of printed handbags, comprising: obtaining regional images of ropes and knots, analyzing the knot materials in the regional images to obtain the knot material type, generating corresponding image compensation parameters according to the knot material type, compensating the regional image using the image compensation parameters to obtain a compensated regional image; inputting the compensated regional image into a preset multi-material detection model for defect detection to obtain a first detection result, and inputting the compensated regional image into a preset single-material detection model for defect detection to obtain a second detection result; jointly analyzing the first detection result and the second detection result, if the first detection result and the second detection result are inconsistent, the current printed handbag is rejected and the corresponding regional image is archived; if the first detection result and the second detection result are consistent, analyzing whether the detection of the first detection result and the second detection result is qualified, if not qualified, the current printed handbag is rejected and the corresponding regional image is archived; if qualified, the corresponding printed handbag is passed to the next process through detection.

[0007] As a preferred embodiment, the steps of obtaining a regional image of the rope and knot, analyzing the knot material in the regional image to obtain the knot material type, and generating corresponding image compensation parameters according to the knot material type include: using a camera to capture the knot to obtain a regional image, preprocessing the regional image to obtain a clear regional image, inputting the clear regional image into a preset deep learning model to obtain the knot material type; matching the knot material type with a preset image compensation strategy to obtain image compensation parameters corresponding to the knot material type.

[0008] As a preferred embodiment, the step of preprocessing the regional image to obtain a clear regional image, inputting the clear regional image into a preset deep learning model to obtain the knot material type includes: denoising the regional image to obtain a denoised regional image, adjusting the brightness and contrast of the denoised regional image to obtain a clear regional image; extracting texture features from the clear regional image according to a grayscale co-occurrence matrix to obtain texture features of the knot area, extracting color features from the clear regional image to identify the color features of the knot material; and inputting the texture features and the color features into a preset deep learning model for material identification to obtain the knot material type.

[0009] As a preferred solution, the step of compensating the regional image using the image compensation parameters to obtain a compensated regional image includes: inputting the image compensation parameters into a preset compensation module, adjusting the color and brightness of the regional image based on the image compensation parameters through the compensation module to obtain an adjusted regional image, sharpening the adjusted regional image using an image enhancement algorithm to obtain a sharpened regional image; extracting knot area features at different scales on the sharpened regional image to obtain a feature response map, and inputting the feature response map into a preset Gaussian filter module for filtering and denoising to obtain a compensated regional image.

[0010] As a preferred solution, the step of performing knot area feature extraction at different scales on the sharpened regional image to obtain a feature response map, inputting the feature response map into a preset Gaussian filter module for filtering and denoising to obtain a compensated regional image includes: performing multi-scale analysis on the sharpened regional image using an image pyramid algorithm to obtain a multi-scale image set, extracting edge texture features and structural features in images of each scale according to the multi-scale image set, and constructing a scale-invariant feature response map based on the edge texture features and the structural features; performing feature aggregation on the feature response map to obtain a fused knot candidate feature map, applying an edge attention enhancement mechanism to enhance the fused knot candidate feature map to obtain an enhanced feature map, and inputting the enhanced feature map into a preset Gaussian filter module for filtering and denoising to obtain a compensated regional image.

[0011] As a preferred solution, the step of inputting the compensated regional image into a preset multi-material detection model for defect detection to obtain a first detection result includes: inputting the compensated regional image into a preset multi-material detection model, and in the multi-material detection model, using a multi-scale convolutional neural network to extract features of the compensated regional image to obtain a texture feature map; applying a morphological segmentation algorithm to extract a target edge region and a potential knot region of the texture feature map, generating a rope contour set based on the target edge region, and inputting the rope contour set into a rope recognition module constructed based on curve fitting and topological analysis to calculate the compensated regional image. The present invention relates to a method for detecting the number of ropes in a domain image; generating a set of knot candidate regions according to the potential knot region, inputting the set of knot candidate regions into a target detection sub-model based on YOLOv8, and obtaining the number of knots and knot positions in the compensated regional image; classifying and clustering the distribution of the number of knots and the knot positions on the number of ropes through a statistical morphological analysis algorithm, and obtaining a spatial distribution feature vector of the knots on each rope; using a judgment sub-model based on a decision tree to perform matching analysis on the spatial distribution feature vector, and obtaining a first judgment result; comparing the first judgment result with a preset rope knot structure standard, and generating a first detection result.

[0012] As a preferred solution, the step of inputting the compensated area image into a preset single material detection model for defect detection to obtain a second detection result includes: inputting the compensated area image into a single material detection model, and in the single material detection model, using a convolutional neural network to extract image partition features of the compensated area image to obtain an edge enhancement map and a target response heat map; inputting the edge enhancement map into an image segmentation module based on an adaptive edge detection algorithm to extract the rope area, and using Hough transform to identify the rope trajectory of the rope area to obtain the number of ropes in the compensated area image; and inputting the target response heat map into an image segmentation module based on a Faster Image Detection Algorithm to extract the rope area. The knot detection sub-model of R-CNN detects and marks the knot area in the compensated regional image to obtain the number of knots and the distribution of knot spatial coordinates; the positional relationship between the number of knots and the distribution of knot spatial coordinates on the number of ropes is modeled through a spatial relationship graph construction algorithm to obtain a knot symmetry index, and a symmetry discrimination model based on cluster analysis is applied to perform cluster analysis and group matching on the knot symmetry index, extract knot combinations that conform to the knot logical structure, compare the knot combinations with the rope knot structure standard for logical consistency, and output a second detection result.

[0013] The present application also provides a rope and knot detection system based on deep learning, which is applied to rope and knot detection of printed handbags, comprising: an acquisition module, configured to acquire regional images of ropes and knots, analyze the knot material in the regional image to obtain the knot material type, generate corresponding image compensation parameters according to the knot material type, and compensate the regional image using the image compensation parameters to obtain a compensated regional image; a detection module, configured to input the compensated regional image into a preset multi-material detection model for defect detection to obtain a first detection result, and input the compensated regional image into a preset single-material detection model for defect detection to obtain a second detection result; an analysis module, configured to jointly analyze the first detection result and the second detection result; if the first detection result and the second detection result are inconsistent, the current printed handbag is rejected and the corresponding regional image is archived; if the first detection result and the second detection result are consistent, the first detection result and the second detection result are analyzed to determine whether the detection is qualified; if not, the current printed handbag is rejected and the corresponding regional image is archived; if qualified, the corresponding printed handbag is passed to the next process after detection.

[0014] Compared with existing technologies, this application has the following advantages: stable and highly accurate detection. By combining a deep learning model with image compensation and automated analysis and comparison of multiple detection models, it can efficiently and accurately detect and classify rope and knot defects, significantly improving detection accuracy and production efficiency. This ensures that rope and knot defects can be effectively identified and processed in complex situations with different scenes and materials, and improves the poor detection stability and low accuracy of existing solutions when faced with ropes of different materials, knots of different structures, and complex background interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] The structures, proportions, sizes, etc. depicted in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not intended to limit the conditions under which the present invention can be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportional relationships, or adjustments in size should still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and objectives that can be achieved by the present invention.

[0017] Figure 1 1 is a flowchart of a rope and knot detection method based on deep learning provided by an embodiment of the present invention; Figure 2 This is a schematic block diagram of the structure of a rope and knot detection system based on deep learning provided by an embodiment of the present invention.

[0018] Description of reference numerals: 10. Rope and knot detection system based on deep learning; 11. Acquisition module; 12. Detection module; 13. Analysis module. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0021] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings and through specific implementation methods.

[0024] Example 1: like Figure 1 As shown, a rope and knot detection method and system based on deep learning provided by an embodiment of the present application includes steps S100 to S300.

[0025] Example 1: The present application also provides a rope and knot detection method based on deep learning, which is applied to rope and knot detection of printed handbags, including steps S100 to S300.

[0026] Step S100: Acquire a regional image of the rope and knot, analyze the knot material in the regional image to obtain the knot material type, generate corresponding image compensation parameters based on the knot material type, and compensate the regional image using the image compensation parameters to obtain a compensated regional image.

[0027] In this step, a camera (such as a CCD camera) captures real-time images of the area containing the rope and knot on the printed bag. Specifically, a high-resolution camera captures the knot area, converts the image to grayscale, and performs denoising to ensure image quality suitable for subsequent processing. Next, an image analysis algorithm is used to classify the image and analyze the knot material type. To this end, a pretrained deep convolutional neural network (such as ResNet50) is used to classify the knot material and output the material type. Based on the obtained knot material type (e.g., kraft paper, glossy plastic, fabric, etc.), corresponding image compensation parameters are generated. These compensation parameters are generated by combining a deep learning model with a pre-set image compensation strategy to optimize color, brightness, and contrast for images of different materials.

[0028] For example, for knots on reflective materials, a higher brightness enhancement strategy is applied to reduce the interference caused by reflections, while for dark materials, the contrast needs to be adjusted to highlight the edge information of the knot. Through this compensation process, the resulting regional image will be able to more clearly display the details of the knot area.

[0029] Step S200: Input the compensated area image into a preset multi-material detection model for defect detection to obtain a first detection result, and input the compensated area image into a preset single-material detection model for defect detection to obtain a second detection result.

[0030] In this step, the compensated regional images are fed into pre-set multi-material and single-material detection models for defect detection. Specifically, for the multi-material model, a deep convolutional neural network (such as VGG16 or ResNet50) is used to extract multi-scale features from the image for material differentiation and defect detection. The model is trained using a multi-material dataset containing samples of different knot and buckle materials. By learning the characteristic information from this data, the model can effectively identify rope and knot defects on various materials in real-world applications.

[0031] For single-material models, a training model specifically tailored to a specific material is used to optimize the network structure to enhance knot defect detection on that material. For example, a single-material model can be trained on kraft paper, focusing on recognizing its texture and morphological characteristics, enabling accurate detection of defects in this type of knot (e.g., missing knots or cords). These two detection models output primary and secondary detection results, respectively, for subsequent joint analysis.

[0032] Step S300: Jointly analyze the first detection result and the second detection result. If the first detection result and the second detection result are inconsistent, the current printed handbag is discarded and the corresponding area image is archived. If the first detection result and the second detection result are consistent, analyze whether the first detection result and the second detection result are qualified. If not, the current printed handbag is discarded and the corresponding area image is archived. If qualified, the corresponding printed handbag is passed the inspection and flows into the next process.

[0033] In this step, the quality of the current knot is determined by jointly analyzing the first and second test results. If the test results from the two models are consistent, further analysis is performed to determine its quality. If the test result is unsatisfactory, the current printed bag is rejected, and the corresponding area image is archived for subsequent traceability. Specifically, if both models determine that the knot is defective, the bag is immediately rejected. If the model outputs are consistent and qualified, the bag proceeds to the next step.

[0034] For example, when comparing inspection results, if the multi-material model detects a knot with missing rope, while the single-material model shows the knot intact, manual inspection and correction are required. This "human-machine integration" approach helps improve overall inspection accuracy and efficiency, especially in complex backgrounds and with varying materials.

[0035] In this embodiment, a camera captures regional images of the ropes and knots on a printed handbag. The captured regional images are preprocessed to analyze the material type of the knots. Image compensation parameters are generated based on the material type, and these compensation parameters are used to compensate the regional images, resulting in a compensated regional image. The compensated regional images are then input into a pre-set multi-material detection model and a single-material detection model for defect detection, yielding a first detection result and a second detection result. These two detection results are then jointly analyzed. If the results disagree, the bag is deemed unqualified and the corresponding regional image is archived. If the results agree, the acceptance of the test results is further analyzed. If unqualified, the bag is rejected and the image is archived. If qualified, the bag is sent to the next process. By combining the multi-material detection model with the single-material detection model, efficient and accurate defect detection can be performed on ropes and knots of different materials. This image compensation technology can significantly improve image quality and enhance the ability to recognize knot details, especially against complex backgrounds. Compared with traditional manual visual inspection and traditional machine vision methods, this solution can reduce human intervention and the risk of missed detection and misjudgment, and has a high degree of automation, effectively improving detection efficiency and accuracy, ensuring the stability and reliability of knot quality during the production process, and improving the existing solutions' problems of poor detection stability and low accuracy when facing ropes of different materials, knots of different structures, and complex background interference.

[0036] Example 2: In step S100, a regional image of the rope and knot is acquired, and the knot material in the regional image is analyzed to obtain the knot material type. The steps of generating corresponding image compensation parameters according to the knot material type specifically include: The knots are captured using a camera to obtain a regional image, which is preprocessed to obtain a clear regional image. The clear regional image is input into a preset deep learning model to obtain the knot material type.

[0037] High-resolution cameras (such as CCD cameras) capture real-time images of the knotted areas on handbags on the production line. The camera is selected based on image clarity and capture speed requirements to ensure that the images clearly capture the knot details. An image preprocessing module performs denoising and halo removal to enhance image clarity and facilitate subsequent analysis. The processed, clear images are then fed into a pre-defined deep learning model (such as a ResNet50 or VGG16 network). Trained on a large dataset of knotted material types, this network accurately identifies the knotted material type (e.g., kraft paper, glossy plastic, fabric, etc.) in the image and outputs a material classification result.

[0038] For example, when a camera captures an image of a kraft paper knot, the deep learning model automatically identifies the image as kraft paper and outputs the corresponding material label. The accuracy of this step lays the foundation for subsequent image compensation and defect detection.

[0039] The knot material type is matched with a pre-set image compensation strategy to obtain image compensation parameters corresponding to the knot material type.

[0040] Based on the knot characteristics of different materials (such as color, texture, and reflectivity), different image compensation parameters are set for each material. For example, for kraft paper knots, the image contrast and brightness need to be enhanced to better highlight the knot details; while for glossy plastic materials, the image reflectivity needs to be adjusted to reduce the distracting effects of reflections. Based on the knot material type output by the deep learning model, the system automatically selects compensation parameters that match the material and applies them to the image for compensation.

[0041] For example, for an image of a knot on kraft paper, the compensation strategy is to increase the image's brightness and contrast and apply appropriate color correction to make the knot more visible. Through image compensation, image quality is optimized, and knot details are more distinct, providing a high-quality input image for subsequent defect detection.

[0042] The steps of preprocessing the regional image to obtain a clear regional image, and inputting the clear regional image into a preset deep learning model to obtain the knot material type specifically include: The regional image is subjected to denoising processing to obtain a denoised regional image, and the brightness and contrast of the denoised regional image are adjusted to obtain a clear regional image.

[0043] The image is processed using denoising algorithms (such as Gaussian filtering and bilateral filtering) to remove noise, particularly noise caused by background noise, lighting variations, and reflections. The denoised image preserves the knot's edge information, making subsequent image processing more accurate. Subsequently, contrast enhancement algorithms (such as histogram equalization) are used to adjust the image's brightness and contrast, further enhancing the contrast of the knotted area and highlighting its details, facilitating subsequent material analysis.

[0044] For example, for an image of a knot against a complex background, after using Gaussian filtering to remove background noise, brightness enhancement and contrast adjustment are performed. The knot area in the image will be more conspicuous and the edges will be clearer, which can effectively reduce the recognition difficulty caused by the complexity of the background.

[0045] The texture features of the clear regional image are extracted according to the gray-level co-occurrence matrix to obtain the texture features of the knot region. The color features of the clear regional image are extracted to identify the color features of the knot material.

[0046] The gray-level co-occurrence matrix (GLCM) is used to extract the texture features of the knot region from the clear region image. The GLCM describes the grayscale relationship between pixel pairs in an image and can effectively extract texture features, such as roughness and smoothness, which are crucial for identifying knot materials. Furthermore, color features can be extracted using color spaces (such as HSV or Lab) to further aid in identifying the color characteristics of the knot material. For example, kraft paper knots typically appear brown, while glossy plastics appear glossy white or transparent. Using the GLCM to extract texture features of the knot region in the image effectively identifies the texture characteristics of kraft paper knots, and further distinguishes the knot material through color feature extraction. This approach allows deep learning models to more accurately identify the knot material, avoiding misidentification caused by background interference by traditional methods.

[0047] The texture features and color features are input into the preset deep learning model for material recognition to obtain the knot material type.

[0048] By inputting the extracted texture and color features into a pre-set deep learning model (such as a convolutional neural network (CNN)), the network extracts deeper features from texture and color through multiple layers of convolution and pooling operations to perform material classification. The deep learning model learns the characteristics of materials based on the training data and can automatically identify the type of knot material in the image.

[0049] For example, when the input image is of kraft paper, the deep learning model can accurately identify the material based on texture and color features, and output a knot material label of "kraft paper." This process automatically and accurately classifies the material of each knot in the image, providing key parameters for subsequent image compensation and defect detection.

[0050] In step S100, the step of compensating the regional image using the image compensation parameter to obtain the compensated regional image specifically includes: The image compensation parameters are input into a preset compensation module. The compensation module adjusts the color and brightness of the regional image based on the image compensation parameters, thereby enhancing the contrast of the image and highlighting the details of the rope and knot area to obtain an adjusted regional image. The adjusted regional image is sharpened using an image enhancement algorithm to strengthen the edge features in the image, making the boundaries of the rope and knot area more clearly discernible, and obtaining a sharpened regional image.

[0051] In this step, the compensation module first receives image compensation parameters, which are dynamically generated based on the material type of the knot in the image (for example, for kraft paper, plastic, etc.). The compensation module then performs color adjustments (such as color enhancement and color temperature correction) and brightness enhancement (such as contrast enhancement) on the image to highlight the rope and knot details. Subsequently, an image enhancement algorithm (such as Unsharp Masking or Laplace sharpening) is applied to the adjusted area image to enhance the image's edge features, particularly the knot's outline and details, thereby making the knot area's boundaries more clearly visible. Ultimately, the sharpened area image displays more distinct knot features, providing a clear image input for subsequent defect detection.

[0052] For example, for an image printed with a kraft paper knot, the compensation module enhances the image's contrast and color saturation, making the details of the knot area more prominent. At the same time, through the image sharpening algorithm, the knot edge is further enhanced, making it more clearly distinguishable from the background, thereby improving the subsequent model's ability to detect rope and knot defects.

[0053] The knot area features at different scales are extracted from the sharpened regional image to ensure that the details of the rope and knot can be clearly presented at different viewing angles, and a feature response map is obtained. The feature response map is input into the preset Gaussian filter module for filtering and denoising, so that the rope and knot areas can still be accurately identified in complex backgrounds, and a compensated regional image is obtained.

[0054] Multi-scale feature extraction is performed on the sharpened region image. Specifically, the image is analyzed using layers of images at different resolutions (e.g., using an image pyramid method). Features of the knot region are extracted at each layer, ensuring that knot details are clearly captured from different viewing angles. These features include information such as the knot's edges, texture, and morphology. After feature extraction, a feature response map is generated, which displays the distribution of feature intensities across the image, particularly those in the knot region.

[0055] Next, the characteristic response spectrum is fed into the Gaussian filtering module for denoising. Gaussian filtering is a common denoising technique that applies a Gaussian kernel function to the image to smooth it, removing noise while preserving edge information. After filtering, the details of the knotted area in the image are preserved, effectively suppressing interference from complex backgrounds, ensuring that subsequent defect detection can accurately identify the knotted area.

[0056] For example, for an image with a complex background, after multi-scale feature extraction and Gaussian filtering, the knotted area in the image is still clearly visible, while the complex texture and cluttered information in the background are effectively removed, allowing the detection model to perform defect judgment on a clean image.

[0057] The steps of extracting knot region features at different scales from the sharpened region image to ensure that the details of the rope and knot can be clearly presented at different viewing angles, obtaining a feature response map, and inputting the feature response map into a preset Gaussian filter module for filtering and denoising, so that the rope and knot region can still be accurately identified under complex backgrounds, and obtaining a compensated region image, specifically include: The image pyramid algorithm is used to perform multi-scale analysis on the sharpened regional image to obtain a multi-scale image set. The edge texture features and structural features in images of each scale are extracted based on the multi-scale image set, and a scale-invariant feature response map is constructed based on the edge texture features and structural features.

[0058] Using an image pyramid algorithm, the sharpened regional image is decomposed into multiple scales. Each scale level contains different details and global information. Low-scale images emphasize the overall structure of the image, while high-scale images highlight details. By extracting edge texture features (such as Sobel edge detection) and structural features (such as extracting lines and curves through Hough transform) from images at different scales, these features help construct a scale-invariant feature response map. This map retains stable feature information at different image scales, ensuring that the system can accurately identify knots even if their position or size in the image changes.

[0059] For example, when performing multi-scale image analysis using an image pyramid algorithm, we can identify the overall structure of a knot in low-scale images, while high-scale images clearly display image details, such as tiny wrinkles or details within the knot. These features are crucial for subsequent defect detection.

[0060] Feature aggregation is performed on the feature response map to improve the ability to express structural details in low-contrast scenes, and the fused knot candidate feature map is obtained. The edge attention enhancement mechanism is applied to enhance the fused knot candidate feature map to strengthen the boundary clarity and geometric contour accuracy of the knot area in the image to obtain the enhanced feature map. The enhanced feature map is input into the preset Gaussian filtering module for filtering and denoising. A filtering algorithm based on adaptive Gaussian kernel parameter estimation is used to automatically match the optimal smoothing parameters for regions of different scales, and edge-preserving filtering processing is performed to suppress background noise, retain key edges, and obtain the compensated regional image.

[0061] Through feature aggregation, edge texture features and structural features from images of different scales are combined to enhance the ability to express structural details of knot regions in low-contrast scenes. The aggregated knot candidate feature map contains rich feature information extracted from different scales, enabling more accurate capture of knot details.

[0062] The fused knot candidate feature map is then enhanced using an edge attention enhancement mechanism (such as one based on deep learning). This mechanism automatically focuses on image edges, particularly knot boundaries, thereby enhancing the clarity and geometric accuracy of the knot region. The enhanced feature map is then fed into a Gaussian filter module for denoising. Using an adaptive Gaussian kernel parameter estimation algorithm, the filter automatically matches the most appropriate filter parameters based on the noise characteristics of different image regions. This process preserves the edge information of the knot region while suppressing background noise.

[0063] For example, when processing images with complex backgrounds, through the enhancement of feature aggregation and edge attention mechanism, the boundaries of the knot area will be clearer, while the cluttered information in the background will be effectively suppressed, so that the final image can clearly present the knot area, providing accurate input for subsequent defect detection.

[0064] In step S200, the compensated regional image is input into a preset multi-material detection model for defect detection to obtain a first detection result, which specifically includes: The compensated area image is input into a preset multi-material detection model. In the multi-material detection model, a multi-scale convolutional neural network is used to extract features of the compensated area image to obtain a texture feature map.

[0065] The compensated area image is fed into a pre-set multi-material detection model. This model extracts image features based on a multi-scale convolutional neural network (such as ResNet50 or VGG16). By processing images at multiple scales, multi-scale convolutional neural networks can capture features at different levels and details within the image, particularly the texture, structure, and morphology of the knot area. During this process, the network automatically extracts texture features from the image and generates a texture feature map. This texture feature map contains texture information for every pixel in the image, reflecting detailed information about the knot material.

[0066] For example, when fed an image containing a fabric knot, a multi-scale convolutional neural network can extract local texture features (such as the tiny wrinkles and lines on the knot's surface) and global structural features (such as the knot's overall shape) through multiple layers of convolution kernels, generating a texture feature map. This texture feature map provides a powerful basis for subsequent defect detection and material identification.

[0067] The morphological segmentation algorithm is applied to extract the target edge area and potential knot area of ​​the texture feature map. A rope contour set is generated based on the target edge area. The rope contour set is input into the rope recognition module based on curve fitting and topological analysis to calculate the number of ropes in the compensated regional image.

[0068] Morphological operations (such as dilation and erosion) are used to process the texture feature map to extract target edge regions and potential knot regions within the image. This step aims to effectively distinguish knot regions from the background and avoid misidentification due to background complexity. Through edge detection and morphological segmentation, the outline of the knot can be accurately extracted, and these edge regions can be used as candidate regions for subsequent analysis.

[0069] After extracting the target edge region, curve fitting and topological analysis methods are used to construct a set of rope profiles. This set of profiles represents the morphological structure of the ropes in the image. This profile set is then fed into the rope recognition module to calculate the number of ropes in the compensated region image. This recognition module accurately calculates the number of ropes based on the rope morphological characteristics in the image.

[0070] For example, if there are three ropes in an image, the morphological segmentation and rope recognition modules can accurately identify the outlines of the three ropes and output the number of ropes as 3. This step provides basic information for subsequent knot detection and defect judgment.

[0071] A set of knot candidate regions is generated based on the potential knot area, and the knot candidate region set is input into the object detection sub-model based on YOLOv8 to obtain the number and location of knots in the compensated regional image.

[0072] These candidate regions are then fed into the YOLOv8 object detection sub-model for further defect detection. YOLOv8 is a deep learning-based object detection algorithm that performs multi-layer convolutional feature extraction and region extraction on the input image. This allows it to accurately identify knot regions within the image and simultaneously predict their number and location. The YOLOv8 model has learned how to identify different types of knots during training, enabling it to accurately detect the number and location of knots in an image.

[0073] For example, when an input image contains four knots, the YOLOv8 model, through object detection, not only locates the precise location of each knot, but also counts the total number of knots and outputs the coordinates of each knot. This process accurately identifies the number and location of knots in the compensated area image and provides feedback to the system.

[0074] The distribution of knot numbers and knot positions on the ropes was classified and clustered using a statistical morphological analysis algorithm to obtain the spatial distribution feature vector of knots on each rope.

[0075] After obtaining the number and location of knots, the distribution of knots on the rope is further classified and clustered using a statistical morphological analysis algorithm. Specifically, a clustering algorithm (such as K-means or DBSCAN) is used to cluster the spatial locations of the knots, generating a spatial distribution feature vector for each rope. This feature vector represents the arrangement of the knots on the rope and can be used to determine whether the knots meet pre-defined structural standards (e.g., each rope should contain two knots, and the knot positions must conform to a logical structure).

[0076] For example, suppose there are two ropes in the image and the distribution of knots does not meet the requirements (for example, there are too many or too few knots on one rope). The algorithm will identify this non-compliant pattern based on the spatial distribution feature vector of the knots and perform subsequent processing.

[0077] A decision tree-based judgment sub-model is used to match and analyze the spatially distributed feature vectors to determine whether a knot pattern that conforms to the logical structure exists. This results in a first judgment result, which is then compared with a preset rope knot structure standard to generate a first detection result. The rope knot structure standard requires two ropes and four knots, with each two knots located at the beginning and end of the same rope.

[0078] By inputting the spatial distribution feature vectors of the knots on each rope into a decision tree-based judgment sub-model, the model analyzes whether these features meet the preset knot structure standards. Specifically, the decision tree model matches the spatial distribution feature vectors with the knot structure patterns in historical data to determine whether the knot pattern in the current image meets the preset standards. For example, the standard stipulates that every two ropes should have four knots, and the knots should be located at the beginning and end of the same rope. If the model determines that the knot pattern meets this structure, it generates a qualified test result; otherwise, it outputs a failed result.

[0079] For example, knot structure standards require that two ropes each have two knots, and the knots must be evenly distributed at both ends of the rope. If the knot distribution does not meet the standard (for example, too many or too few knots), the decision tree model will determine that the rope is unqualified and mark it as defective.

[0080] In step S200, the compensated regional image is input into a preset single-material detection model for defect detection to obtain a second detection result, specifically including: The compensated area image is input into the single material detection model. In the single material detection model, a convolutional neural network is used to extract image partition features of the compensated area image to obtain an edge enhancement map and a target response heat map.

[0081] Specifically, the compensated area image is analyzed using a single-material detection model, which uses a convolutional neural network (CNN) such as VGG16 or ResNet50 to extract image partition features. CNNs extract features from the input image through convolutional layers, gradually identifying low-level features (such as edges, corners, and texture) as well as high-level features (such as knot shape and structure). This process extracts feature information layer by layer to generate an edge enhancement map and a target response heat map. The edge enhancement map is used to highlight the edges of knots and ropes in the image, while the target response heat map shows the intensity of the response of each part of the image to the target (knot), helping the system identify potential knot areas.

[0082] For example, for an image of a knot made of kraft paper, the convolutional neural network will extract the texture and morphological features in the image and generate a clear edge enhancement image, making the difference between the knot details and the background more obvious. At the same time, it will generate a target response heat map to highlight the knot area.

[0083] The edge enhancement image is input into the image segmentation module based on the adaptive edge detection algorithm to extract the rope area. The rope trajectory of the rope area is identified using Hough transform to obtain the number of ropes in the compensated area image.

[0084] Using an adaptive edge detection algorithm (such as Canny edge detection), the edge-enhanced image is fed into the image segmentation module to accurately extract the rope region within the image. This adaptive edge detection algorithm dynamically adjusts the edge detection threshold based on local changes in the image, adapting to varying material, lighting, and background conditions. The extracted rope region is then fed into the Hough transform algorithm for trajectory identification. The Hough transform is a classic morphological algorithm that effectively identifies straight and curved lines in images and is used in this step to detect the rope's path.

[0085] The system can accurately identify the number and location of ropes in an image using the Hough transform, which calculates the projection of each pixel in the image to find the intersection of straight lines or curves, thereby locating the position of the ropes and counting the number of ropes.

[0086] For example, for an image, the Hough transform detects that there are three ropes in the image, and the system outputs that the number of ropes is 3, providing basic information for subsequent knot detection.

[0087] The target response heat map is input into the knot detection sub-model based on Faster R-CNN. The knot areas in the compensated regional image are detected and labeled to obtain the number of knots and the spatial coordinate distribution of knots.

[0088] By inputting the target response heatmap into the knot detection sub-model based on Faster R-CNN, the system can automatically detect and annotate knot regions in the compensated region image. Faster R-CNN is a highly efficient object detection algorithm that combines a region proposal network (RPN) and a convolutional neural network to automatically locate and identify knot regions in an image. The Faster R-CNN model first generates potential knot region candidate boxes using the RPN. Then, using a classification and regression network, it accurately determines whether each candidate box contains a knot and outputs the knot's location coordinates and category label.

[0089] For example, after inputting the compensated region image, Faster R-CNN can detect four knots in the image and give the bounding box coordinates of each knot, and finally output the number of each knot (4) and their position coordinates in the image.

[0090] A spatial relationship graph construction algorithm is used to model the relationship between the number of knots and their spatial coordinates across the number of ropes, generating a knot symmetry index. A symmetry discrimination model based on cluster analysis is then applied to cluster analysis and group matching of the knot symmetry index. Knot combinations that conform to the knot's logical structure are extracted and compared for logical consistency with the rope knot structure standard, outputting a second test result. The rope knot structure standard requires two ropes and four knots, with each pair of knots located at the beginning and end of a rope, respectively.

[0091] The spatial relationship graph construction algorithm uses the number of detected knots and their spatial coordinates to create a spatial relationship graph of the knots on the rope. This graph shows the arrangement of the knots on the rope. Based on this spatial relationship, the system calculates the knot's symmetry index. This symmetry index reflects the balance of the knots' distribution at both ends of the rope, indicating whether they meet pre-set structural standards.

[0092] Next, a cluster analysis-based symmetry discrimination model is applied to cluster analysis and group matching of the knot symmetry indicators. This step classifies the knot symmetry indicators into different categories to determine whether the knot arrangement meets the predetermined structural requirements. Through logical consistency comparison, the extracted knot combinations are matched with the preset rope knot structure standard. For example, the standard stipulates that each rope must contain two knots, one at the beginning and one at the end. If the detected knot distribution does not meet this structural standard, the system will output a failure result.

[0093] For example, if the system detects that the knot position in the image does not meet the standard (for example, the knots are unevenly distributed at both ends of the rope), it will analyze it through the spatial relationship diagram and symmetry discrimination model, and ultimately judge it as unqualified, generating a second detection result as a defect.

[0094] In this embodiment, a highly efficient and accurate knot detection method is provided by combining deep learning and image processing technologies. First, a camera captures regional images of the rope and knot. Image preprocessing and deep learning models are then used to perform material analysis, automatically identifying the knot material type. Based on the material type, a corresponding image compensation strategy is automatically selected to adjust image parameters such as color, brightness, and contrast to further enhance the details of the knot area and provide a clear input image for subsequent inspection. The compensated regional images are then input into multi-material and single-material detection models for defect detection, ensuring stable operation across a variety of material environments. Through statistical and cluster analysis, the system accurately determines the spatial distribution of knots and compares knot patterns based on pre-set structural criteria, outputting a qualified or unqualified detection result. Advanced algorithms, such as YOLOv8 for knot detection and Faster R-CNN for object detection, are used in conjunction with multi-scale convolutional neural networks and Gaussian filtering to optimize image processing and feature extraction, significantly improving detection accuracy and robustness. Ultimately, this method reduces manual intervention through automated inspection, improves production efficiency, and ensures that knot quality meets industrial production requirements.

[0095] Example 3: like Figure 2 As shown, the present application also provides a rope and knot detection system 10 based on deep learning, including an acquisition module 11, a detection module 12 and an analysis module 13.

[0096] The acquisition module 11 is mainly used to obtain the regional image of the rope and knot, and analyze the knot material in the regional image to obtain the knot material type, generate corresponding image compensation parameters according to the knot material type, and use the image compensation parameters to compensate the regional image to obtain the compensated regional image.

[0097] The detection module 12 is mainly used to input the compensated area image into the preset multi-material detection model for defect detection to obtain a first detection result, and input the compensated area image into the preset single-material detection model for defect detection to obtain a second detection result.

[0098] The analysis module 13 is mainly used to jointly analyze the first detection result and the second detection result. If the first detection result and the second detection result are inconsistent, the current printed handbag is discarded and the corresponding area image is archived; if the first detection result and the second detection result are consistent, the first detection result and the second detection result are analyzed to see whether they are qualified. If they are unqualified, the current printed handbag is discarded and the corresponding area image is archived; if they are qualified, the corresponding printed handbag is passed to the next process after detection.

[0099] In this embodiment, an efficient and accurate knot detection system is provided by combining deep learning and automated detection technology. The acquisition module 11 first captures regional images of ropes and knots, analyzes the knot material in the image, automatically determines the material type, and generates corresponding image compensation parameters based on different material types, thereby enhancing image quality and providing clear input for subsequent detection. Next, the detection module 12 inputs the compensated regional images into a preset multi-material detection model and a single-material detection model for defect detection, obtaining two sets of detection results respectively. In the analysis module 13, the system jointly analyzes the two sets of detection results. If the results are consistent, the system proceeds to the next step. If the results are inconsistent, the system performs a material removal process and archives the regional image. Through this process, the system can efficiently and accurately detect defects in ropes and knots, ensure quality control during the production process, reduce manual intervention and errors, and improve production efficiency and stability.

[0100] It should be noted that, those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the above-described system and each module can refer to the corresponding processes in the aforementioned embodiment 1 and will not be repeated here.

[0101] The structures, proportions, sizes, etc. depicted in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not intended to limit the conditions under which the present invention can be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportional relationships, or adjustments in size should still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and objectives that can be achieved by the present invention.

[0102] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rope and knot detection method based on deep learning, applied to rope and knot detection of printed handbags, characterized by: include: Acquiring a regional image of the rope and knot, analyzing the knot material in the regional image to obtain the knot material type, generating corresponding image compensation parameters according to the knot material type, and compensating the regional image using the image compensation parameters to obtain a compensated regional image; Inputting the compensated regional image into a preset multi-material detection model for defect detection to obtain a first detection result, and inputting the compensated regional image into a preset single-material detection model for defect detection to obtain a second detection result; Performing a joint analysis on the first detection result and the second detection result. If the first detection result and the second detection result are inconsistent, the current printed handbag is discarded and the corresponding area image is archived; If the first detection result and the second detection result are consistent, then analyze whether the first detection result and the second detection result are qualified. If they are unqualified, the current printed handbag is rejected and the corresponding area image is archived; If qualified, the corresponding printed bags will flow into the next process after testing.

2. The rope and knot detection method based on deep learning according to claim 1, characterized in that: The step of acquiring a regional image of the rope and knot, analyzing the material of the knot in the regional image to obtain the knot material type, and generating corresponding image compensation parameters according to the knot material type includes: The knot is captured using a camera to obtain a regional image, the regional image is preprocessed to obtain a clear regional image, and the clear regional image is input into a preset deep learning model to obtain the knot material type; The knot material type is matched with a preset image compensation strategy to obtain image compensation parameters corresponding to the knot material type.

3. The rope and knot detection method based on deep learning according to claim 2, characterized in that: The step of preprocessing the regional image to obtain a clear regional image, and inputting the clear regional image into a preset deep learning model to obtain the knot material type includes: Performing denoising on the regional image to obtain a denoised regional image, and adjusting the brightness and contrast of the denoised regional image to obtain a clear regional image; Performing texture feature extraction on the clear regional image according to the gray level co-occurrence matrix to obtain texture features of the knot region, and performing color feature extraction on the clear regional image to identify color features of the knot material; The texture features and the color features are input into a preset deep learning model for material recognition to obtain the knot material type.

4. The rope and knot detection method based on deep learning according to claim 1, characterized in that: The step of compensating the regional image using the image compensation parameters to obtain a compensated regional image includes: inputting the image compensation parameters into a preset compensation module, adjusting the color and brightness of the regional image based on the image compensation parameters by the compensation module to obtain an adjusted regional image, and sharpening the adjusted regional image using an image enhancement algorithm to obtain a sharpened regional image; The knot area features at different scales are extracted from the sharpened area image to obtain a feature response map, and the feature response map is input into a preset Gaussian filter module for filtering and denoising to obtain a compensated area image.

5. The rope and knot detection method based on deep learning according to claim 4, characterized in that: The step of extracting knot region features at different scales from the sharpened region image to obtain a feature response map, and inputting the feature response map into a preset Gaussian filter module for filtering and denoising to obtain a compensated region image comprises: performing multi-scale analysis on the sharpened regional image using an image pyramid algorithm to obtain a multi-scale image set, extracting edge texture features and structural features in each scale image according to the multi-scale image set, and constructing a scale-invariant feature response map based on the edge texture features and the structural features; Feature aggregation is performed on the feature response map to obtain a fused knot candidate feature map, and the edge attention enhancement mechanism is applied to enhance the fused knot candidate feature map to obtain an enhanced feature map. The enhanced feature map is input into a preset Gaussian filter module for filtering and denoising to obtain a compensated regional image.

6. The rope and knot detection method based on deep learning according to claim 1, characterized in that: The step of inputting the compensated regional image into a preset multi-material detection model for defect detection to obtain a first detection result includes: Inputting the compensated regional image into a preset multi-material detection model, and in the multi-material detection model, performing feature extraction on the compensated regional image using a multi-scale convolutional neural network to obtain a texture feature map; applying a morphological segmentation algorithm to extract target edge regions and potential knot regions from the texture feature map, generating a rope profile set based on the target edge regions, inputting the rope profile set into a rope recognition module based on curve fitting and topological analysis, and calculating the number of ropes in the compensated regional image; generating a knot candidate region set according to the potential knot region, inputting the knot candidate region set into a target detection sub-model based on YOLOv8, and obtaining the number and positions of knots in the compensated regional image; Classifying and clustering the distribution of the number of knots and the positions of the knots on the number of ropes using a statistical morphological analysis algorithm to obtain a spatial distribution feature vector of the knots on each rope; A matching analysis is performed on the spatial distribution feature vector using a judgment sub-model based on a decision tree to obtain a first judgment result, and the first judgment result is compared with a preset rope knot structure standard to generate a first detection result.

7. The rope and knot detection method based on deep learning according to claim 1, characterized in that: The step of inputting the compensated regional image into a preset single-material detection model for defect detection to obtain a second detection result includes: Inputting the compensated regional image into a single material detection model, wherein a convolutional neural network is used in the single material detection model to perform image partition feature extraction on the compensated regional image to obtain an edge enhancement map and a target response heat map; Inputting the edge enhancement image into an image segmentation module based on an adaptive edge detection algorithm to extract the rope area, performing rope trajectory identification on the rope area using Hough transform, and obtaining the number of ropes in the compensated area image; Inputting the target response heat map into a knot detection sub-model based on Faster R-CNN, detecting and marking knot areas in the compensated regional image, and obtaining the number of knots and the spatial coordinate distribution of knots; The spatial relationship graph construction algorithm is used to model the positional relationship between the number of knots and the spatial coordinates of the knots on the number of ropes to obtain a knot symmetry index. The symmetry discrimination model based on cluster analysis is used to perform cluster analysis and group matching on the knot symmetry index, and knot combinations that conform to the knot logical structure are extracted. The knot combinations are compared with the rope knot structure standard for logical consistency, and a second detection result is output.

8. A rope and knot detection system based on deep learning, applied to rope and knot detection of printed handbags, characterized by: include: an acquisition module, configured to acquire an area image of the rope and knot, analyze the knot material in the area image to obtain the knot material type, generate corresponding image compensation parameters according to the knot material type, and compensate the area image using the image compensation parameters to obtain a compensated area image; a detection module, configured to input the compensated area image into a preset multi-material detection model for defect detection to obtain a first detection result, and input the compensated area image into a preset single-material detection model for defect detection to obtain a second detection result; an analysis module, configured to jointly analyze the first detection result and the second detection result; if the first detection result and the second detection result are inconsistent, reject the current printed handbag and archive the corresponding area image; If the first detection result and the second detection result are consistent, then analyze whether the first detection result and the second detection result are qualified. If they are unqualified, the current printed handbag is rejected and the corresponding area image is archived; If qualified, the corresponding printed bags will flow into the next process after testing.

Images