Image processing system and method for automatic can unloading of glass fiber roving

Abstract

The application provides an image processing system and method for automatic glass fiber yarn bundle drum unloading. The original monitoring image of the glass fiber yarn bundle stacking scene is obtained. The direction discrete graph representing the ring winding structure of the glass fiber yarn bundle is generated based on the original monitoring image, and the potential gap area is segmented. The potential gap area is used as a guide boundary to identify the constraint feature graph in the original monitoring image with the potential gap boundary as a constraint, and the local maximum points in the constraint feature graph are used as marker points to perform watershed segmentation on the original monitoring image to generate a yarn bundle segmentation mask. The symmetrical concave features caused by the center hole in the original monitoring image region are identified based on the yarn bundle segmentation mask. The grabbing target point for automatically unloading the glass fiber yarn bundle drum is identified based on the symmetrical concave features. The technical scheme provided by the application can identify the grabbing target point of the drum unloading mechanical arm under the interference of the ring winding structure of the glass fiber yarn bundle.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image processing system and method for automatic unloading of glass fiber yarn bundles. Background Technology

[0002] Image processing aims to improve image quality or extract useful information by analyzing, manipulating, and understanding digital images. Its core is to treat an image as a two-dimensional matrix composed of pixels and process it using mathematical transformations and algorithms. With the improvement of computing power, modern image processing has deeply integrated machine learning and deep learning, especially convolutional neural networks (CNNs), which have made breakthrough progress in fields such as image classification, object detection, and image generation. Image processing has been widely used in scenarios such as medical imaging, autonomous driving, and security monitoring, and is evolving towards real-time, lightweight, and multimodal fusion, becoming a key force empowering the intelligent upgrading of thousands of industries.

[0003] In existing image processing, especially in the context of glass fiber yarn bundles, the image processing for the appearance inspection of glass fiber yarn bundles mainly relies on the principle that there are quantifiable visual feature differences between defects and the background. That is, image preprocessing is first performed to stabilize imaging conditions and highlight the surface texture of the yarn bundle. Then, threshold segmentation is used to separate the yarn bundle area from the background and further locate possible defect areas. Finally, morphological processing and feature parameter calculation are used to classify and determine defects. However, glass fiber yarn bundles themselves have highly repetitive and directional annular winding textures. In the stacked state, the edge gradient information of the contact area between adjacent yarn bundles is weak, and the reflective properties of the yarn bundle surface are prone to forming highlights or shadows, producing grayscale change artifacts similar to the real gaps. This leads to recognition deviations of the target point grasped by the unloading robot arm. Therefore, how to identify the target point grasped by the unloading robot arm under the interference of the annular winding structure of glass fiber yarn bundles has become a difficult problem for the industry. Summary of the Invention

[0004] This application provides an image processing system and method for automatic unloading of glass fiber yarn bundles, which can identify the gripping target point of the unloading robot arm under the interference of the annular winding structure of the glass fiber yarn bundle.

[0005] In a first aspect, this application provides an image processing method for automatic unloading of glass fiber yarn bundles, comprising the following steps:

[0006] The original monitoring image of the glass fiber yarn bundle stacking scene is acquired, and the dispersion analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image is performed to generate a direction dispersion map characterizing the annular winding structure of the glass fiber yarn bundle.

[0007] Based on the directional dispersion map, potential gap regions in the glass fiber yarn stacking scene are segmented, and the potential gap regions are used as guiding boundaries to determine a constraint feature map with the potential gap boundaries as constraints in the original monitoring image.

[0008] Using the local maxima points in the constraint feature map as marker points, watershed segmentation is performed on the original monitoring image to generate a yarn ball segmentation mask, and based on the yarn ball segmentation mask, symmetrical concave features caused by the central hole are identified in the corresponding original monitoring image region.

[0009] The target point for automatically unloading glass fiber yarn bundles is identified based on the symmetrical concave feature.

[0010] In some embodiments, performing a dispersion analysis of the gray-level gradient direction within the neighborhood of each pixel in the original monitoring image to generate a direction dispersion map characterizing the annular winding structure of the glass fiber yarn bundle specifically includes:

[0011] The local neighborhood region corresponding to each pixel in the original monitoring image is divided according to the preset neighborhood window size;

[0012] For all pixel pairs within the local neighborhood of each pixel, the gray-level gradient direction is calculated to obtain the set of gray-level gradient directions corresponding to each pixel. The pixel pair is a combination of pixels formed by the center pixel and other non-center pixels within the local neighborhood.

[0013] Calculate the directional dispersion value of the corresponding pixel based on the gray-level gradient direction set;

[0014] Based on the pixel coordinate distribution of the original monitoring image, the directional dispersion values ​​of all pixels are spatially mapped to generate a directional dispersion map.

[0015] In some embodiments, segmenting the potential gap region in a glass fiber bundle stacking scenario based on the orientation dispersion map specifically includes:

[0016] Determine the discreteness segmentation threshold of the directional discreteness map;

[0017] The dispersion map of the direction is binarized using the dispersion segmentation threshold to obtain a binary map of the dispersion region.

[0018] Perform a morphological opening operation on the binary image of the discrete region to remove isolated noise points and generate a preliminary gap region image;

[0019] Extract the effective connected regions from the preliminary gap region map to obtain the potential gap regions in the glass fiber yarn stacking scenario.

[0020] In some embodiments, using the potential gap region as a guiding boundary, determining a constraint feature map in the original monitoring image constrained by the potential gap boundary specifically includes:

[0021] Morphological boundary extraction is performed on the potential gap region to obtain the contour boundary map of the potential gap region;

[0022] The contour boundary map is spatially registered with the original monitoring image to obtain a boundary registration map.

[0023] Based on the boundary registration map, the shortest Euclidean distance from each pixel in the original monitoring image to the contour boundary is calculated to obtain a pixel-level distance matrix.

[0024] The pixel-level distance matrix is ​​spatially mapped according to the spatial pixel distribution of the original monitoring image to generate a constraint feature map constrained by the potential gap boundary.

[0025] In some embodiments, performing watershed segmentation on the original monitoring image using local maxima points in the constraint feature map as marker points to generate a yarn cluster segmentation mask specifically includes:

[0026] The constraint feature map is subjected to maximum suppression processing, and then the set of local maxima points in the constraint feature map is obtained by filtering.

[0027] The local maxima points in the set of local maxima points are used as marker points and spatially aligned with the original monitoring image to obtain an image to be segmented with initial markers;

[0028] The grayscale gradient map is calculated for the image to be segmented with initial labels, and watershed segmentation is performed based on the initial label points to generate a yarn tuft segmentation mask.

[0029] In some embodiments, identifying symmetrical depression features caused by a central hole in the corresponding original monitoring image region based on the yarn clump segmentation mask specifically includes:

[0030] Based on the yarn clump segmentation mask, the principal axis analysis of the original monitoring image is performed to determine the principal axis direction of the corresponding original monitoring image region;

[0031] Using the main axis direction as the projection direction, perform gray-scale integral projection on the area covered by the yarn clump segmentation mask in the original monitoring image to obtain a one-dimensional gray-scale projection curve;

[0032] Gaussian smoothing filter is applied to the one-dimensional grayscale projection curve to obtain a smoothed grayscale projection curve.

[0033] Local minima are detected in the smoothed grayscale projection curve, and symmetrical concave features that conform to the characteristics of a central hole are selected through symmetry verification.

[0034] In some embodiments, raw monitoring images of a scene of stacked glass fiber yarn bundles are acquired using an industrial camera.

[0035] Secondly, this application provides an image processing system for automatic unloading of glass fiber yarn bundles, used to execute an image processing method for automatic unloading of glass fiber yarn bundles, the system comprising:

[0036] The acquisition module is used to acquire the original monitoring image of the glass fiber yarn bundle stacking scene, and to perform a discreteness analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image to generate a directional discreteness map characterizing the annular winding structure of the glass fiber yarn bundle.

[0037] The processing module is used to segment the potential gap region in the glass fiber yarn stacking scene based on the directional dispersion map, and use the potential gap region as a guiding boundary to determine the constraint feature map with the potential gap boundary as the constraint in the original monitoring image;

[0038] The processing module is further configured to perform watershed segmentation on the original monitoring image using local maxima points in the constraint feature map as marker points, generate a yarn ball segmentation mask, and identify symmetrical concave features caused by the central hole in the corresponding original monitoring image region based on the yarn ball segmentation mask.

[0039] The execution module is used to identify the gripping target point when automatically unloading glass fiber yarn based on the symmetrical concave feature.

[0040] Thirdly, this application provides a computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described image processing method for automatic unloading of glass fiber yarn bundles.

[0041] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing method for automatic unloading of glass fiber yarn bundles.

[0042] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0043] The image processing system and method for automatic unloading of glass fiber yarn bundles provided in this application first acquires an original monitoring image of a glass fiber yarn bundle stacking scene, and performs a dispersion analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image to generate a direction dispersion map characterizing the annular winding structure of the glass fiber yarn bundle; secondly, based on the direction dispersion map, potential gap regions in the glass fiber yarn bundle stacking scene are segmented, and the potential gap regions are used as guiding boundaries to determine a constraint feature map in the original monitoring image constrained by the potential gap boundaries; then, watershed segmentation is performed on the original monitoring image using local maxima points in the constraint feature map as marker points to generate a yarn bundle segmentation mask, and based on the yarn bundle segmentation mask, symmetrical concave features caused by the central hole are identified in the corresponding original monitoring image region; finally, based on the symmetrical concave features, the grasping target point for automatic unloading of the glass fiber yarn bundle is identified.

[0044] Therefore, this application can identify the grasping target point of the unloading robot arm under the interference of the annular winding structure of glass fiber yarn. First, it acquires the original monitoring image of the glass fiber yarn stacking scene, providing a real and complete data source for all subsequent image processing and feature extraction. Second, it performs a dispersion analysis of the gray-level gradient direction in the neighborhood of the original monitoring image and generates a direction dispersion map to obtain the gradient direction distribution characteristics of the annular winding structure of the glass fiber yarn, realizing the preliminary distinction between the yarn area and the background and gap areas, and effectively segmenting the potential gap area, solving the problem of distinguishing the yarn boundary in the stacking scene due to similar gray levels and interwoven textures. Furthermore, it uses the potential gap area as a guiding boundary to generate a constraint feature map, and combines the local maximum point to guide the watershed segmentation to generate a yarn segmentation mask. The constraint effect of the gap boundary avoids the oversegmentation problem that is prone to occur in the watershed algorithm. The method precisely defines the independent spatial range of a single yarn bundle, effectively solving the problem of regional confusion caused by the adhesion of adjacent yarn bundles, and providing a clear regional basis for subsequent feature extraction. Then, based on the yarn bundle segmentation mask, the method identifies symmetrical concave features, providing a precise region for the identification of the grasping target point of the unloading robot arm. This avoids the weak edge gradient information of the contact area between adjacent yarn bundles in the stacked state, and the reflective properties of the yarn bundle surface, which easily form highlights or shadows, leading to the identification deviation of the unloading robot arm's grasping target point. Finally, the center projection coordinates of the symmetrical concave features are mapped to the robot arm's grasping positioning coordinates, completing the precise conversion from image features to physical space coordinates, providing a precise grasping target point for the unloading robot arm, and effectively improving the operating efficiency of automatic unloading of glass fiber yarn bundles. In summary, the technical solution provided in this application can identify the grasping target point of the unloading robot arm under the interference of the annular winding structure of glass fiber yarn bundles. Attached Figure Description

[0045] Figure 1This is an exemplary flowchart of an image processing method for automatic unloading of glass fiber yarn bundles, according to some embodiments of this application;

[0046] Figure 2 This is an exemplary flowchart illustrating the determination of a directional dispersion map according to some embodiments of this application;

[0047] Figure 3 This is a schematic diagram of the structure of an image processing system for automatic unloading of glass fiber yarn bundles, according to some embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the structure of a computer device for implementing an image processing method for automatic unloading of glass fiber yarn bundles, according to some embodiments of this application. Detailed Implementation

[0049] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] refer to Figure 1 The figure is an exemplary flowchart of an image processing method for automatic unloading of glass fiber yarn bundles according to some embodiments of this application. The figure mainly includes the following steps:

[0051] In step S101, the original monitoring image of the glass fiber yarn stacking scene is acquired, and the dispersion analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image is performed to generate a direction dispersion map characterizing the annular winding structure of the glass fiber yarn.

[0052] In a specific implementation, an industrial camera is used to acquire the original monitoring image of the glass fiber yarn stacking scene. In addition, in other embodiments, other acquisition devices can be used to acquire the original monitoring image of the glass fiber yarn stacking scene, which is not limited here.

[0053] It should be noted that the original monitoring image in this application refers to the digital image obtained by image acquisition of the glass fiber yarn stack scene. The original monitoring image includes all original visual features such as the annular texture on the surface of the glass fiber yarn, the physical gap between the yarn, the shape of the central hole, and the ambient light. The acquisition of the original monitoring image provides a data basis for the subsequent identification of the accurate coordinates of the unloading robot arm.

[0054] In some embodiments, reference Figure 2As shown, this figure is an exemplary flowchart of determining the orientation dispersion map according to some embodiments of this application. In this embodiment, the dispersion analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image is performed to generate the orientation dispersion map characterizing the annular winding structure of the glass fiber yarn bundle. This can be achieved by the following steps:

[0055] In step S1011, the local neighborhood region corresponding to each pixel in the original monitoring image is divided according to the preset neighborhood window size;

[0056] In step S1012, the gray-level gradient direction is calculated for all pixel pairs in the local neighborhood region of each pixel to obtain the set of gray-level gradient directions corresponding to each pixel. The pixel pair is a pixel combination formed by the center pixel in the local neighborhood region and other non-center pixels respectively.

[0057] In step S1013, the directional dispersion value of the corresponding pixel is calculated based on the gray-level gradient direction set;

[0058] In step S1014, according to the pixel coordinate distribution of the original monitoring image, the orientation dispersion values ​​of all pixels are spatially mapped to generate an orientation dispersion map.

[0059] In specific implementation, firstly, a preset neighborhood window is obtained. This preset neighborhood window can be set to a 3×3 size. Centered on each pixel in the original monitoring image, the neighborhood window defines the pixel region as the local neighborhood region of the corresponding pixel. The local neighborhood region refers to the set of pixels centered on the target pixel and defined by the neighborhood window. Secondly, for each pixel's local neighborhood region, the center pixel of the local neighborhood region is taken as the target pixel. The target pixel and all other non-center pixels within the local neighborhood region are paired. The horizontal and vertical grayscale differences between each pair of pixels are calculated. Then, the arctangent function is used to solve for the grayscale gradient direction angle corresponding to each pair of pixels. The gray-level gradient directions of all pixel pairs within the same local neighborhood are combined to form the gray-level gradient direction set of the corresponding pixel. The gray-level gradient direction set refers to the set of gray-level gradient direction angles of all pixel pairs within the same local neighborhood. Then, for each pixel, the standard deviation of the gray-level gradient direction set corresponding to the pixel is calculated, and this standard deviation is used as the directional dispersion value of the corresponding pixel. The directional dispersion value refers to the quantitative index of the dispersion of the direction angles in the gray-level gradient direction set. Finally, according to the pixel coordinate arrangement order of the original monitoring image, the directional dispersion value corresponding to each pixel is used as the mapping value of the corresponding coordinate pixel in the new image, thereby generating a directional dispersion map characterizing the annular winding structure of the glass fiber yarn bundle.

[0060] It should be noted that the directional dispersion map in this application refers to the distribution map used to reflect the discrete features of the gray-level gradient direction in different regions. Since there is an essential difference in the physical form between the annular winding structure of the glass fiber yarn bundle and the gap of the yarn bundle stack, the structural differences that are difficult to distinguish with traditional gray-level features are transformed into quantifiable image features. In order to solve the problem of distinguishing between yarn bundles and gaps in complex stacking scenarios, the dispersion of the gray-level gradient direction in the neighborhood of each pixel is quantified. This makes the multi-directional gradient distribution in the yarn bundle region due to the annular winding present a high dispersion value, while the gap region presents a low dispersion value due to the single gray-level change direction. This numerical difference is then transformed into a clear gray-level difference, providing a direct quantitative basis for the accurate segmentation of potential gap regions.

[0061] In step S102, the potential gap region in the glass fiber yarn stacking scene is segmented based on the directional dispersion map, and the potential gap region is used as the guiding boundary to determine the constraint feature map with the potential gap boundary as the constraint in the original monitoring image.

[0062] In some embodiments, segmenting potential gap regions in a glass fiber bundle stacking scenario based on the orientation dispersion map is achieved using the following steps:

[0063] Determine the discreteness segmentation threshold of the directional discreteness map;

[0064] The dispersion map of the direction is binarized using the dispersion segmentation threshold to obtain a binary map of the dispersion region.

[0065] Perform a morphological opening operation on the binary image of the discrete region to remove isolated noise points and generate a preliminary gap region image;

[0066] Extract the effective connected regions from the preliminary gap region map to obtain the potential gap regions in the glass fiber yarn stacking scenario.

[0067] In specific implementation, firstly, the Otsu method in image segmentation is used to statistically analyze the gray-level histogram distribution of the directional dispersion map, and the gray-level value that maximizes the inter-class variance between the foreground (i.e., low-dispersion gap candidate regions) and the background (high-dispersion tuft regions) is calculated. This gray-level value is determined as the dispersion segmentation threshold, which is the gray-level critical value used to distinguish between high and low dispersion regions in the directional dispersion map. Secondly, using this dispersion segmentation threshold as the criterion, each pixel in the directional dispersion map is binarized. Pixels with directional dispersion values ​​below the dispersion segmentation threshold are marked as foreground pixels, and pixels with directional dispersion values ​​above the dispersion segmentation threshold are marked as background pixels, resulting in a dispersion region binary map. This dispersion region binary map refers to a binary image containing only foreground and background pixels. The foreground pixels represent candidate gap regions, and the background pixels represent the fiber filament region. Then, morphological opening operations in image processing are used to first erode the discrete region binary image to eliminate isolated noise points, and then dilate the eroded image to restore the original shape of the foreground region, generating a preliminary gap region map. The preliminary gap region map refers to the binary image that retains the candidate gap regions. Finally, the foreground pixels in the preliminary gap region map are traversed, and adjacent foreground pixels are divided into the same connected region. Connected regions with an area greater than an area threshold are selected as effective connected regions, and these effective connected regions are used as potential gap regions in the fiber filament stacking scene. The area threshold can be set according to actual needs or expert knowledge, and it is used to select boundary values ​​that meet the requirements of the gap region.

[0068] It should be noted that, in this application, the potential gap region refers to the set of image regions initially determined to be gaps between glass fiber yarn bundles. In the scenario of stacked glass fiber yarn bundles, the boundaries of adjacent yarn bundles are prone to segmentation adhesion due to similar gray levels and interwoven textures. Although the image features of the gap, as the physical separation boundary between yarn bundles, are initially highlighted by the directional dispersion map, the fuzzy gap candidates still need to be transformed into clear and reliable boundary references through threshold screening. Therefore, by clarifying the physical separation region between yarn bundles, the constraint feature map constructed based on the gap boundary has a clear constraint dimension, effectively avoiding the over-segmentation problem caused by the lack of clear boundaries in watershed segmentation. At the same time, it ensures that the yarn bundle segmentation mask can accurately distinguish the independent regions of adjacent yarn bundles, ultimately ensuring the accuracy and reliability of the robotic arm's grasping and positioning coordinates, and meeting the technical requirements for automatic unloading of glass fiber yarn bundles.

[0069] In some embodiments, the potential gap region is used as a guiding boundary, and the constraint feature map constrained by the potential gap boundary in the original monitoring image is determined by the following steps:

[0070] Morphological boundary extraction is performed on the potential gap region to obtain the contour boundary map of the potential gap region;

[0071] The contour boundary map is spatially registered with the original monitoring image to obtain a boundary registration map.

[0072] Based on the boundary registration map, the shortest Euclidean distance from each pixel in the original monitoring image to the contour boundary is calculated to obtain a pixel-level distance matrix.

[0073] The pixel-level distance matrix is ​​spatially mapped according to the spatial pixel distribution of the original monitoring image to generate a constraint feature map constrained by the potential gap boundary.

[0074] In specific implementation, firstly, existing morphological gradient extraction algorithms are used to construct structural elements of a preset shape. Since the gaps between stacked glass fiber bundles are mostly irregular curves, they can be set as circular structural elements with a 3×3 boundary. A dilation operation is performed on the binary image corresponding to the potential gap region to expand the gap region boundary. Then, an erosion operation is performed on the same binary image to shrink the gap region boundary. By subtracting the pixel grayscale values ​​of the dilated image and the eroded image, the difference between the two images is retained, which represents the edge pixels of the glass fiber bundle, thus obtaining the contour boundary map of the potential gap region. The contour boundary map refers to a binary image containing only the edge pixels of the potential gap region, where the edge pixels represent the spatial contour position of the gap region. Secondly, based on the co-origin shooting characteristics of the original monitoring image and the orientation dispersion map (i.e., both were acquired by the same camera from the same viewpoint, with pixel coordinates corresponding one-to-one), the edge pixels of the contour boundary map are superimposed onto the pixel coordinate system of the original monitoring image according to their corresponding coordinates to obtain... A boundary registration map is a bounding image that is perfectly aligned with the pixel coordinates of the original monitoring image, with boundary pixels marked only at the contour positions of potential gap regions, providing clear spatial boundary constraints. Then, an Euclidean distance transformation is applied, using the boundary registration map as input. The distance values ​​of the boundary pixels in the boundary registration map are set to 0. For each non-boundary pixel, the Euclidean distance between it and all boundary pixels in the image is calculated, and the minimum Euclidean distance is selected as the distance value for the corresponding non-boundary pixel. After traversing all pixels, a pixel-level distance matrix is ​​obtained. This pixel-level distance matrix is ​​a two-dimensional numerical matrix where each element stores the shortest Euclidean distance from the corresponding pixel to the contour boundary, indexed by the pixel coordinates of the original monitoring image. Finally, maintaining a pixel coordinate distribution structure completely consistent with the original monitoring image, the distance value corresponding to each coordinate in the pixel-level distance matrix is ​​used as the pixel grayscale value at that location, and these values ​​are filled into the corresponding pixel positions sequentially according to the coordinates, generating a constraint feature map constrained by the potential gap boundary.

[0075] It should be noted that the constraint feature map constrained by the potential gap boundary in this application refers to a map characterizing the spatial constraint positional relationship between each pixel in the original monitoring image and the potential gap boundary. Specifically, based on the pixel coordinates of the original monitoring image, the gray value of each pixel represents the grayscale image of the shortest Euclidean distance from that position to the contour boundary of the potential gap region. This is used to quantify the spatial positional relationship between each pixel in the image and the potential gap boundary. When identifying the positioning coordinates of the unloading robot arm grasping the fiberglass yarn bundle, this feature map helps to avoid regional adhesion or boundary blurring during segmentation due to the boundary being only a qualitative marker. It also addresses the issue of the spatial constraint positional relationship between the yarn bundle and the gap in the original monitoring image. The pain point of difficulty in directly distinguishing based on grayscale similarity is addressed by transforming the contour boundaries of potential gap regions into quantified distance features. This allows the grayscale value of each pixel in the original monitoring image to correspond to its shortest Euclidean distance to the gap boundary. Consequently, each independent yarn cluster region forms a distance distribution with the center of the yarn cluster as the peak (i.e., the distance from the center pixel of the yarn cluster to the gap boundary is the farthest, presenting a local maximum, and adjacent yarn clusters are isolated by the zero-distance region of the gap boundary). This provides a precise basis for the initial marker points for subsequent watershed segmentation, ensuring that the generated yarn cluster segmentation mask can accurately distinguish the independent regions of adjacent yarn clusters, thus guaranteeing the accuracy of the robotic arm's grasping and positioning.

[0076] In step S103, the original monitoring image is segmented using local maxima points in the constraint feature map as markers to generate a yarn cluster segmentation mask. Based on the yarn cluster segmentation mask, symmetrical concave features caused by the central hole are identified in the corresponding original monitoring image region.

[0077] In some embodiments, watershed segmentation is performed on the original monitoring image using local maxima points in the constraint feature map as marker points to generate a yarn cluster segmentation mask, which is achieved through the following steps:

[0078] The constraint feature map is subjected to maximum suppression processing, and then the set of local maxima points in the constraint feature map is obtained by filtering.

[0079] The local maxima points in the set of local maxima points are used as marker points and spatially aligned with the original monitoring image to obtain an image to be segmented with initial markers;

[0080] The grayscale gradient map is calculated for the image to be segmented with initial labels, and watershed segmentation is performed based on the initial label points to generate a yarn tuft segmentation mask.

[0081] In specific implementation, firstly, a square neighborhood window matching the minimum physical size of the glass fiber yarn bundle is set according to actual needs. Centered on each pixel in the constraint feature map, the distance value of each pixel is compared with the distance values ​​of all other pixels within the neighborhood window. Only pixels whose center pixel distance value is greater than the distance values ​​of all other pixels within the neighborhood window are retained as local maxima points. This process filters to obtain a set of local maxima points in the constraint feature map. The set of local maxima points refers to the dataset composed of pixels in the constraint feature map with the largest distance value within the preset neighborhood window, with each point corresponding to the central region of a potential yarn bundle. Then, based on the shared acquisition characteristics of the constraint feature map and the original monitoring image, each local maxima point in the set is used as a marker point, and its corresponding pixel coordinates are mapped to the same coordinate position in the original monitoring image. This marks the corresponding coordinates in the original monitoring image, obtaining an image to be segmented with initial labels. The image to be segmented with initial labels refers to an image that retains the grayscale information of the original monitoring image and marks the center position of the potential yarn bundle, providing clear seed points for watershed segmentation. Finally, the Sobel gradient calculation method is used to process the image with initial labels... The image to be segmented is subjected to pixel-wise sliding convolution to obtain horizontal and vertical gray-level gradient maps. The gray-level gradient values ​​of corresponding pixels in the two maps are then fused to obtain a gray-level gradient map of the image to be segmented with initial labels. This gray-level gradient map represents the intensity of gray-level changes for each pixel in the image; a larger gradient value indicates a more drastic gray-level change at that location, corresponding to image edges. A watershed segmentation algorithm is then used, with the labeled points in the image to be segmented as initial seed points. These initial seed points are the starting expansion centers of the segmentation region, corresponding to a single... The central candidate region of the yarn bundle is used as a constraint condition for region expansion based on the grayscale gradient map. During the segmentation process, only the region where the seed point is located is allowed to expand into regions with gradient values ​​lower than a preset gradient threshold. Regions with gradient values ​​higher than the preset gradient threshold are used as boundary barriers between adjacent regions. Through iterative calculation, the expansion region of each seed point gradually covers the complete range of the corresponding glass fiber yarn bundle until all expansion regions stop expanding at the gradient maximum boundary. Finally, yarn bundle segmentation masks with independent labels are generated for different yarn bundle regions. The gradient threshold can be set according to actual needs or expert knowledge, and is not limited here.

[0082] It should be noted that the yarn cluster segmentation mask in this application refers to the image that defines the spatial range of each independent glass fiber yarn cluster. The determination of the yarn cluster segmentation mask provides a precise basis for defining independent regions for the entire process of automatic positioning and grasping of glass fiber yarn clusters. By clearly defining the spatial boundaries of each yarn cluster, it achieves effective regional isolation between adjacent yarn clusters, avoiding regional adhesion or boundary confusion during subsequent feature extraction and positioning. At the same time, the mask provides a complete image region of a single yarn cluster for subsequent main axis analysis, ensuring the accuracy of gradient direction distribution statistics to accurately identify the main axis direction of the annular winding structure, providing a limited analysis range for grayscale projection, and ensuring the positioning accuracy of the robotic arm's grasping coordinates.

[0083] In some embodiments, the identification of symmetrical concave features caused by the central hole in the corresponding original monitoring image region based on the yarn clump segmentation mask is achieved through the following steps:

[0084] Based on the yarn clump segmentation mask, the principal axis analysis of the original monitoring image is performed to determine the principal axis direction of the corresponding original monitoring image region;

[0085] Using the main axis direction as the projection direction, perform gray-scale integral projection on the area covered by the yarn clump segmentation mask in the original monitoring image to obtain a one-dimensional gray-scale projection curve;

[0086] Gaussian smoothing filter is applied to the one-dimensional grayscale projection curve to obtain a smoothed grayscale projection curve.

[0087] Local minima are detected in the smoothed grayscale projection curve, and symmetrical concave features that conform to the characteristics of a central hole are selected through symmetry verification.

[0088] In specific implementation, firstly, based on the pixel grayscale attributes of the fiber optic yarn cluster segmentation mask, the weights of each pixel in the original monitoring image are set (i.e., non-zero pixels in the fiber optic yarn cluster region are weighted by their own grayscale value, while zero pixels in the background and other irrelevant areas are weighted at 0). The total weights of all pixels in the fiber optic yarn cluster region are calculated to obtain the zeroth moment, which reflects the total number of effective pixels in the fiber optic yarn cluster region. Simultaneously, the x and y coordinates of each pixel in the fiber optic yarn cluster region are multiplied by their own weights and summed to obtain the first moment, which reflects the positional distribution of the fiber optic yarn cluster region. Based on the ratio of the first moment to the zeroth moment, the geometric center coordinates of the fiber optic yarn cluster region are calculated. Then, using these geometric center coordinates as a reference, the weights of each fiber optic yarn cluster region are calculated separately. The squared deviations of the pixel's x-coordinate from the geometric center's x-coordinate, the squared deviations of the pixel's y-coordinate from the geometric center's y-coordinate, and the product of these deviations are multiplied by the corresponding pixel weights and summed to obtain three second-order central moments reflecting the shape distribution characteristics of the glass fiber bundle region. An inertia matrix is ​​constructed using these three second-order central moments, and then decomposed using an eigenvalue decomposition algorithm to obtain eigenvalues ​​and corresponding eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue is taken as the principal axis direction of the corresponding original monitoring image region. The principal axis direction refers to the vector direction representing the longest extension direction of the geometric shape of the glass fiber bundle region, which is consistent with the central axis of the glass fiber bundle's annular winding structure. Secondly, based on the determined principal axis direction... Using the projection direction as the projection direction, for the pixel region covered by the yarn cluster segmentation mask in the original monitoring image, the grayscale values ​​of all pixels on straight lines parallel to the main axis are summed along the main axis. Each line position corresponds to an integral summation result. A one-dimensional grayscale projection curve is generated according to the arrangement of the lines perpendicular to the main axis. The one-dimensional grayscale projection curve is a curve with the position coordinates perpendicular to the main axis as the horizontal axis and the grayscale integral sum at the corresponding position as the vertical axis, used to characterize the grayscale distribution characteristics of the yarn cluster region along the main axis. Then, a Gaussian smoothing filter algorithm is used to perform point-by-point sliding convolution operation on the one-dimensional grayscale projection curve to obtain a smoothed grayscale projection curve. The smoothed grayscale projection curve refers to the curve after Gaussian filtering and noise reduction. A one-dimensional curve retains the grayscale variation trend. Finally, a preset neighborhood window (which can be set according to the minimum physical diameter of the glass fiber yarn bundle, for example, the pixel size corresponding to 1 / 3 to 1 / 2 of the minimum physical diameter) is used to traverse the smoothed grayscale projection curve. The region with the smallest value in the window that is smaller than the values ​​of the two adjacent points is marked as a local minimum region. Then, the correlation coefficient of the grayscale value distribution on the left and right sides of each local minimum region with the midpoint of the curve as the axis of symmetry is calculated. Local minimum regions with a correlation coefficient greater than a preset correlation coefficient threshold are selected as symmetrical concave features that conform to the central hole feature. The specific correlation coefficient threshold can be set according to actual needs or according to expert knowledge, and is not limited here.

[0089] It should be noted that the symmetrical concave feature in this application refers to the local minimum value region that exhibits a centrally symmetrical distribution in the one-dimensional grayscale projection curve, corresponding to the grayscale value reduction region caused by the central hole of the glass fiber yarn bundle. Its symmetry is directly related to the circular structure of the hole. As an inherent physical characteristic of the annular winding structure, the central hole of the glass fiber yarn bundle has a circular cross-section and is symmetrical about the main axis of the yarn bundle. Reflected on the grayscale projection curve in the direction of the main axis, the grayscale value will decrease due to the lack of yarn coverage in the hole area, forming a symmetrical concave shape corresponding to the symmetrical structure of the hole. However, the non-hole interference in the grayscale projection curve mostly exhibits an asymmetrical distribution. Therefore, by locking the position of the central hole of the yarn bundle through this symmetrical concave feature, a clear quantitative basis can be provided for the subsequent calculation of the center coordinates of the yarn bundle, ensuring that the center positioning is not affected by the uneven stacking of the yarn bundle, and meeting the technical requirements for rapid and accurate identification and positioning of the central hole of the yarn bundle during the automatic unloading process of the glass fiber yarn bundle.

[0090] In step S104, the grasping target point for automatically unloading the glass fiber yarn bundle is identified based on the symmetrical concave feature.

[0091] In some embodiments, the identification of the gripping target point during automatic unloading of glass fiber yarn bundles based on the symmetrical concave feature is achieved through the following steps:

[0092] Calculate the center projection coordinates of the symmetrical concave feature based on the grayscale projection curve corresponding to the symmetrical concave feature;

[0093] Convert the center projection coordinates into two-dimensional pixel coordinates in the original monitoring image;

[0094] The two-dimensional pixel coordinates are converted into positioning coordinates in the robotic arm grasping coordinate system by camera intrinsic parameter calibration, so as to obtain the grasping target point when automatically unloading the glass fiber yarn bundle.

[0095] In specific implementation, firstly, the smoothed grayscale projection curve corresponding to the symmetrical concave feature is obtained. From the obtained smoothed grayscale projection curve, the horizontal axis coordinates (i.e., the projection position coordinates perpendicular to the main axis direction) corresponding to the left and right boundary points of the symmetrical concave feature are extracted. By calculating the arithmetic mean of the coordinates of the two boundary points, the center projection coordinates of the symmetrical concave feature are obtained. The center projection coordinates refer to the center horizontal axis position coordinates of the symmetrical concave feature on the grayscale projection curve, corresponding to the projection center of the yarn ball's central hole along the direction perpendicular to the main axis. Then, based on the definition of the direction of grayscale projection (i.e., the projection direction is the yarn ball's main axis direction, and the horizontal axis of the projection curve is the position perpendicular to the main axis direction), the following steps are taken. Combining the geometric center coordinates of the glass fiber yarn bundle region (i.e., calculated from the principal axis analysis stage), the center projection coordinates are used as pixel coordinate components perpendicular to the principal axis direction, and the component of the geometric center coordinates in the principal axis direction is used as another pixel coordinate component. The two-dimensional pixel coordinates in the original monitoring image are obtained by combining the coordinate components. The two-dimensional pixel coordinates refer to the pixel coordinates that uniquely correspond to the position of the center hole of the yarn bundle in the original monitoring image. Finally, camera calibration technology is used to obtain the camera's intrinsic and extrinsic parameters in advance. Through the perspective projection inverse transformation algorithm, the two-dimensional pixel coordinates are substituted into the coordinate transformation model constructed based on the intrinsic and extrinsic parameters to obtain the grasping target point when automatically unloading the glass fiber yarn bundle.

[0096] It should be noted that the target point used for grasping in this application refers to the target point grasped by the robotic arm identified in the coordinate system of the unloading robotic arm.

[0097] In another aspect, in some embodiments, this application provides an image processing system for automatic unloading of glass fiber yarn bundles, see reference. Figure 3 The figure is a schematic diagram of the structure of an image processing system for automatic unloading of glass fiber yarn bundles according to some embodiments of this application. The image processing system for automatic unloading of glass fiber yarn bundles includes: an acquisition module 201, a processing module 202, and an execution module 203, which are described below:

[0098] The acquisition module 201 in this application is mainly used to acquire the original monitoring image of the glass fiber yarn bundle stacking scene, and to perform a discreteness analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image to generate a directional discreteness map characterizing the annular winding structure of the glass fiber yarn bundle.

[0099] Processing module 202, in this application, is mainly used to segment the potential gap region in the glass fiber yarn stacking scene based on the directional dispersion map, and use the potential gap region as a guiding boundary to determine the constraint feature map with the potential gap boundary as a constraint in the original monitoring image;

[0100] The processing module 202 is further configured to perform watershed segmentation on the original monitoring image using local maxima points in the constraint feature map as marker points, generate a yarn ball segmentation mask, and identify symmetrical concave features caused by the central hole in the corresponding original monitoring image region based on the yarn ball segmentation mask.

[0101] The execution module 203 in this application is mainly used to identify the gripping target point when automatically unloading the glass fiber yarn ball based on the symmetrical concave feature.

[0102] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described image processing method for automatic unloading of glass fiber yarn bundles.

[0103] In some embodiments, reference Figure 4 The figure is a schematic diagram of the structure of a computer device implementing an image processing method for automatic unloading of glass fiber yarn bundles, according to some embodiments of this application. The image processing method for automatic unloading of glass fiber yarn bundles in the above embodiments can be... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0104] The processor 301 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more image processing methods for automatically unloading glass fiber yarn bundles in this application.

[0105] The communication bus 302 can be used to transmit information between the aforementioned components.

[0106] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0107] The memory 303 stores program code for executing the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the image processing method for automatic unloading of glass fiber yarn bundles can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0108] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0109] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0110] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0111] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing method for automatic unloading of glass fiber yarn bundles.

[0112] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0113] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image processing method for automatic unloading of glass fiber yarn bundles, characterized in that, Includes the following steps: The original monitoring image of the glass fiber yarn bundle stacking scene is acquired, and the dispersion analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image is performed to generate a direction dispersion map characterizing the annular winding structure of the glass fiber yarn bundle. Based on the directional dispersion map, potential gap regions in the glass fiber yarn stacking scene are segmented, and the potential gap regions are used as guiding boundaries to determine a constraint feature map with the potential gap boundaries as constraints in the original monitoring image. Specifically, determining a constraint feature map in the original monitoring image, using the potential gap region as a guiding boundary, includes: performing morphological boundary extraction on the potential gap region to obtain a contour boundary map of the potential gap region; registering the contour boundary map with the original monitoring image in spatial coordinates to obtain a boundary registration map; calculating the shortest Euclidean distance from each pixel in the original monitoring image to the contour boundary based on the boundary registration map to obtain a pixel-level distance matrix; and spatially mapping the pixel-level distance matrix according to the spatial pixel distribution of the original monitoring image to generate a constraint feature map constrained by the potential gap boundary. The constraint feature map refers to a map characterizing the spatial constraint positional relationship between each pixel in the original monitoring image and the potential gap boundary. Using the local maxima points in the constraint feature map as marker points, watershed segmentation is performed on the original monitoring image to generate a yarn ball segmentation mask, and based on the yarn ball segmentation mask, symmetrical concave features caused by the central hole are identified in the corresponding original monitoring image region. Specifically, identifying symmetrical concavity features caused by a central hole in the corresponding original monitoring image region based on the yarn cluster segmentation mask includes: performing principal axis analysis on the original monitoring image based on the yarn cluster segmentation mask to determine the principal axis direction of the corresponding original monitoring image region; performing gray-scale integral projection on the region covered by the yarn cluster segmentation mask in the original monitoring image with the principal axis direction as the projection direction to obtain a one-dimensional gray-scale projection curve; performing Gaussian smoothing filtering on the one-dimensional gray-scale projection curve to obtain a smoothed gray-scale projection curve; detecting local minimum value regions in the smoothed gray-scale projection curve, and filtering out symmetrical concavity features that conform to the central hole feature through symmetry verification, wherein the symmetrical concavity feature refers to local minimum value regions in the one-dimensional gray-scale projection curve that exhibit a centrally symmetrical distribution. The target point for automatically unloading glass fiber yarn bundles is identified based on the symmetrical concave feature.

2. The method as described in claim 1, characterized in that, Performing a discrete analysis of the gray-level gradient direction within the neighborhood of each pixel in the original monitoring image to generate a directional discrete map characterizing the annular winding structure of the glass fiber yarn bundle specifically includes: The local neighborhood region corresponding to each pixel in the original monitoring image is divided according to the preset neighborhood window size; For all pixel pairs within the local neighborhood of each pixel, the gray-level gradient direction is calculated to obtain the set of gray-level gradient directions corresponding to each pixel. The pixel pair is a combination of pixels formed by the center pixel and other non-center pixels within the local neighborhood. Calculate the directional dispersion value of the corresponding pixel based on the gray-level gradient direction set; Based on the pixel coordinate distribution of the original monitoring image, the directional dispersion values ​​of all pixels are spatially mapped to generate a directional dispersion map.

3. The method as described in claim 1, characterized in that, The segmentation of potential gap regions in the glass fiber yarn stacking scenario based on the aforementioned directional dispersion map specifically includes: Determine the discreteness segmentation threshold of the directional discreteness map; The dispersion map of the direction is binarized using the dispersion segmentation threshold to obtain a binary map of the dispersion region. Perform a morphological opening operation on the binary image of the discrete region to remove isolated noise points and generate a preliminary gap region image; Extract the effective connected regions from the preliminary gap region map to obtain the potential gap regions in the glass fiber yarn stacking scenario.

4. The method as described in claim 1, characterized in that, Using local maxima points in the constrained feature map as markers, watershed segmentation is performed on the original monitoring image to generate a yarn cluster segmentation mask, specifically including: The constraint feature map is subjected to maximum suppression processing, and then the set of local maxima points in the constraint feature map is obtained by filtering. The local maxima points in the set of local maxima points are used as marker points and spatially aligned with the original monitoring image to obtain an image to be segmented with initial markers; The grayscale gradient map is calculated for the image to be segmented with initial labels, and watershed segmentation is performed based on the initial label points to generate a yarn tuft segmentation mask.

5. The method as described in claim 1, characterized in that, Raw monitoring images of stacked glass fiber yarn bundles were acquired using industrial cameras.

6. An image processing system for automatic unloading of glass fiber yarn bundles, used to execute the image processing method for automatic unloading of glass fiber yarn bundles as described in any one of claims 1 to 5, characterized in that, The system includes: The acquisition module is used to acquire the original monitoring image of the glass fiber yarn bundle stacking scene, and to perform a discreteness analysis of the gray-level gradient direction in the neighborhood of each pixel in the original monitoring image to generate a directional discreteness map characterizing the annular winding structure of the glass fiber yarn bundle. The processing module is used to segment potential gap regions in the glass fiber yarn stacking scene based on the directional dispersion map, and to determine a constraint feature map in the original monitoring image with the potential gap boundary as the constraint, using the potential gap region as the guide boundary. Specifically, determining the constraint feature map in the original monitoring image with the potential gap boundary as the constraint includes: performing morphological boundary extraction on the potential gap region to obtain a contour boundary map of the potential gap region; performing spatial coordinate registration between the contour boundary map and the original monitoring image to obtain a boundary registration map; calculating the shortest Euclidean distance from each pixel in the original monitoring image to the contour boundary based on the boundary registration map to obtain a pixel-level distance matrix; and performing spatial mapping on the pixel-level distance matrix according to the spatial pixel distribution of the original monitoring image to generate a constraint feature map with the potential gap boundary as the constraint. The constraint feature map refers to a map characterizing the spatial constraint positional relationship between each pixel in the original monitoring image and the potential gap boundary. The processing module is further configured to perform watershed segmentation on the original monitoring image using local maxima points in the constraint feature map as markers, generate a yarn cluster segmentation mask, and identify symmetrical concave features caused by a central hole in the corresponding original monitoring image region based on the yarn cluster segmentation mask. Specifically, identifying symmetrical concave features caused by a central hole in the corresponding original monitoring image region based on the yarn cluster segmentation mask includes: performing principal axis analysis on the original monitoring image based on the yarn cluster segmentation mask to determine the principal axis direction of the corresponding original monitoring image region; performing grayscale integral projection on the area covered by the yarn cluster segmentation mask in the original monitoring image using the principal axis direction as the projection direction to obtain a one-dimensional grayscale projection curve; performing Gaussian smoothing filtering on the one-dimensional grayscale projection curve to obtain a smoothed grayscale projection curve; detecting local minima in the smoothed grayscale projection curve, and filtering out symmetrical concave features that conform to the central hole feature through symmetry verification. The symmetrical concave features refer to local minima that exhibit a centrally symmetrical distribution in the one-dimensional grayscale projection curve. The execution module is used to identify the gripping target point when automatically unloading glass fiber yarn based on the symmetrical concave feature.

7. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the image processing method for automatic unloading of glass fiber yarn bundles as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image processing method for automatic unloading of glass fiber yarn bundles as described in any one of claims 1 to 5.

Images