A training-free workpiece edge detection method for three-axis machines

Abstract

The application provides a three-axis machine-oriented no-training workpiece edge detection method, relates to the technical field of image segmentation, and solves the technical problems of high-cost pixel labeling dependence, insufficient model generalization ability and decreased segmentation precision in the prior art. The method comprises the following steps: obtaining standard and target workpiece images and their characteristics; screening and position clustering are performed based on the standard and target block characteristic sets to obtain a positive and negative prompt point set; and a range is framed to obtain a prompt box; the standard and target workpiece images, the positive and negative prompt point set and the prompt box are jointly input into an image segmentation model to obtain a preliminary segmentation mask; false positive elimination is performed to obtain a foreground mask; and workpiece edges are obtained through edge processing. The application is used in the workpiece edge detection process and provides a no-training workpiece edge detection scheme for a three-axis machine, which only needs a single standard image. The application significantly reduces data dependence and manual cost by automatically generating prompt points, and improves segmentation precision by using a feature re-matching module.

Description

Technical Field

[0001] This application relates to the field of image segmentation technology, and in particular to a training-free workpiece edge detection method for three-axis machines. Background Technology

[0002] In industrial automation, three-axis mechanical systems are commonly used for spatial positioning and path planning on workpiece surfaces, and their operation often relies on the accurate identification of the target workpiece edges. Edge detection, as a key technology in industrial vision systems, directly affects the stability and accuracy of subsequent processes such as workpiece positioning, trajectory planning, contour fitting, and machining path control. Currently, industrial edge detection technology mainly relies on image segmentation methods. Image segmentation is a fundamental task in computer vision, aiming to divide an image into several regions with independent semantic or structural features to facilitate the identification and extraction of object boundaries. In recent years, the development of deep learning has driven the widespread application of image segmentation in scenarios such as industrial quality inspection and surface defect identification.

[0003] Existing deep learning-based image segmentation models primarily rely on supervised learning, requiring a large amount of time-consuming, labor-intensive, and highly specialized pixel-level labeled data. However, the objects to be detected in industrial scenarios (such as different materials, reflections, textures, and boundary clarity) are highly diverse, leading to insufficient generalization ability of traditional models when encountering new situations, and a significant decline in edge detection performance. Although the SAM model, which has zero-shot segmentation capability, offers a new approach, current models pre-trained on natural images struggle to adapt to the weak edges, blurred contours, and complex background characteristics common in industrial images, easily producing false positive or false negative predictions. Ultimately, this severely impacts the accuracy of three-axis machine trajectory planning and operation, which relies on its segmentation results. Summary of the Invention

[0004] This application provides a training-free workpiece edge detection method for three-axis machines, which solves the technical problems of high-cost pixel annotation dependence, insufficient model generalization ability, and reduced segmentation accuracy in the prior art.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, a training-free workpiece edge detection method for three-axis machines is provided, including:

[0007] S1. Obtain the target workpiece image, the standard workpiece image, and the corresponding standard feature set; input the target workpiece image into the feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein, the standard feature set is obtained by segmenting and marking the standard workpiece image and extracting block-level features;

[0008] S2. Based on the standard feature set and the target block feature set, filter several target blocks to obtain several filtered target blocks; cluster the several filtered target blocks to obtain a set of positive prompt points and a set of negative prompt points;

[0009] S3. Define the target area based on the set of positive prompt points to obtain the prompt box;

[0010] S4. Input the standard workpiece image, the target workpiece image, the set of positive prompt points, the set of negative prompt points, and the prompt box into the image segmentation model to obtain the preliminary segmentation mask;

[0011] S5. Based on the mask of the standard workpiece image, remove the false positive areas in the preliminary segmentation mask to obtain the foreground mask;

[0012] S6. The workpiece edge is obtained by extracting the contour of the foreground mask and processing it with a polygon fitting algorithm.

[0013] In conjunction with the first aspect above, in one possible implementation, the standard feature set includes: a standard foreground block feature set, a standard background block feature set, and a standard sub-block feature set;

[0014] The standard sub-block feature set is extracted by a standard workpiece image input feature extractor.

[0015] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the standard foreground block feature set and the standard background block feature set includes:

[0016] The standard workpiece image is divided into a mask grid, and the divided image is classified, labeled and feature extracted according to the position of the mask in the standard workpiece image to obtain the standard foreground block feature set and the standard background block feature set.

[0017] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the set of positive prompts and the set of negative prompts includes:

[0018] S11. Perform forward matching between the standard foreground block feature set and the block-level features corresponding to several target blocks, and filter the matched target blocks to obtain the first target block set.

[0019] S12. Perform reverse matching between the standard sub-block feature set and the features corresponding to the first target block set, and filter the matched first target block set; filter the filtered first target block set again according to the foreground mask of the standard workpiece image, and retain the first target blocks located in the foreground area to obtain the second target block set;

[0020] S13. Perform self-matching between the features corresponding to the second target block set to obtain a similarity matrix; calculate the average adjacent distance of each block in the similarity matrix and then calculate its mean to obtain the overall average distance; filter the second target blocks whose average adjacent distance is lower than the overall average distance to obtain a number of filtered target blocks; obtain a positive prompt point candidate set by calculating the center point of each filtered target block.

[0021] S14. Perform K-means clustering on the candidate set of positive prompts to obtain the set of positive prompts;

[0022] S15. Replace the standard foreground block feature set in step S11 with the standard background block feature set; repeat steps S11 to S14 to obtain the negative cue point set.

[0023] In conjunction with the first aspect above, in one possible implementation, the K-means clustering of the candidate set of positive prompts includes:

[0024] Based on the number of target blocks in the candidate set of positive prompts, the range of K values ​​for the number of candidate clusters is determined, and K-means clustering is performed traversing this range to obtain preliminary clustering results.

[0025] Based on the preliminary clustering results, the calculation formula was used. The contour coefficients corresponding to each K value are obtained. ;in, It is the first The average distance from a point to other points within its cluster. It is the first The average distance from a point to all points in the nearest cluster;

[0026] Iterate through a number of silhouette coefficients and select the optimal number of clusters corresponding to the largest silhouette coefficient.

[0027] Based on the optimal number of clusters, the candidate set of positive prompts is further clustered using K-means to obtain the set of positive prompts.

[0028] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the prompt box includes:

[0029] Iterate through the set of positive feedback points to obtain the coordinates of the minimum and maximum positive feedback points; where the coordinates of the minimum positive feedback point are... The coordinates of the maximum positive feedback point are ;in, ;

[0030] The bounding box is calculated based on the coordinates of the minimum and maximum positive cue points; this bounding box is used as the cue box in the image segmentation model; the coordinates of the top-left corner of the bounding box are... The coordinates of the lower right corner are .

[0031] In conjunction with the first aspect above, in one possible implementation, the image segmentation model includes: an image encoder, a cue encoder, and a mask decoder;

[0032] The standard workpiece image and the target workpiece image are processed by an image encoder to obtain the image feature tensor;

[0033] The prompt encoder processes the set of positive prompt points, the set of negative prompt points, and the prompt box to obtain the prompt feature tensor;

[0034] The image feature tensor and the cue feature tensor are fused and decoded by a mask decoder to obtain a preliminary segmentation mask.

[0035] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the foreground mask includes:

[0036] The initial segmentation mask is divided and labeled to obtain several foreground blocks; the feature set of the standard foreground blocks is matched with the features corresponding to the several foreground blocks, and the matched foreground blocks are then filtered.

[0037] Determine whether some or all pixels of the filtered foreground block are located within the mask background area of ​​the standard workpiece image; if yes, the area is determined to be a false positive and is removed at the corresponding position in the initial segmentation mask; if no, the corresponding area is retained, and the false positive areas are removed by filtering block by block to obtain the foreground mask.

[0038] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the workpiece edge includes:

[0039] An initial set of contours is obtained by performing edge detection on the foreground mask using a contour extraction algorithm.

[0040] The initial contour set is simplified and fitted using a polygon approximation algorithm to obtain an optimized contour set;

[0041] By adapting the optimized contour set to the process through path point generation rules, a three-axis trajectory path point set is obtained.

[0042] By encapsulating data structures, the information of the three-axis trajectory path point set and the optimized contour set is integrated to obtain the workpiece edge detection result.

[0043] Based on the above technical solutions, the training-free workpiece edge detection method for three-axis machines provided in this application reduces the need for a large amount of labeled data and the corresponding labeling and training costs by relying on a single standard workpiece image for training-free segmentation. Simultaneously, it utilizes prior knowledge of industrial data combined with a pre-trained visual model for bidirectional feature matching, and designs a cue point sampling strategy based on location clustering, achieving automatic generation of the cue input required by the image segmentation model, significantly improving the automation level of the segmentation process. Furthermore, the feature re-matching module refines the initial segmentation mask, effectively eliminating false positive pixels and significantly improving segmentation accuracy, especially when handling blurred boundaries. In addition, this method fully utilizes the capabilities of a general feature extraction model, enhancing the model's adaptability and stability in diverse industrial scenarios, thereby significantly improving overall generalization performance and application effectiveness.

[0044] In a second aspect, an electronic device is provided, comprising: a communication unit and a processing unit; the communication unit is used to acquire a target workpiece image, a standard workpiece image and a corresponding standard feature set; input the target workpiece image to a feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein, the standard feature set is obtained by segmenting and marking the standard workpiece image and extracting block-level features;

[0045] The processing unit is used to filter several target blocks according to a standard feature set and a target block feature set to obtain several filtered target blocks; to cluster the several filtered target blocks to obtain a set of positive prompt points and a set of negative prompt points; to define the target range according to the set of positive prompt points to obtain a prompt box; to input the standard workpiece image, the target workpiece image, the set of positive prompt points, the set of negative prompt points, and the prompt box into the image segmentation model to obtain a preliminary segmentation mask; to remove false positive regions in the preliminary segmentation mask according to the mask of the standard workpiece image to obtain a foreground mask; and to obtain the workpiece edge by performing contour extraction and polygon fitting algorithms on the edge of the foreground mask.

[0046] Thirdly, this application provides an electronic device, including: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the methods described in the first aspect and any possible implementation thereof. This electronic device may be an electronic device or a chip within an electronic device.

[0047] Fourthly, this application provides a training-free workpiece edge detection system for a three-axis machine, comprising: an image data acquisition module, an image data processing module, and an acquisition result module; wherein, the image data acquisition module is used to acquire a target workpiece image, a standard workpiece image, and a corresponding standard feature set; the target workpiece image is input to a feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein, the standard feature set is obtained by segmenting and marking the standard workpiece image and extracting block-level features; the image data processing module is used to process several target blocks according to the standard feature set and the target block feature set. The target blocks are filtered to obtain several filtered target blocks; these filtered target blocks are clustered to obtain a set of positive and negative prompt points; the target range is bounded according to the set of positive prompt points to obtain a prompt box; the standard workpiece image, the target workpiece image, the set of positive and negative prompt points, and the prompt box are input into the image segmentation model to obtain a preliminary segmentation mask; false positive regions in the preliminary segmentation mask are removed according to the mask of the standard workpiece image to obtain a foreground mask; the result acquisition module is used to obtain the workpiece edge by performing contour extraction and polygon fitting algorithms on the edge of the foreground mask.

[0048] Fifthly, this application provides a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0049] In a sixth aspect, this application provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0050] This application provides a training-free workpiece edge detection method for three-axis machines. It reduces the need for large amounts of labeled data and the associated labeling and training costs by relying solely on a single standard image for training-free segmentation. Simultaneously, it utilizes prior knowledge of industrial data combined with a pre-trained visual model for bidirectional feature matching and designs a cue point sampling strategy based on location clustering, achieving automatic generation of the input prompts required by the SAM (Search Engine Analysis) and significantly improving the automation level of the segmentation process. Furthermore, the initial segmentation mask is refined through a feature re-matching module, effectively eliminating false positive pixels and significantly improving segmentation accuracy, especially when handling blurred boundaries. In addition, this method fully utilizes the capabilities of a general feature extraction model, enhancing the model's adaptability and stability in diverse industrial scenarios, thereby significantly improving overall generalization performance and application effectiveness.

[0051] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0052] Figure 1 A system architecture diagram of a workpiece edge detection system provided in this application embodiment;

[0053] Figure 2 A flowchart illustrating a training-free workpiece edge detection method for a three-axis machine provided in this application embodiment;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0057] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0058] The training-free workpiece edge detection method for three-axis machines provided in this application embodiment can be applied to, for example... Figure 1 In the workpiece edge detection system 100 shown, such as Figure 1 As shown, the communication system includes: an information capture terminal 101, a cloud computing device 102, and an edge computing node 103.

[0059] The information acquisition terminal 101 is used to acquire the target workpiece image, the standard workpiece image, and the corresponding standard feature set; input the target workpiece image to the feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein, the standard feature set is obtained by segmenting and marking the standard workpiece image and extracting block-level features.

[0060] The cloud computing device 102 is used to filter several target blocks according to a standard feature set and a target block feature set to obtain several filtered target blocks; to cluster the filtered target blocks to obtain a set of positive prompt points and a set of negative prompt points; to define the target range according to the set of positive prompt points to obtain a prompt box; to input the standard workpiece image, the target workpiece image, the set of positive prompt points, the set of negative prompt points, and the prompt box into the image segmentation model to obtain a preliminary segmentation mask; and to remove false positive regions in the preliminary segmentation mask according to the mask of the standard workpiece image to obtain a foreground mask.

[0061] Edge computing node 103 is used to obtain the workpiece edge by performing contour extraction and polygon fitting algorithm on the edge of the foreground mask.

[0062] To address the technical problems of high-cost pixel labeling dependence, insufficient model generalization ability, and decreased segmentation accuracy in existing technologies, this application provides a training-free workpiece edge detection method for three-axis machines. The method includes: acquiring a target workpiece image, a standard workpiece image, and a corresponding standard feature set; inputting the target workpiece image into a feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein the standard feature set is obtained by segmenting and marking the standard workpiece image and extracting block-level features; filtering the target blocks according to the standard feature set and the target block feature set to obtain several filtered target blocks; clustering the filtered target blocks to obtain a set of positive and negative cue points; defining the target range according to the set of positive cue points to obtain a cue box; inputting the standard workpiece image, the target workpiece image, the set of positive and negative cue points, and the cue box into an image segmentation model to obtain a preliminary segmentation mask; and then, based on the standard workpiece image... The image mask is used to remove false positive regions from the initial segmentation mask, resulting in a foreground mask. The edges of the foreground mask are then processed using contour extraction and polygon fitting algorithms to obtain the workpiece edges. Based on this, a training-free segmentation method relying solely on a single standard image reduces the need for large amounts of labeled data and the associated labeling and training costs. Simultaneously, prior knowledge of industrial data is combined with a pre-trained visual model for bidirectional feature matching, and a cue point sampling strategy based on location clustering is designed to automatically generate the cue input required by the image segmentation model, significantly improving the automation level of the segmentation process. Furthermore, the initial segmentation mask is refined through a feature rematching module, effectively eliminating false positive pixels and significantly improving segmentation accuracy, especially when handling blurred boundaries. In addition, this method fully utilizes the capabilities of a general feature extraction model, enhancing the model's adaptability and stability in diverse industrial scenarios, thereby significantly improving overall generalization performance and application effectiveness.

[0063] like Figure 2 As shown in the embodiment of this application, a method for detecting the edge of a workpiece without training for a three-axis machine is provided, comprising:

[0064] S201. Obtain the target workpiece image, the standard workpiece image, and the corresponding standard feature set; input the target workpiece image into the feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and target block feature sets.

[0065] The standard feature set is obtained by dividing and marking standard workpiece images and extracting block-level features.

[0066] In some implementations, the target block feature set corresponding to several target blocks is: ;in, , Indicates the number of blocks in the target workpiece image. The feature embedding dimension of the feature extractor .

[0067] For example, the feature extractor is the DINOv2 model.

[0068] S202. Based on the standard feature set and the target block feature set, filter several target blocks to obtain several filtered target blocks; cluster the several filtered target blocks to obtain a set of positive prompt points and a set of negative prompt points.

[0069] It should be noted that feature matching can be used not only for matching between different images to identify similar regions between them, but also within the same image to filter out specific regions, thereby improving the accuracy and efficiency of subsequent processing.

[0070] For example, k-means clustering is used when clustering several target blocks for screening.

[0071] S203. Define the target area based on the set of positive prompt points to obtain the prompt box.

[0072] In some implementations, the method for obtaining the prompt box includes:

[0073] Iterate through the set of positive feedback points to obtain the coordinates of the minimum and maximum positive feedback points; where the coordinates of the minimum positive feedback point are... The coordinates of the maximum positive feedback point are ;in, ;

[0074] The bounding box is calculated based on the coordinates of the minimum and maximum positive cue points; this bounding box is used as the cue box in the image segmentation model; the coordinates of the top-left corner of the bounding box are... The coordinates of the lower right corner are .

[0075] It should be noted that by determining the minimum and maximum coordinates of all positive prompts, a rectangular frame enclosing all positive prompts can be calculated.

[0076] S204. Input the standard workpiece image, the target workpiece image, the set of positive prompt points, the set of negative prompt points, and the prompt box into the image segmentation model to obtain the preliminary segmentation mask.

[0077] In some implementations, the image segmentation model includes: an image encoder, a cue encoder, and a mask decoder;

[0078] The standard workpiece image and the target workpiece image are processed by an image encoder to obtain the image feature tensor;

[0079] The prompt encoder processes the set of positive prompt points, the set of negative prompt points, and the prompt box to obtain the prompt feature tensor;

[0080] The image feature tensor and the cue feature tensor are fused and decoded by a mask decoder to obtain a preliminary segmentation mask.

[0081] It should be noted that the image encoder is a Vision Transformer (ViT), which maps the input image to an image feature space to extract overall image features. The cue encoding module uses positional encoding to represent the input cue information (including dots and bounding boxes), thus obtaining cue features. The mask decoding module is responsible for fusing the two feature embeddings from the image encoding and cue encoding, using the Transformer decoder to generate a high-quality segmentation mask, and ensuring the accuracy and alignment of the mask through upsampling and quality prediction; positive cue dots and cue boxes provide the model with clear guidance on the region to be segmented, while negative cue dots help the model identify regions irrelevant to the target object, thereby improving segmentation accuracy.

[0082] For example, the image segmentation model uses the SAM model, whose core capability is to generate high-quality segmentation masks based on user-provided prompts (points, boxes, masks, or text).

[0083] S205. Based on the mask of the standard workpiece image, the false positive areas in the preliminary segmentation mask are removed to obtain the foreground mask.

[0084] In some implementations, the method for obtaining the foreground mask includes:

[0085] The initial segmentation mask is divided and labeled to obtain several foreground blocks; the feature set of the standard foreground blocks is matched with the features corresponding to the several foreground blocks, and the matched foreground blocks are then filtered.

[0086] Determine whether some or all pixels of the filtered foreground block are located within the mask background area of ​​the standard workpiece image; if yes, the area is determined to be a false positive and is removed at the corresponding position in the initial segmentation mask; if no, the corresponding area is retained, and the false positive areas are removed by filtering block by block to obtain the foreground mask.

[0087] S206. The workpiece edge is obtained by extracting the contour of the foreground mask and processing it with a polygon fitting algorithm.

[0088] In some implementations, the method for obtaining the workpiece edge includes:

[0089] An initial set of contours is obtained by performing edge detection on the foreground mask using a contour extraction algorithm.

[0090] The initial contour set is simplified and fitted using a polygon approximation algorithm to obtain an optimized contour set;

[0091] By adapting the optimized contour set to the process through path point generation rules, a three-axis trajectory path point set is obtained.

[0092] The workpiece edge detection results are obtained by integrating information through data structure encapsulation based on the three-axis trajectory path point set and the optimized contour set.

[0093] The workpiece edge detection results are input to the three-axis machine control module to perform subsequent operations such as grinding, deburring, and edge cutting, ensuring the accuracy and consistency of edge processing tasks.

[0094] For example, the contour extraction algorithm is the OpenCV's findContours() function; the polygon approximation algorithm is the approxPolyDP() function; and the path point generation rules are coordinate system transformation and path smoothing strategies.

[0095] Based on the above technical solutions, this application provides a training-free workpiece edge detection method for three-axis machines. By relying solely on a single standard image for training-free segmentation, the requirement for a large amount of labeled data and the corresponding labeling and training costs are reduced. Simultaneously, prior knowledge of industrial data is combined with a pre-trained visual model for bidirectional feature matching, and a cue point sampling strategy based on location clustering is designed to automatically generate the cue input required by the image segmentation model, significantly improving the automation level of the segmentation process. Furthermore, the initial segmentation mask is refined through a feature re-matching module, effectively eliminating false positive pixels and significantly improving segmentation accuracy, especially when dealing with blurred boundaries. In addition, this method fully utilizes the capabilities of a general feature extraction model, enhancing the model's adaptability and stability in diverse industrial scenarios, thereby significantly improving overall generalization performance and application effectiveness.

[0096] In one possible implementation of this application embodiment, the above-mentioned S201 can be implemented through the following steps, which will be described in detail below:

[0097] The standard feature set includes: standard foreground block feature set, standard background block feature set, and standard sub-block feature set;

[0098] The standard sub-block feature set is obtained by automatically dividing and extracting features at the image block level from the standard workpiece image input feature extractor.

[0099] In some implementations, the standard sub-block feature set is: ;in, ; The number of blocks in a standard workpiece image , representing the feature embedding dimension of the feature extractor. The feature extractor is the DINOv2 model.

[0100] Methods for obtaining standard foreground block feature sets and standard background block feature sets include:

[0101] The standard workpiece image is divided into a mask grid, and the divided image is classified, labeled and feature extracted according to the position of the mask in the standard workpiece image to obtain the standard foreground block feature set and the standard background block feature set.

[0102] It should be noted that when some or all of the pixels of the segmented image are located within the preset mask foreground area, the segmented image is marked as a foreground block; when the segmented image does not intersect with the preset mask foreground area, the segmented image is marked as a background block; the size and stride of the image block can be adjusted according to application requirements to ensure coverage of important image information.

[0103] For example, to facilitate correspondence with the features output by the DINOv2 model, the block size is set to 14×14.

[0104] In one possible implementation of this application embodiment, the above-mentioned S202 can be implemented by S11, S12, S13, S14 and S15, which are described in detail below:

[0105] S11. Perform forward matching based on the standard foreground block feature set and the block-level features corresponding to several target blocks, and filter the matched target blocks to obtain the first target block set.

[0106] In some implementations, the standard foreground block set... Feature extraction is performed on each standard foreground block to obtain the standard foreground block feature set. ;in, Indicates the number of standard foreground blocks. Represents a set Index in;

[0107] Through calculation formula Calculate standard foreground block features Block-level features The similarity between them is used to obtain the forward similarity matrix; where, ;

[0108] Based on the forward similarity matrix, nearest neighbor matching is performed on the block-level features corresponding to several target blocks for the standard foreground block features, to obtain the first target block set in the target workpiece image. ,in, , Represents a set The index in.

[0109] It should be noted that, due to the low contrast between the foreground and background of the workpiece image, simple forward matching may produce outliers, resulting in inaccurate segmentation results; therefore, outliers are removed through reverse matching.

[0110] S12. Perform reverse matching based on the features corresponding to the standard sub-block feature set and the first target block set, and filter the matched first target block set; filter the filtered first target block set again based on the foreground mask of the standard workpiece image, and retain the first target blocks located in the foreground area to obtain the second target block set.

[0111] In some implementations, the first target block set Feature extraction is performed on each first target block to obtain the features corresponding to the first target block set. ;

[0112] Through calculation formula Calculate the features of standard sub-blocks Features corresponding to the first target block The similarity between them is used to obtain the inverse similarity matrix; where, ;

[0113] Based on the inverse similarity matrix, nearest neighbor matching is performed on the features of the first target block in the standard sub-block feature set to obtain the standard matching block set in the standard workpiece image. ,in, , Represents a set Index in;

[0114] Determine whether some or all pixels of an image block in the standard matching block set are located within the masked foreground region of the standard workpiece image; if yes, retain the corresponding first target block; otherwise, remove the corresponding first target block; construct a second target block set by determining each block individually. ;in N represents the number of second target blocks to be retained.

[0115] S13. Perform self-matching between the features corresponding to the second target block set to obtain a similarity matrix; calculate the average adjacent distance of each block in the similarity matrix and then calculate its mean to obtain the overall average distance; filter the second target blocks whose average adjacent distance is lower than the overall average distance to obtain several filtered target blocks; obtain a positive prompt point candidate set by calculating the center point of each filtered target block.

[0116] In some implementations, through calculation formulas Calculate the second target block set The similarity matrix between each second target block; where ;

[0117] Through calculation formula Calculate several average neighbor distances; and calculate the average of these average neighbor distances to obtain the overall average distance. ;

[0118] when The second target blocks in the second target block set corresponding to distances less than the overall average distance are retained to obtain the filtered target block set. ,in This represents the number of target blocks to retain for filtering.

[0119] By calculating the center point of each filter target block in the filter target block set, the candidate set of positive prompt points is obtained. ,in This is the number of target blocks to retain.

[0120] It should be noted that since the internal structure of the same component will exhibit relatively consistent characteristics, outlier filtering is performed on the second target block set through self-matching.

[0121] when At that time, appropriately reduce The screening criteria were relaxed to allow more relatively suitable second target blocks to be retained. Further screening was then conducted in conjunction with other constraints. The rationality of the image block division and feature extraction algorithms was also checked, and the feature matching strategy was re-optimized.

[0122] S14. Perform K-means clustering on the candidate set of positive prompts to obtain the set of positive prompts.

[0123] In some implementations, performing K-means clustering on the candidate set of positive prompts includes:

[0124] Based on the number of target blocks in the candidate set of positive prompts, the range of K values ​​for the number of candidate clusters is determined, and K-means clustering is performed traversing this range to obtain preliminary clustering results.

[0125] Based on the preliminary clustering results, the calculation formula was used. The contour coefficients corresponding to each K value are obtained. ;in, It is the first The average distance from a point to other points within its cluster. It is the first The average distance from a point to all points in the nearest cluster;

[0126] Iterate through several silhouette coefficients and select the optimal number of clusters corresponding to the largest silhouette coefficient. ;

[0127] Based on the optimal number of clusters K-means clustering was performed on the candidate set of positive prompts to obtain Cluster center points , as a set of positive prompts.

[0128] S15. Replace the standard foreground block feature set in step S11 with the standard background block feature set, and repeat steps S11 to S14 to obtain the negative cue point set.

[0129] It should be noted that all features were obtained through a feature extractor.

[0130] The foregoing mainly describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as an electronic device, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0131] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0132] When using integrated units, Figure 3 A possible structural schematic diagram of the electronic device (referred to as electronic device 30) involved in the above embodiments is shown. The electronic device 30 includes a processing unit 301 and a communication unit 302, and may also include a storage unit 303. Figure 3 The structural diagram shown can be used to illustrate the structure of the electronic device involved in the above embodiments.

[0133] when Figure 3The schematic diagram shown is used to illustrate the structure of the electronic device involved in the above embodiments. The processing unit 301 is used to control and manage the operation of the electronic device, the communication unit 302 is used for the electronic device to communicate with other devices, and the storage unit 303 is used to store the program code and data of the electronic device.

[0134] For example, communication unit 302 is used to acquire a target workpiece image, a standard workpiece image, and a corresponding standard feature set; input the target workpiece image to a feature extractor for image segmentation and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein, the standard feature set is obtained by segmenting and marking the standard workpiece image and extracting block-level features;

[0135] Processing unit 301 is used to filter several target blocks according to a standard feature set and a target block feature set to obtain several filtered target blocks; cluster the several filtered target blocks to obtain a set of positive prompt points and a set of negative prompt points; define the target range according to the set of positive prompt points to obtain a prompt box; input the standard workpiece image, the target workpiece image, the set of positive prompt points, the set of negative prompt points, and the prompt box into the image segmentation model to obtain a preliminary segmentation mask; remove false positive regions in the preliminary segmentation mask according to the mask of the standard workpiece image to obtain a foreground mask; and obtain the workpiece edge by performing contour extraction and polygon fitting algorithm on the edge of the foreground mask.

[0136] The processing unit 301 can be a processor or a controller, and the communication unit 302 can be a communication interface, transceiver, transceiver circuit, transceiver device, etc. The term "communication interface" is a general term and may include one or more interfaces. The storage unit 303 can be a memory. When the electronic device 30 is a chip, the processing unit 301 can be a processor or a controller, and the communication unit 302 can be an input interface and / or an output interface, pins, or circuits, etc. The storage unit 303 can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip (e.g., read-only memory (ROM), random access memory (RAM, etc.).

[0137] The communication unit can also be called a transceiver unit. The antenna and control circuit with transceiver functions in the electronic device 30 can be considered as the communication unit 302 of the electronic device 30, and the processor with processing functions can be considered as the processing unit 301 of the electronic device 30. Optionally, the device in the communication unit 302 used to implement the receiving function can be considered as a communication unit. The communication unit is used to execute the receiving steps in the embodiments of this application, and the communication unit can be a receiver, a receiver circuit, etc. The device in the communication unit 302 used to implement the transmitting function can be considered as a transmitting unit. The transmitting unit is used to execute the transmitting steps in the embodiments of this application, and the transmitting unit can be a transmitter, a transmitter, a transmitting circuit, etc.

[0138] Figure 3 If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0139] Figure 3 The units in the process can also be called modules; for example, a processing unit can be called a processing module.

[0140] This application also provides a hardware structure diagram of an electronic device (denoted as electronic device 40), see [link to diagram]. Figure 4 The electronic device 40 includes a processor 401, and optionally, a memory 402 connected to the processor 401.

[0141] In the first possible implementation, see Figure 4 The electronic device 40 also includes a transceiver 403. The processor 401, memory 402, and transceiver 403 are connected via a bus. The transceiver 403 is used to communicate with other devices or communication networks. Optionally, the transceiver 403 may include a transmitter and a receiver. The device in the transceiver 403 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 403 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0142] Based on the first possible implementation method Figure 4 The structural diagram shown can be used to illustrate the structure of the electronic device involved in the above embodiments.

[0143] in, Figure 4 This can also be illustrated by a system chip in an electronic device. In this case, the actions performed by the aforementioned electronic device can be implemented by this system chip; the specific actions performed can be found above and will not be repeated here.

[0144] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0145] The processor in this application may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a separate semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (System-on-a-Chip), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0146] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto.

[0147] This application also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0148] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0149] This application also provides a chip including a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0150] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0151] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0152] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A training-free workpiece edge detection method for a three-axis machine, characterized by, The method comprises the following steps: Obtain target workpiece image, standard workpiece image and corresponding standard feature set; Input the target workpiece image into the feature extractor to perform image division and block-level feature extraction, and obtain a plurality of target blocks and corresponding block-level features; wherein the standard feature set is obtained by dividing and marking the standard workpiece image and block-level feature extraction; Screen the plurality of target blocks according to the standard feature set and the target block feature set to obtain a plurality of screened target blocks; cluster the plurality of screened target blocks to obtain a positive prompt point set and a negative prompt point set; Frame the target range according to the positive prompt point set to obtain a prompt box; Input the standard workpiece image, the target workpiece image, the positive prompt point set, the negative prompt point set and the prompt box into an image segmentation model to obtain a preliminary segmentation mask; Remove false positive regions in the preliminary segmentation mask according to the mask of the standard workpiece image to obtain a foreground mask; Obtain the workpiece edge by performing contour extraction and polygon fitting algorithm on the edge of the foreground mask.

2. The method of claim 1, wherein, The standard feature set comprises a standard foreground block feature set, a standard background block feature set and a standard sub-block feature set. The standard sub-block feature set is obtained by inputting the standard workpiece image into the feature extractor.

3. The method of claim 2, wherein, The method for obtaining the standard foreground block feature set and the standard background block feature set comprises: Divide the standard workpiece image into a mask grid, and classify and mark the divided image according to the position of the mask in the standard workpiece image to obtain the standard foreground block feature set and the standard background block feature set.

4. The method according to claim 1 or 2, characterized in that, The method for obtaining the positive prompt point set and the negative prompt point set comprises: S11, forward match the standard foreground block feature set with the block-level features corresponding to the plurality of target blocks, and screen the plurality of target blocks after matching to obtain a first target block set; S12, reverse match the standard sub-block feature set with the features corresponding to the first target block set, and screen the first target block set after matching; re-screen the screened first target block set according to the foreground mask of the standard workpiece image, and retain the first target block located in the foreground region to obtain a second target block set; S13, self-match the features corresponding to the second target block set to obtain a similarity matrix; calculate the average adjacent distance of each block in the similarity matrix, and then obtain the average value to obtain the overall average distance; screen the second target block with the average adjacent distance lower than the overall average distance to obtain a plurality of screened target blocks; and obtain a positive prompt point candidate set by calculating the center point of each screened target block; S14, K-means cluster the positive prompt point candidate set to obtain a positive prompt point set; S15, replace the standard foreground block feature set in step S11 with the standard background block feature set; repeat steps S11 to S14 to obtain a negative prompt point set.

5. The method of claim 4, wherein, The K-means clustering of the positive prompt point candidate set comprises: Estimate the number of target blocks based on the positive prompt point candidate set, determine the K value range of the candidate cluster, and traverse the range to perform K-means clustering to obtain a preliminary clustering result; According to the preliminary clustering result, the profile coefficient corresponding to each K value is obtained by calculation formula ; wherein, ; wherein, is the average distance from the first point to other points in the cluster where the first point is located, is the average distance from the first point to all points in the nearest cluster; and is the average distance from the first point to other points in the cluster where the first point is located, is the average distance from the first point to all points in the nearest cluster. Iterate several profile coefficients to screen the optimal clustering cluster number corresponding to the maximum profile coefficient; According to the optimal clustering cluster number, the positive prompt point candidate set is further clustered by K-means to obtain a positive prompt point set.

6. The method of claim 4, wherein, The prompt box acquisition method comprises: The positive prompt point set is traversed to obtain a minimum positive prompt point coordinate and a maximum positive prompt point coordinate; wherein the minimum positive prompt point coordinate is , and the maximum positive prompt point coordinate is ; wherein ; According to the minimum positive prompt point coordinate and the maximum positive prompt point coordinate row calculation, an outer bounding box is obtained; the outer bounding box is taken as a prompt box of an image segmentation model; wherein the upper left corner coordinate of the outer bounding box is , and the lower right corner coordinate is .

7. The method of claim 6, wherein, The image segmentation model comprises: an image encoder, a prompt encoder and a mask decoder; The standard workpiece image and the target workpiece image are processed by the image encoder to obtain an image feature tensor; The positive prompt point set, the negative prompt point set and the prompt box are processed by the prompt encoder to obtain a prompt feature tensor; The image feature tensor and the prompt feature tensor are fused and decoded by the mask decoder to obtain a preliminary segmentation mask.

8. The method of claim 7, wherein, The foreground mask acquisition method comprises: The preliminary segmentation mask is divided and labeled to obtain several foreground blocks; the standard foreground block feature set is matched with the features corresponding to the several foreground blocks, and the matched several foreground blocks are screened; It is judged whether part or all of the pixels of the screened foreground blocks are located in the mask background area of the standard workpiece image; if yes, it is determined that the region is false positive, and the corresponding position of the preliminary segmentation mask is removed; otherwise, the corresponding region is retained, and the false positive region is removed by block-by-block screening to obtain a foreground mask.

9. The method of claim 8, wherein, The workpiece edge acquisition method comprises: An edge detection is performed on the foreground mask by a contour extraction algorithm to obtain an initial contour set; An optimization contour set is obtained by simplifying and fitting the initial contour set by a polygon approximation algorithm; A three-axis trajectory path point set is obtained by process adaptation of the optimization contour set by a path point generation rule; A workpiece edge detection result is obtained by information integration of the three-axis trajectory path point set and the optimization contour set by data structure packaging.

10. An electronic device, comprising: It comprises: A communication unit and a processing unit; The communication unit is configured to acquire a target workpiece image, a standard workpiece image and a corresponding standard feature set; The target workpiece image is input to a feature extractor for image division and block-level feature extraction to obtain several target blocks and corresponding block-level features; wherein the standard feature set is obtained by dividing and labeling the standard workpiece image and block-level feature extraction; The processing unit is configured to screen the several target blocks according to the standard feature set and the target block feature set to obtain several screened target blocks; cluster the several screened target blocks to obtain a positive prompt point set and a negative prompt point set; The target range is framed according to the positive prompt point set to obtain a prompt box; the standard workpiece image, the target workpiece image, the positive prompt point set, the negative prompt point set and the prompt box are input into an image segmentation model to obtain a preliminary segmentation mask; the false positive region in the preliminary segmentation mask is removed according to the mask of the standard workpiece image to obtain a foreground mask; the edge of the foreground mask is extracted by a contour extraction algorithm and a polygon fitting algorithm to obtain a workpiece edge.

Images