Metal welding seam promptable segmentation method and system based on semantic and prototype matching fusion

Abstract

The invention belongs to the technical field, and particularly relates to a metal welding seam promptable segmentation method and system based on semantic and prototype matching fusion. Obtaining a query image of the surface of the to-be-segmented metal material; inputting the query image into a trained semantic branch network for forward reasoning, and outputting a binary classification semantic logarithm probability difference graph and a query feature graph of the foreground and background of the welding seam; processing the query feature map and the binary-classification semantic logarithmic probability difference map by adopting a trained core clue network to obtain a foreground core probability map, and generating a core point prompt or a core mask prompt according to the foreground core probability map; and inputting the core point prompt or the core mask prompt and the query image into a promptable segmentation model, namely outputting a weld pixel-level segmentation mask. According to the method, the pixel-level labeling requirement can be reduced, the cross-working-condition robustness can be improved, and the method is suitable for metal welding line online detection and positioning scenes.

Description

Technical Field

[0001] This invention belongs to the field of industrial visual inspection technology, specifically relating to a method and system for prompting segmentation of metal welds based on semantic and prototype matching fusion. Background Technology

[0002] Laser cutting of metal materials requires pre-positioning of weld seams, but weld seams are usually long and thin, and are prone to unclear boundaries and false textures due to interference from metal surface reflection, oxidation, and processing scratches. In actual production, there are often challenges such as limited samples and cross-condition processing. The high cost of annotation limits the amount of data, resulting in insufficient generalization ability of traditional fully supervised segmentation methods when dealing with complex textures.

[0003] In the existing technology, deep learning solutions (such as patent CN116765635B) usually rely on sufficient training data and have limited anti-interference ability against reflection and pseudo texture; solutions based on traditional image algorithms (such as patent CN105427295A) are easily affected by positioning drift and require frequent parameter adjustments to adapt to different batches of pipes, making it difficult to meet the needs of flexible production.

[0004] Furthermore, while generalized cue-based segmentation models (such as SAM) possess strong generalization capabilities, they cannot rely on human interaction in online detection. Without an effective automatic cueing mechanism, the model is highly susceptible to being misled by reflective spots or scratches. Therefore, there is an urgent need for a technical solution that can automatically generate high-precision foreground core cueing under limited sample conditions and drive the cue-based model to stably output weld seam masks. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for suggestive segmentation of metal welds based on semantic and prototype matching fusion, which can reduce the need for pixel-level annotation and improve robustness across working conditions.

[0006] To achieve the above objectives, the present invention provides a method for suggestible segmentation of metal welds based on semantic and prototype matching fusion, comprising the following steps:

[0007] S1. Obtain the query image of the surface of the metal material to be segmented; S2. Input the query image into the trained semantic branch network for forward inference, and output a binary classification semantic log probability difference map of weld foreground and background and a query feature map. S3. The trained core cue network is used to process the query feature map and the binary semantic log odds difference map to obtain the foreground core probability map, and core point hints or core mask hints are generated based on the foreground core probability map. S4. Input the core point hint or core mask hint with the query image into the hint segmentation model, that is, output the weld seam pixel-level segmentation mask.

[0008] Furthermore, in step S2, the semantic branch network is the Mask2Former semantic segmentation network, and ResNet-101 is used as the encoder; the query feature map is the C4 layer feature map of the encoder. The binary classification semantic log-probability difference map is a map showing the difference between the weld foreground channel score and the background channel score output by the semantic branch network.

[0009] Furthermore, the training of the semantic branch network includes: acquiring sample images of the surface of the metal material, constructing a labeled sample set containing pixel-level weld mask annotations; training the semantic branch network using the labeled sample set, and iteratively updating the parameters of the semantic branch network end-to-end to minimize the semantic segmentation loss function.

[0010] Furthermore, in step S3, the core clue network includes a prototype matching branch and a consistency fusion module; the prototype matching branch is used to perform learnable mapping on the query feature map and perform similarity matching with the weld prototype vector to obtain a similarity map, which is then formed into a prototype branch log odds map after temperature scaling and bias calibration; the consistency fusion module is used to perform consistency fusion between the binary classification semantic log odds difference map and the prototype branch log odds map to obtain a foreground core probability map.

[0011] Furthermore, the method for obtaining the weld prototype vector includes: acquiring several sample images of different types of metal material surfaces, and performing pixel-level weld mask annotation on each sample image to construct a support set consisting of support images and their corresponding pixel-level weld mask annotations; The encoder parameters of the trained semantic branch network are frozen. The frozen encoder is used to extract several support feature maps from the support images of the support set. Based on the pixel-level weld mask annotation, the several support feature maps are aggregated in the weld coverage area to obtain several support prototype vectors. The several support prototype vectors are aggregated and normalized to obtain weld prototype vectors and stored in a fixed manner.

[0012] Furthermore, the training of the core clue network includes: training the core clue network based on the query feature map of the labeled sample set, the binary classification semantic log odds difference map, and the pixel-level weld mask annotation; core region supervision and hard negative sample mining are introduced during the training phase. Among them, difficult negative samples are determined in the background area marked by the weld mask, and the pixel with the largest response is selected as a difficult negative sample based on at least one of the semantic branch network output, the prototype branch probability map and the foreground core probability map. At the same time, the prototype branch probability map and the foreground core probability map are constrained so that the prototype branch log probability map provides a basis for the generation of the foreground core probability map. The prototype branch probability map is obtained by mapping the prototype branch log odds map through a nonlinear activation function.

[0013] Furthermore, in step S3, when using core point cues, the top K pixel positions with the highest probability values ​​are selected from the foreground core probability map as the point cues set, and a minimum distance constraint is set to avoid excessive concentration of cues; when using core mask cues, the highest response pixel is selected from the foreground core probability map according to a preset ratio and a binary mask is formed as a mask cues.

[0014] Furthermore, the similarity matching adopts cosine similarity or normalized dot product similarity; the suggestive segmentation model is the SAM model.

[0015] The present invention also provides a metal weld seam prompting segmentation system based on semantic and prototype matching fusion, comprising: The image acquisition unit is used to acquire a query image of the surface of the metal material to be segmented; The semantic branching unit is used to input the query image into the trained semantic branching network for forward inference and output a binary semantic log probability difference map of weld foreground and background and a query feature map. The core cue network unit is used to process the query feature map and the binary semantic log odds difference map using a trained core cue network to obtain a foreground core probability map. The prompt generation unit is used to generate a core point prompt or a core mask prompt based on the foreground core probability map. The prompting segmentation unit is used to input the core point prompt or core mask prompt and the query image into the prompting segmentation model, that is, to output a weld seam pixel-level segmentation mask.

[0016] The present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the above embodiments.

[0017] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages: 1. Advantage of few samples: This invention only requires a small number of pixel-level labeled samples to complete semantic branch training and fine-tuning, and uses a small number of support sample masks to guide aggregation to obtain reusable weld prototype vectors, thereby reducing pixel-level labeling costs and improving the ability to transfer to new working conditions.

[0018] 2. Advantages of high-precision core clues: Semantic branches provide global semantic constraints, prototype matching branches provide fine-grained discrimination, and consistency fusion filters high-confidence regions shared by both branches; combined with core region supervision and difficult negative sample mining, it can suppress false responses to reflections, scratches and pseudo-textures, reduce foreground leakage and improve robustness.

[0019] 3. Advantages of high-quality masks: Under conditions of few samples, high-precision core point cues or core mask cues are generated first, and then the pre-trained segmentation model is driven to complete boundary refinement and connectivity completion, thereby obtaining a more complete and stable weld seam pixel-level segmentation mask. Attached Figure Description

[0020] Figure 1 This is a flowchart of the offline preparation phase.

[0021] Figure 2 This is a flowchart of the weld seam segmentation stage. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0023] Metal weld seams typically exhibit a long, thin strip-like structure and are affected by surface factors such as material reflection, brushed texture, oxide scale, oil stains, and scratches, resulting in unstable appearance differences between the weld seam and the background in local areas. With a limited number of pixel-level labeled samples, existing schemes relying solely on semantic segmentation are prone to foreground response diffusion into reflective pseudo-textures, leading to foreground leakage and impure generated cue points. When these impure cue points are input into a cue-enabled segmentation model (such as SAM), mis-adsorption, segmentation drift, or breakage can easily occur, making it difficult to consistently obtain high-quality pixel-level weld seam masks. Therefore, this invention provides a cue-enabled segmentation method for metal weld seams based on semantic and prototype matching fusion, as detailed below: Please see Figure 1 and 2 A method for suggestive segmentation of metal welds based on semantic and prototype matching fusion includes the following steps: S1. Obtain the query image of the surface of the metal material to be segmented; S2. Input the query image into the trained semantic branch network for forward inference, and output a binary classification semantic log probability difference map of weld foreground and background and a query feature map. S3. The trained core cue network is used to process the query feature map and the binary semantic log odds difference map to obtain the foreground core probability map, and core point hints or core mask hints are generated based on the foreground core probability map. S4. Input the core point hint or core mask hint with the query image into the hint segmentation model, that is, output the weld seam pixel-level segmentation mask.

[0024] Before performing weld segmentation, offline preparation is required, namely, obtaining reusable weld priors (i.e., weld prototype vectors), establishing semantic discrimination capabilities (semantic branch network training), and training a core clue network that can stably output foreground core clues, as detailed below: S21. Collect sample images of the metal material surface and construct a small set of pixel-level labeled samples. Label the weld seam pixel-level mask for each sample image. Preferably, the small set of pixel-level labeled samples is divided into a training subset and a test subset. The training subset is used for semantic branch training and fine-tuning, prototype construction, and core cue network training, while the test subset is only used for performance evaluation and does not participate in training and prototype construction, in order to avoid data leakage.

[0025] The metal material of this invention can be metal pipes, plates, profiles, forgings, castings, etc.

[0026] In one embodiment, the training subset consists of 20 labeled images. Five representative samples are selected from the training subset to form the support set. The support samples consist of support images and their precise corresponding weld mask labels. The test subset consists of the remaining labeled samples other than the training subset. The representative samples typically refer to weld samples that include different materials (e.g., stainless steel or carbon steel) and different weld types (e.g., thickness, inner wall or outer wall welds).

[0027] S22, the Mask2Former semantic segmentation network is used as the semantic branch network, and it is initialized with publicly available pre-trained parameters. Training or fine-tuning is performed using a small subset of labeled samples to improve convergence stability and cross-condition generalization under limited sample conditions. The semantic branch network outputs binary semantic scores for weld foreground and background, and simultaneously outputs the query feature map required for subsequent prototype matching and the core cue network. In a preferred embodiment, the semantic branch network uses ResNet-101 as the encoder, and preferably uses the feature map of encoder C4 layer as the query feature map. C, H, and W represent the number of feature map channels, feature map height, and feature map width, respectively.

[0028] In one embodiment, the training fine-tuning process of the Mask2Former semantic segmentation network includes: constructing training sample pairs based on a training subset. ,in, Image of a metallic material surface. Pixel-level weld seam mask annotations are used; during training, only the default data preprocessing procedures required for model input are used to process the input images.

[0029] The default data preprocessing procedure includes at least the following: scaling or aligning the image and mask to a preset input scale; normalizing the image according to the mean and variance corresponding to the pre-trained parameters; and aligning the mask to the network output resolution using nearest neighbor interpolation. The semantic branch network is pre-trained and initialized with parameters for both weld foreground and background outputs. End-to-end updates of the semantic branch network parameters are performed using a mini-batch iterative approach to minimize the semantic segmentation loss function. The semantic branch network outputs a weld foreground channel score for any input image. Background channel score A binary classification semantic log-probability difference map is constructed from the difference between the two values. ,in For the sigmoid function, Foreground probability of semantic branch.

[0030]

[0031]

[0032] Pixel-level binary cross-entropy loss can be expressed as:

[0033] In particular, to enhance the regional consistency of slender structural targets and suppress fractures and voids, Dice loss is further introduced:

[0034] thereby ,in These are the weighting coefficients. To prevent division by zero by extremely small constants, the preferred optimization strategy is to use an adaptive optimizer based on gradient descent to update network parameters, and to employ learning rate warm-up and decay strategies to improve convergence stability under few-shot conditions. During training, the semantic branch network is evaluated on a test subset. The evaluation metrics preferably include at least one of foreground IoU or mIoU, as well as Precision, Recall, and F1-score. The model parameters with the best evaluation performance are selected as the trained semantic branch network.

[0035] S23, under the condition of freezing the encoder parameters of the trained semantic branch network, the frozen encoder is used to extract support feature maps from the support set image, and the weld region features are aggregated according to the weld mask annotation to construct a weld prototype vector and store it in a fixed manner. Specifically, let the first... The support images for each support sample are The weld mask is marked as The support feature map output by the frozen encoder is Where C represents the number of channels supporting the feature map, This represents the pixel location of the feature map. The weld mask annotations are mapped to the supporting features. Figure 1 The scale of the result is obtained This supports mask-guided aggregation of prototype vectors, where To prevent division by zero of extremely small constants.

[0036]

[0037] For the first Support prototype vectors of each supporting sample.

[0038] Multiple support prototype vectors are aggregated to obtain weld prototype vectors, which are then stored in a fixed manner, where N is the number of support samples (preferably 5). This is a normalization operation (preferably L2 normalization). With the support set unchanged, the weld prototype vector is directly called in subsequent inference stages. Participate in similarity matching:

[0039] S31, freeze the trained semantic branch network parameters and fix the weld prototype vector. Under the given conditions, images from the training subset are used as query samples. A core cue network is trained using the binary semantic log-odds difference map output by the semantic branch and the query feature map to learn how to output "high-precision, low-spillover" core cues under limited sample constraints. The core cue network includes a prototype matching branch and a consistency fusion module. The prototype matching branch includes a mapping layer that performs a learnable mapping on the query feature map; the prototype matching branch is used to perform a learnable mapping on the query feature map. And calculate the similarity (preferably cosine similarity) between it and the weld prototype vector:

[0040] To query the feature map, For learnable mapping layer Transformed query feature map This is a similarity graph.

[0041] Introducing temperature scaling parameters With bias parameters Similarity is calibrated to form a prototype branch log-odds plot. ,in, and The trainable parameters for the prototype matching branch are used to scale and translate the similarity map to form the log-odds map of the prototype branch. This is the prototype branch foreground probability diagram.

[0042]

[0043] The consistency fusion module is used to perform consistency fusion between the semantic branch foreground probability map and the prototype branch foreground probability map to obtain the foreground core probability map:

[0044] During the training phase, core region supervision and hard negative sample mining must be introduced, and constraints must be imposed on the prototype branch output and the foreground core probability map at the same time. This will enable the prototype branch log odds map to provide fine-grained discrimination criteria for the generation of the foreground core probability map, so as to obtain a trained core cue network that can stably output high-precision core cues.

[0045] Specifically, the weld mask of the training samples is labeled. Corrosion is performed to obtain a mask for the core area of ​​the weld. and define the set of pixels in the core region. Background pixel set .

[0046] Firstly, supervision of core areas: Limit the location of candidate positive samples, and within this set, based on and Confidence level selection Positive sample subset and at the location of the positive sample subset and Imposing a foreground constraint that "approaches 1" is preferred, and binary cross-entropy loss is preferred:

[0047]

[0048] The core foreground probability map in the positive sample subset Binary cross-entropy loss on For the prototype branch probability graph in the positive sample subset Binary cross-entropy loss on the above.

[0049] Secondly: Hard negative sample mining: within the background pixel set Based on the semantic branch network output, the prototype branch probability map, and the foreground core probability map, high-response pixels are selected as the set of difficult negative samples:

[0050]

[0051]

[0052] in, Indicates according to proportion Select the set of pixels with the largest response, and take the union of the three sets to obtain the unified hard negative sample set:

[0053] And for the difficult negative samples At least one of the prototype branch probability map and the foreground core probability map at the location is subject to suppression constraints, preferably achieved by minimizing the average response of high-response pixels, in order to reduce foreground leakage and false texture response.

[0054]

[0055] For the core foreground map on the difficult negative sample set The average response suppresses loss. For the prototype branch probability graph on the difficult negative sample set The average response suppresses loss.

[0056] Considering the above constraints, the training objective of the core cue network can be expressed as a weighted sum, where... , , These are the weighting coefficients.

[0057]

[0058] During training, only the learnable mapping parameters in the core cue network are updated. and The parameters are calibrated, and the parameters of the semantic branch network are frozen. To improve training stability, the gradients of the core cue network can be pruned, and an adaptive optimizer can be used to update the parameters. After completing the above training, the trained core cue network for inference is obtained.

[0059] S41, Load the pre-trained parameters of the SAM model to generate a pixel-level segmentation mask for the weld seam based on core point cues or core mask cues in the subsequent weld seam segmentation stage.

[0060] like Figure 2 As shown, the weld segmentation stage is used to generate core hints and output weld masks for each query image, including: S1, obtain the query image of the metal material surface.

[0061] S2, input the query image into the trained semantic branch network for forward inference to obtain the binary classification semantic log-odds difference map. With query feature map And calculate the semantic probability graph. The binary classification semantic log-probability difference map is used to characterize the discrimination strength of a pixel belonging to the weld foreground relative to the background. S3, invoke the trained core cue network to process the query feature map and the binary classification semantic log-odds difference map: the prototype matching branch processes the query feature map. Perform learnable mapping and the fixed-storage weld prototype vector Calculate similarity Then, after temperature scaling and bias calibration, the result is obtained. and ; The foreground core probability map is calculated by the consistency fusion module. And generate hints based on the foreground core probability map: when using core point hints, from The system selects the top K pixels with the highest probability values ​​as the cue point set, and a minimum distance constraint can be set to avoid over-concentration of cue points; when using a core mask cue, the cue points can be selected from a set of preset proportions. The highest-response pixel is selected and a binary mask is formed as a mask cue. In a preferred embodiment, the binary classification semantic log-odds difference map and the prototype branch log-odds map are first mapped by sigmoid to obtain a semantic probability map and a prototype probability map, respectively. Then, they are multiplied at the pixel position to obtain a fusion probability. The fusion probability is used as the foreground core probability map to highlight the weld core region with "high confidence of both branches" and suppress the branch inconsistency region. S4. Input the point hints or mask hints and the query image into the hint segmentation model to obtain the weld seam pixel-level segmentation mask output. When the hint segmentation model outputs multiple candidate masks, the candidate mask with the highest model confidence or the highest consistency with the hints is preferably selected as the final weld seam mask. The weld seam pixel-level segmentation mask is output for industrial application scenarios such as weld seam positioning, processing avoidance, or quality inspection.

[0062] In particular, the present invention also provides a weld-free rejection mode: when the high confidence pixel statistics of the core probability map are lower than the threshold, or the overlap ratio of the semantic branch and the prototype branch in the high confidence region is lower than the threshold, or the length or elongation of the connected component formed by the high confidence pixels does not satisfy the weld prior, the query image is determined to be weld-free, and an empty mask is output or the inference of the suggestible segmentation model is skipped.

[0063] The present invention also provides a metal weld seam prompting segmentation system based on semantic and prototype matching fusion, comprising: The image acquisition unit is used to acquire a query image of the surface of the metal material to be segmented; The semantic branching unit is used to input the query image into the trained semantic branching network for forward inference and output a binary semantic log probability difference map of weld foreground and background and a query feature map. The core cue network unit is used to process the query feature map and the binary semantic log odds difference map using a trained core cue network to obtain a foreground core probability map. The prompt generation unit is used to generate a core point prompt or a core mask prompt based on the foreground core probability map. The prompting segmentation unit is used to input the core point prompt or core mask prompt and the query image into the prompting segmentation model, that is, to output a weld seam pixel-level segmentation mask.

[0064] In summary, this invention initializes and trains the Mask2Former semantic branch network with pre-trained parameters, enabling it to output binary semantic results for weld foreground and background, as well as a query feature map, on the input image. After freezing its encoder, the weld region features of the support feature map are aggregated based on the support set weld mask annotation to construct a weld prototype vector. A core cue network is then trained, allowing the prototype branch log-odds map to provide fine-grained discrimination criteria on the query feature map and serve the generation of the foreground core probability map. During training, core region supervision and hard negative sample mining are introduced to suppress false responses from reflections, scratches, and pseudo-textures. In the weld segmentation stage, the semantic branch output and query feature map are obtained from the query image. The prototype matching branch performs similarity matching with the weld prototype vector on the query feature map to obtain the prototype branch output. Consistency fusion generates the foreground core probability map and produces point cues or mask cues, thereby driving the cue-enabled segmentation model to output a pixel-level weld segmentation mask. This invention reduces the need for pixel-level annotation and improves robustness across different working conditions, making it suitable for online detection and localization of metal welds.

[0065] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for suggestive segmentation of metal welds based on semantic and prototype matching fusion, characterized in that, Includes the following steps: S1. Obtain the query image of the surface of the metal material to be segmented; S2. Input the query image into the trained semantic branch network for forward inference, and output a binary classification semantic log probability difference map of weld foreground and background and a query feature map. S3. The trained core cue network is used to process the query feature map and the binary semantic log odds difference map to obtain the foreground core probability map, and core point hints or core mask hints are generated based on the foreground core probability map. S4. Input the core point hint or core mask hint with the query image into the hint segmentation model, that is, output the weld seam pixel-level segmentation mask.

2. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 1, characterized in that, In step S2, the semantic branch network is the Mask2Former semantic segmentation network, and ResNet-101 is used as the encoder; the query feature map is the C4 layer feature map of the encoder. The binary classification semantic log-probability difference map is a map showing the difference between the weld foreground channel score and the background channel score output by the semantic branch network.

3. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 2, characterized in that, The training of the semantic branch network includes: acquiring sample images of the surface of metal materials and constructing a labeled sample set containing pixel-level weld mask annotations; training the semantic branch network using the labeled sample set and iteratively updating the parameters of the semantic branch network end-to-end to minimize the semantic segmentation loss function.

4. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 3, characterized in that, In step S3, the core clue network includes a prototype matching branch and a consistency fusion module; the prototype matching branch is used to perform learnable mapping on the query feature map and perform similarity matching with the weld prototype vector to obtain a similarity map, which is then formed into a prototype branch log odds map after temperature scaling and bias calibration; the consistency fusion module is used to perform consistency fusion between the binary classification semantic log odds difference map and the prototype branch log odds map to obtain a foreground core probability map.

5. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 4, characterized in that, The method for obtaining the weld prototype vector includes: acquiring several sample images of different types of metal material surfaces, and performing pixel-level weld mask annotation on each sample image to construct a support set consisting of support images and their corresponding pixel-level weld mask annotations; The encoder parameters of the trained semantic branch network are frozen. The frozen encoder is used to extract several support feature maps from the support images of the support set. Based on the pixel-level weld mask annotation, the several support feature maps are aggregated in the weld coverage area to obtain several support prototype vectors. The several support prototype vectors are aggregated and normalized to obtain weld prototype vectors and stored in a fixed manner.

6. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 4, characterized in that, The training of the core clue network includes: training the core clue network based on the query feature map, binary classification semantic log odds difference map, and pixel-level weld mask annotation based on the labeled sample set; core region supervision and hard negative sample mining are introduced during the training phase. Among them, difficult negative samples are determined in the background area marked by the weld mask, and the pixel with the largest response is selected as a difficult negative sample based on at least one of the semantic branch network output, the prototype branch probability map and the foreground core probability map. At the same time, the prototype branch probability map and the foreground core probability map are constrained so that the prototype branch log probability map provides a basis for the generation of the foreground core probability map. The prototype branch probability map is obtained by mapping the prototype branch log odds map through a nonlinear activation function.

7. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 1, characterized in that, In step S3, when using core point cues, the top K pixel positions with the highest probability values ​​are selected from the foreground core probability map as the point cues set, and a minimum distance constraint is set to avoid excessive concentration of cues; when using core mask cues, the highest response pixel is selected from the foreground core probability map according to a preset ratio and a binary mask is formed as the mask cues.

8. The method for suggestive segmentation of metal welds based on semantic and prototype matching fusion according to claim 4, characterized in that, The similarity matching uses cosine similarity or normalized dot product similarity; the suggestive segmentation model is the SAM model.

9. A metal weld seam prompting and segmentation system based on semantic and prototype matching fusion, characterized in that, include: The image acquisition unit is used to acquire a query image of the surface of the metal material to be segmented; The semantic branching unit is used to input the query image into the trained semantic branching network for forward inference and output a binary semantic log probability difference map of weld foreground and background and a query feature map. The core cue network unit is used to process the query feature map and the binary semantic log odds difference map using a trained core cue network to obtain a foreground core probability map. The prompt generation unit is used to generate a core point prompt or a core mask prompt based on the foreground core probability map. The prompting segmentation unit is used to input the core point prompt or core mask prompt and the query image into the prompting segmentation model, that is, to output a weld seam pixel-level segmentation mask.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 8.

Images