Additive manufacturing detection method and system based on target detection and semantic segmentation fusion

Abstract

This invention belongs to the field of additive manufacturing technology. To address the problem of inaccurate molten pool state detection in existing methods, it proposes an additive manufacturing detection method and system based on the fusion of object detection and semantic segmentation. The method involves using an object detection model to locate the molten pool region in the molten pool image; inputting the molten pool bounding box into a semantic segmentation model to obtain the molten pool's geometric dimensions; introducing a consistency loss function during the training of both models; utilizing a bidirectional cross-task attention mechanism to perform deep interaction and complementarity between the aligned features for detection and segmentation, resulting in fused features; generating a spatial weight map using the semantic segmentation model, introducing channel attention and combining it with the fused features to obtain fused shared features; and constructing a consistency loss function based on the consistency of the predictions made by the detection head and the segmentation head for the fused shared features. This invention achieves more accurate identification of the molten pool state.

Description

Technical Field

[0001] This invention belongs to the field of additive manufacturing technology, and in particular relates to an additive manufacturing detection method and system based on the fusion of target detection and semantic segmentation. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Laser additive manufacturing (such as direct laser deposition (L-DED) and selective laser melting (SLM) technologies have been widely applied in key fields such as aerospace, high-end equipment, and medical implants due to their advantages such as high degree of freedom in forming, high material utilization, and ability to achieve integrated manufacturing of complex structures.

[0004] In laser additive manufacturing, the molten pool, as the core region for material melting and solidification, directly reflects the dynamic changes in process parameters (such as laser power, scanning speed, and powder feed rate) through its morphological characteristics (including location, boundary contour, and size distribution). It is also a key characterizer for evaluating the internal quality of the formed part (such as density and grain size) and predicting the formation of defects (such as porosity, cracks, and lack of fusion). Therefore, achieving high-precision and robust online identification of the molten pool state is a core technological requirement for ensuring the stability of the laser additive manufacturing process and the reliability of the products.

[0005] Existing detection methods mostly employ either object detection or semantic segmentation alone. When object detection is used alone, although it can quickly locate the approximate area of ​​the molten pool in the image (e.g., through bounding box annotation), it cannot obtain the fine morphology of the molten pool edge (e.g., irregular boundaries, local protrusions / concavities), resulting in insufficient quantization accuracy of key features such as molten pool size and shape. When semantic segmentation is used alone, although it can achieve pixel-level molten pool region division and thus restore boundary details, such algorithms are highly sensitive to interference factors in the image (e.g., strong reflections from laser spots, metal powder splashes, background noise), and lack robustness. In complex manufacturing environments, problems such as segmentation region distortion and boundary misjudgment are prone to occur.

[0006] To overcome the shortcomings of the aforementioned single algorithm, the current mainstream technical solutions adopt a detection-segmentation combined architecture, which can be divided into two types: serial and parallel. The serial approach first outputs the location information of the melt pool (such as bounding box and region of interest ROI) through the detection network, and then inputs the sub-image corresponding to the location into the segmentation network for pixel-level analysis. The parallel approach performs detection and segmentation tasks separately through a dual-branch network, and then performs a simple fusion of the two types of output results. However, these combined solutions still suffer from the following key technical problems, limiting their application effectiveness in online monitoring scenarios of laser additive manufacturing: First, the interaction between the detection network and the segmentation network relies heavily on low-dimensional geometric information such as bounding box coordinates and ROI sub-regions, failing to achieve the sharing of high-dimensional semantic features (such as the gray-scale distribution pattern and texture features of the molten pool region) and fine-grained detail features. This results in a low degree of fusion between the detection and segmentation results, easily leading to deviations between the segmented region and the actual molten pool position, and mismatches between boundary details and the overall position. Second, the detection network focuses on the rapid localization of the target region, typically employing a shallower network structure and a larger receptive field. In contrast, the segmentation network focuses on the accurate restoration of boundary details, requiring a deeper network structure and a finer-grained feature scale. The significant differences between the two types of networks in terms of structural depth, feature scale, and task objectives make simple network splicing or feature stacking prone to feature space conflicts, leading to semantic shifts and gradient competition, affecting the stability of model training and the final recognition accuracy. Third, existing combined solutions generally rely on post-processing alignment operations (such as ROIAlign, Mask...). Cutting and coordinate mapping are used to correct the deviation between detection and segmentation results. These operations significantly increase the inference latency and computational load of the model. The process control of laser additive manufacturing requires millisecond-level feedback on the state of the molten pool. Existing solutions lack real-time performance and cannot meet the needs of online monitoring scenarios.

[0007] In summary, existing methods for molten pool state recognition combining detection and segmentation suffer from insufficient cross-task information interaction, inconsistent feature spaces, and low inference efficiency. These limitations prevent the full exploitation of the spatiotemporal correlations of molten pool morphology, resulting in a tradeoff between accuracy, robustness, and real-time performance in molten pool state recognition. Consequently, these factors affect the timeliness and accuracy of process parameter control, limiting the further application of laser additive manufacturing technology in the production of high-precision, high-reliability products. Therefore, developing a technical solution capable of simultaneously achieving high-precision molten pool positioning, high-quality boundary segmentation, and highly robust recognition has become a pressing technical challenge in this field. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, this invention provides an additive manufacturing inspection method and system based on the fusion of target detection and semantic segmentation, which takes into account both positioning accuracy and boundary quality. It can maintain the stability of the molten pool geometry and energy field during additive manufacturing, improve the interlayer fusion quality, reduce defect generation, and achieve higher forming consistency and final part accuracy.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides an additive manufacturing inspection method based on the fusion of object detection and semantic segmentation, comprising:

[0011] Acquire images of the molten pool during additive manufacturing;

[0012] The target detection model is used to locate the molten pool region in the acquired molten pool image during the additive manufacturing process, and the molten pool bounding box and the initial state category of the molten pool are obtained.

[0013] The bounding box of the molten pool is input as the region of interest into the semantic segmentation model to obtain the geometric dimensions of the molten pool;

[0014] In the training of the object detection model and the semantic segmentation model, a consistency loss function is introduced to achieve deep interaction and complementarity between object detection and semantic segmentation; the consistency loss function is specifically determined as follows:

[0015] Feature alignment is performed on the feature maps of the backbone network of the object detection model and the backbone network of the semantic segmentation model. A bidirectional cross-task attention mechanism is used to perform deep interaction and complementarity between object detection and semantic segmentation on the aligned features to obtain fused features.

[0016] A spatial weight map is generated using a semantic segmentation model, channel attention is introduced, and fusion features are combined to obtain fused shared features;

[0017] A consistency loss function is constructed based on the consistency of predictions made by the detection head of the object detection model and the segmentation head of the semantic segmentation model for the fused shared features.

[0018] Secondly, the present invention provides an additive manufacturing inspection system based on the fusion of target detection and semantic segmentation, comprising:

[0019] The acquisition module is configured to acquire images of the molten pool during the additive manufacturing process.

[0020] The target detection module is configured to: use the target detection model to locate the molten pool region in the acquired molten pool image during the additive manufacturing process, and obtain the molten pool bounding box and the initial state category of the molten pool;

[0021] The semantic segmentation module is configured to input the bounding box of the melt pool as the region of interest into the semantic segmentation model to obtain the geometric dimensions of the melt pool.

[0022] The training module is configured to: introduce a consistency loss function during the training of the object detection model and the semantic segmentation model to achieve deep interaction and complementarity between object detection and semantic segmentation; wherein, the consistency loss function is specifically determined as follows:

[0023] Feature alignment is performed on the feature maps of the backbone network of the object detection model and the backbone network of the semantic segmentation model. A bidirectional cross-task attention mechanism is used to perform deep interaction and complementarity between object detection and semantic segmentation on the aligned features to obtain fused features.

[0024] A spatial weight map is generated using a semantic segmentation model, channel attention is introduced, and fusion features are combined to obtain fused shared features;

[0025] A consistency loss function is constructed based on the consistency of predictions made by the detection head of the object detection model and the segmentation head of the semantic segmentation model for the fused shared features.

[0026] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0027] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0028] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0029] The above one or more technical solutions have the following beneficial effects:

[0030] In this invention, by constructing a cross-network feature bridging and bidirectional cross-task attention mechanism, deep fusion between detection and segmentation is achieved, enabling bidirectional information flow across networks. This solves the problems of inconsistency and insufficient fusion between detection and segmentation at the feature level, and enables melt pool recognition to remain stable in strong noise and high dynamic scenarios, thus achieving more accurate recognition of melt pools.

[0031] In this invention, a lightweight channel attention structure is introduced to dynamically adjust the weights of detection features and segmentation features according to task requirements, thereby improving the model's adaptability under different melt pool morphology conditions.

[0032] In this invention, a spatial weight map is generated based on the high-resolution output of the segmentation network to suppress background noise, strengthen the molten pool region, and add gradient boundary enhancement so that the fused features have both high semantics and high boundary quality.

[0033] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0034] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0035] Figure 1 This is a block diagram illustrating the fusion of target detection and semantic segmentation in an embodiment of the present invention;

[0036] Figure 2 This is an overall block diagram of the additive manufacturing detection method based on the fusion of target detection and semantic segmentation in an embodiment of the present invention. Detailed Implementation

[0037] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0038] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0039] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0040] Example 1

[0041] This embodiment proposes an additive manufacturing inspection method based on the fusion of object detection and semantic segmentation, including:

[0042] Acquire images of the molten pool during additive manufacturing;

[0043] The target detection model is used to locate the molten pool region in the acquired molten pool image during the additive manufacturing process, and the molten pool bounding box and the initial state category of the molten pool are obtained.

[0044] The bounding box of the molten pool is input as the region of interest into the semantic segmentation model to obtain the geometric dimensions of the molten pool;

[0045] In the training of the object detection model and the semantic segmentation model, a consistency loss function is introduced to achieve deep interaction and complementarity between object detection and semantic segmentation; the consistency loss function is specifically determined as follows:

[0046] Feature alignment is performed on the feature maps of the backbone network of the object detection model and the backbone network of the semantic segmentation model. A bidirectional cross-task attention mechanism is used to perform deep interaction and complementarity between object detection and semantic segmentation on the aligned features to obtain fused features.

[0047] A spatial weight map is generated using a semantic segmentation model, channel attention is introduced, and fusion features are combined to obtain fused shared features;

[0048] A consistency loss function is constructed based on the consistency of predictions made by the detection head of the object detection model and the segmentation head of the semantic segmentation model for the fused shared features.

[0049] This embodiment achieves deep fusion between detection and segmentation by constructing a cross-network feature bridging and bidirectional cross-task attention mechanism, generating a shared feature representation with both high semantic expression and high boundary quality. Based on this, the molten pool state recognition and key parameter extraction are completed. Furthermore, the process parameters such as laser power, scanning speed, and powder feeding rate are automatically adjusted according to the molten pool state deviation to achieve dynamic and stable control of the forming process.

[0050] This embodiment not only solves the problems of inconsistency and insufficient fusion between detection and segmentation at the feature level, enabling molten pool identification to remain stable in high-noise and high-dynamic scenarios; but also realizes a closed-loop process control mechanism for actual equipment through state quantization, parameter mapping and adaptive feedback, which significantly improves the processing stability, forming quality and defect prevention capabilities of the additive manufacturing process.

[0051] This embodiment includes a vision module, a target detection module, a semantic segmentation module, a molten pool state recognition module, and a control unit. The vision module uses a coaxial CCD camera to monitor the molten pool; the target detection module is used for rapid localization of the molten pool region in the preprocessed image; the semantic segmentation module is used for fine segmentation of the molten pool contour, extracting the width, area, brightness level, and features of the molten pool. A molten pool state recognition module is set up to perform multi-dimensional analysis and classification of the dynamic state of the molten pool. This module uses a random forest algorithm, and the input is the parameters of the molten pool (such as width, brightness, area, etc.). The control unit uses a PID controller to provide real-time feedback adjustment to key process parameters such as laser power, scanning speed, and powder feeding rate based on the state judgment made by the molten pool state recognition module, achieving online closed-loop control. Through the closed-loop feedback mechanism between the molten pool state recognition module and the control unit, abnormal expansion, contraction, offset, and energy fluctuations of the molten pool can be detected within milliseconds and trigger process parameter corrections. The laser power can be automatically adjusted according to the deviation of the molten pool width, the scanning speed can be adjusted according to the instability of the molten pool shape, and the powder feeding rate can be optimized according to the changes in spot disturbance and spatter density, thereby achieving adaptive heat input control and morphology stabilization adjustment. Through continuous feedback optimization, this embodiment can maintain the stability of the molten pool geometry and energy field during additive manufacturing, improve the interlayer fusion quality, reduce defect generation, and achieve higher forming consistency and final part accuracy.

[0052] The following is combined Figures 1-2 The additive manufacturing inspection method based on the fusion of object detection and semantic segmentation proposed in this embodiment will be described in detail:

[0053] Step 1: Image acquisition.

[0054] This embodiment first uses a high frame rate CCD / CMOS industrial camera arranged coaxially or laterally to acquire a continuous sequence of molten pool images during the additive manufacturing process.

[0055] The molten pool image is transmitted to the preprocessing unit in real time via a high-speed cache. The preprocessing unit performs denoising, histogram enhancement, gamma correction, and brightness normalization on the molten pool image, thereby reducing the impact of interference such as spot reflection, spatter, and smoke on image quality. The stable, high signal-to-noise ratio molten pool image obtained after preprocessing provides reliable visual input for subsequent detection, segmentation, and feature fusion.

[0056] Step 2: Target detection.

[0057] This embodiment utilizes a lightweight, high-speed target detection model (such as YOLOv8) to quickly locate the molten pool region in the preprocessed molten pool image. The detection model outputs the molten pool bounding box, confidence score, and preliminary state category (stable molten pool, under-melted pool, over-melted pool, spatter interference, etc.) through multi-scale feature extraction and decoupling head structure.

[0058] The object detection module has millisecond-level inference speed, can maintain robustness during high-speed prototyping, clearly defines effective ROI regions for subsequent segmentation tasks, and provides global semantic information and location priors for the fusion module.

[0059] Step 3: Semantic segmentation.

[0060] The bounding box of the melt pool obtained by the object detection module will be used as the ROI region input to the semantic segmentation network (such as U-Net, DeepLabv3+, SegFormer), thereby achieving pixel-level parsing of the internal structure of the melt pool with less computation.

[0061] The semantic segmentation module can accurately extract the molten pool contour curve, calculate the geometric dimensions such as the molten pool width, length, and area, and obtain the pseudo-color region of temperature distribution based on multi-channel input (e.g., when using infrared / visible light joint imaging). This step provides high-resolution, fine-grained boundary representation, laying the foundation for the fusion module to enhance local features.

[0062] This embodiment constructs a complete cross-task feature interaction structure to solve problems such as information fragmentation, noise sensitivity, and blurred boundaries in traditional "detection → cropping → segmentation" schemes. Its core includes:

[0063] (1) Cross-network feature bridging structure.

[0064] A feature bridging path is constructed to connect the detection backbone and the segmentation backbone. A multi-scale feature alignment module is used to achieve consistency in spatial scale, semantic depth, and channel representation among different networks.

[0065]

[0066]

[0067] in, These are feature maps from the backbone network of the object detection model; The feature map comes from the backbone network of the semantic segmentation model; Resize is a spatial scale alignment operator that can perform upsampling or downsampling operations according to the difference in feature resolution across networks, so that the detection features and segmentation features are consistent in spatial dimension, making cross-network fusion comparable and fusionable.

[0068] (2) Two-way cross-task attention mechanism.

[0069] We propose a cross-task Cross-Attention structure that enables detection features to guide segmentation of the main melt pool region, while segmentation features can in turn enhance the detection's perception of melt pool boundaries and defect details, achieving deep interaction and complementarity between detection and segmentation.

[0070] The feature maps of the aligned object detection model backbone network are used as query and key vectors, and the feature maps of the aligned semantic segmentation model backbone network are used as value vectors. An attention mechanism is then used to obtain the mapping features from object detection to semantic segmentation. .

[0071] Specifically, detection → segmentation:

[0072]

[0073]

[0074] in, , , is a learnable linear mapping matrix, usually implemented by 1x1Conv; d is the dimension of Key and Query, used for scaling to prevent gradient explosion, and the superscript T indicates transpose.

[0075] The feature maps of the aligned semantic segmentation model backbone network are used as query and key vectors, and the feature maps of the aligned object detection model backbone network are used as value vectors. An attention mechanism is then used to obtain the mapping features from semantic segmentation to object detection. .

[0076] Segmentation → Detection:

[0077]

[0078]

[0079] The final fusion is as follows:

[0080]

[0081] in, , , α is a learnable linear mapping matrix, typically implemented by 1x1Conv; α and β are learnable or manually set fusion weights.

[0082] This step enables deep interaction and information complementarity between the detection and segmentation tasks.

[0083] (3) Spatial guidance and boundary enhancement strategies.

[0084] Generate spatial weight graph based on high-resolution output of segmentation network It is used to suppress background noise and strengthen the molten pool area;

[0085]

[0086]

[0087] Simultaneously, a gradient-based boundary enhancement module was added:

[0088]

[0089]

[0090] Among them, F s high σ represents the high-resolution features output during the decoding stage of the semantic segmentation model; σ is the Sigmoid function, outputting [0, 1]; ⊙ represents the Hadamard element-wise multiplication. For the gradient operator. F s mask γ represents the prediction result of the semantic segmentation model for each pixel; γ is the boundary enhancement coefficient.

[0091] This step enables the fused features to possess both high semantic quality and high boundary quality.

[0092] (4) Learnable adaptive fusion weights.

[0093] A lightweight channel attention structure is introduced to dynamically adjust the weights of detection and segmentation features according to task requirements:

[0094]

[0095] Achieve real-time, automatic optimization and fusion:

[0096]

[0097] in, , It consists of two learnable mappings in the SE channel attention, used to adaptively adjust the fusion weights of detection and segmentation features from a global semantic perspective; GAP stands for global average pooling. The original, unenhanced fusion features are the feature maps of the backbone network of the object detection model. Feature maps of the backbone network of the semantic segmentation model The splicing characteristics.

[0098] This step improves the model's adaptability under different melt pool morphology conditions.

[0099] (5) A unified output multi-task collaborative framework.

[0100] By using the fused shared features to simultaneously drive the detection head and the segmentation head, the two tasks no longer reason independently, but instead make collaborative judgments based on the same enhanced features.

[0101]

[0102]

[0103] in, The detection head for the target detection model; This is the segmentation head for the semantic segmentation model.

[0104] And improve output consistency through consistency constraints:

[0105]

[0106] Where B is the bounding box corresponding to the shared features output after the fusion of the classification head pairs of the object detection model; This refers to the segmentation result output by the segmentation head pair of the semantic segmentation model after fusion of shared features; Indicates the mask; It is an L1 norm.

[0107] This structure significantly improves positioning accuracy, boundary clarity, and recognition consistency without significantly increasing computational load.

[0108] In this embodiment, the loss function of the object detection model is:

[0109] L det ≈L box +L obj

[0110] Among them, the bounding box regression loss L box for:

[0111] L box =CIoU(B, B) gt );

[0112] Target confidence loss L obj for:

[0113] L obj =BCE(P) obj y obj )

[0114] Where B is the bounding box predicted by the object detection network; B gt The actual border is marked; P obj y is the probability of target presence predicted by the target detection network. obj This is a real label.

[0115] Loss function for semantic segmentation:

[0116] L seg =L ce +L Dice

[0117] Among them, the pixel-level classification loss L CE :

[0118] L CE = ∑y i logy i ^

[0119] Region overlap loss L Dice :

[0120]

[0121] Among them, y i Let y be the true label of the i-th pixel; i ^ represents the predicted probability of the i-th pixel; S represents the predicted segmentation region; G represents the true segmentation region; Mask(B) is the binary region mask obtained by converting the bounding box.

[0122] Final total loss: L total =L det +L seg +λL consistency

[0123] Step 5: Extract the state of the molten pool.

[0124] This embodiment extracts several key physical parameters from the contour and positioning results, including molten pool width, length, area, aspect ratio, roundness, boundary fluctuation, molten pool center point offset, spot perturbation value, and spatter density. These parameters can be used to evaluate the molten pool's thermal input stability, energy distribution uniformity, and formed layer quality. The extracted results are highly timely, supporting continuous multi-frame analysis, trend monitoring, and early defect warning.

[0125] This embodiment constructs a Feature-Interaction Bridge that allows direct exchange of multi-scale, high-dimensional semantics between deep network backbones. This enables real-time collaboration between the detection and segmentation networks at the feature level, transforming their unidirectional dependence into deep coupling and fundamentally improving the accuracy, reliability, and real-time performance of melt pool state recognition.

[0126] The core idea of ​​this embodiment is to introduce a semantic segmentation network to provide high-resolution representation of melt pool boundaries, contours, and fine-grained structures while maintaining the sensitivity of the target detection network to global semantics and key regions. The two networks achieve bidirectional information flow across networks through mechanisms such as multi-scale semantic feature alignment, spatially guided attention, and adaptive fusion weights, thereby achieving comprehensive performance far superior to single algorithms or traditional shallow fusion methods.

[0127] This embodiment enables higher-quality real-time identification of molten pool morphology, size, boundary structure, and abnormal states, effectively improving the stability and control response speed of online monitoring in additive manufacturing. This embodiment features a simple structure, high pluggability, low computational cost, and wide adaptability. It can be applied to online visual monitoring systems for various additive manufacturing processes such as laser cladding, directed energy deposition (DED), and selective laser melting (SLM), demonstrating significant engineering value and application prospects.

[0128] Example 2

[0129] The purpose of this embodiment is to provide an additive manufacturing inspection system based on the fusion of object detection and semantic segmentation, including:

[0130] The acquisition module is configured to acquire images of the molten pool during the additive manufacturing process.

[0131] The target detection module is configured to: use the target detection model to locate the molten pool region in the acquired molten pool image during the additive manufacturing process, and obtain the molten pool bounding box and the initial state category of the molten pool;

[0132] The semantic segmentation module is configured to input the bounding box of the melt pool as the region of interest into the semantic segmentation model to obtain the geometric dimensions of the melt pool.

[0133] The training module is configured to: introduce a consistency loss function during the training of the object detection model and the semantic segmentation model to achieve deep interaction and complementarity between object detection and semantic segmentation; wherein, the consistency loss function is specifically determined as follows:

[0134] Feature alignment is performed on the feature maps of the backbone network of the object detection model and the backbone network of the semantic segmentation model. A bidirectional cross-task attention mechanism is used to perform deep interaction and complementarity between object detection and semantic segmentation on the aligned features to obtain fused features.

[0135] A spatial weight map is generated using a semantic segmentation model, channel attention is introduced, and fusion features are combined to obtain fused shared features;

[0136] A consistency loss function is constructed based on the consistency of predictions made by the detection head of the object detection model and the segmentation head of the semantic segmentation model for the fused shared features.

[0137] In further embodiments, the following is also provided:

[0138] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0139] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0140] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0141] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0142] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0143] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0144] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0145] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0146] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0147] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0148] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An additive manufacturing detection method based on the fusion of target detection and semantic segmentation, characterized in that, include: Acquire images of the molten pool during additive manufacturing; The target detection model is used to locate the molten pool region in the acquired molten pool image during the additive manufacturing process, and the molten pool bounding box and the initial state category of the molten pool are obtained. The bounding box of the molten pool is input as the region of interest into the semantic segmentation model to obtain the geometric dimensions of the molten pool; In the training of the object detection model and the semantic segmentation model, a consistency loss function is introduced to achieve deep interaction and complementarity between object detection and semantic segmentation; the consistency loss function is specifically determined as follows: Feature maps of the backbone networks of the object detection model and the semantic segmentation model are aligned. A bidirectional cross-task attention mechanism is then used to perform deep interaction and complementarity between the aligned features and object detection and semantic segmentation, resulting in fused features. Specifically: The feature map of the aligned object detection model backbone network is used as the query vector and key vector, and the feature map of the aligned semantic segmentation model backbone network is used as the value vector. The attention mechanism is used to obtain the mapping features of object detection to semantic segmentation. The feature maps of the aligned semantic segmentation model backbone network are used as query vectors and key vectors, and the feature maps of the aligned object detection model backbone network are used as value vectors. The attention mechanism is used to obtain the mapping features from semantic segmentation to object detection. The mapping features from target detection to semantic segmentation and the mapping features from semantic segmentation to target detection are fused to obtain the fused features; A spatial weight map is generated using a semantic segmentation model, channel attention is introduced, and fusion features are combined to obtain fused shared features; A consistency loss function is constructed based on the consistency of predictions made by the detection head of the object detection model and the segmentation head of the semantic segmentation model for the fused shared features.

2. The additive manufacturing inspection method based on the fusion of target detection and semantic segmentation as described in claim 1, characterized in that, A spatial weight map is generated using a semantic segmentation model. Channel attention is introduced and combined with fused features to obtain the fused shared features, specifically: The mask features predicted by the semantic segmentation model are processed by convolution and activation functions to obtain convolutional activation features. The convolutional activation features are then multiplied element-wise with the fused features to obtain a spatial weight map, and the spatial weight map is then enhanced by boundary enhancement. By introducing channel attention, the weights of the boundary enhancement features and the original unenhanced fusion features are dynamically adjusted based on the fusion features to obtain the fused shared features. 3.The additive manufacturing detection method based on the fusion of target detection and semantic segmentation of claim 2, wherein, Gradient enhancement is introduced to strengthen the boundaries of the spatial weight graph, specifically as follows: ； ； wherein is the Hadamard element-wise multiplication; is a boundary enhancement coefficient; is a spatial weight map; is a gradient operator; is the result predicted by the semantic segmentation model for each pixel. 4.The additive manufacturing detection method based on the fusion of target detection and semantic segmentation of claim 1, wherein, The consistency loss function is: ； Where B is the bounding box corresponding to the shared features output after the fusion of the classification head pairs of the object detection model; This refers to the segmentation result output by the segmentation head pair of the semantic segmentation model after fusion of shared features; Indicates the mask; It is an L1 norm. 5.The additive manufacturing detection method based on the fusion of target detection and semantic segmentation of claim 1, wherein, The total loss function of the object detection model includes a bounding box regression loss function and an object confidence loss function; the total loss function of the semantic segmentation model includes a pixel-level classification loss function and a region overlap loss function; the total loss function for training the object detection model and the semantic segmentation model includes the total loss function of the object detection model, the total loss function of the semantic segmentation model, and a consistency loss function.

6. The additive manufacturing inspection system based on the fusion of target detection and semantic segmentation, characterized in that, include: The acquisition module is configured to acquire images of the molten pool during the additive manufacturing process. The target detection module is configured to: use the target detection model to locate the molten pool region in the acquired molten pool image during the additive manufacturing process, and obtain the molten pool bounding box and the initial state category of the molten pool; The semantic segmentation module is configured to input the bounding box of the melt pool as the region of interest into the semantic segmentation model to obtain the geometric dimensions of the melt pool. The training module is configured to: introduce a consistency loss function during the training of the object detection model and the semantic segmentation model to achieve deep interaction and complementarity between object detection and semantic segmentation; wherein, the consistency loss function is specifically determined as follows: Feature maps of the backbone networks of the object detection model and the semantic segmentation model are aligned. A bidirectional cross-task attention mechanism is then used to perform deep interaction and complementarity between the aligned features and object detection and semantic segmentation, resulting in fused features. Specifically: The feature map of the aligned object detection model backbone network is used as the query vector and key vector, and the feature map of the aligned semantic segmentation model backbone network is used as the value vector. The attention mechanism is used to obtain the mapping features of object detection to semantic segmentation. The feature maps of the aligned semantic segmentation model backbone network are used as query vectors and key vectors, and the feature maps of the aligned object detection model backbone network are used as value vectors. The attention mechanism is used to obtain the mapping features from semantic segmentation to object detection. The mapping features from target detection to semantic segmentation and the mapping features from semantic segmentation to target detection are fused to obtain the fused features; A spatial weight map is generated using a semantic segmentation model, channel attention is introduced, and fusion features are combined to obtain fused shared features; A consistency loss function is constructed based on the consistency of predictions made by the detection head of the object detection model and the segmentation head of the semantic segmentation model for the fused shared features.

7. An electronic device, comprising: It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-5.

9. A computer program product, characterised in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

Images