Defect detection methods, devices, electronic equipment, storage media and products

By using a detection model combining a visual encoder and a text model in the additive manufacturing process, and combining the time information and type of the process image, the problem of real-time and accuracy of defect detection in laser powder bed melting process is solved, realizing real-time monitoring of the printing process and high-precision defect identification.

CN123118653APending Publication Date: 2026-11-17CHINA UNICOM (SHANGHAI) IND INTERNET CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611636112.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-22
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies make it difficult to monitor and identify defects in real time during additive manufacturing, especially in laser powder bed melting processes. Defect detection is costly and abnormal trends cannot be identified in advance, leading to damage to part performance and material waste.

Method used

The detection model employs a visual encoder and a text model. By acquiring process images, time information, and image type during the additive manufacturing process, visual features are extracted using convolutional channels and combined with instruction text for defect description, enabling real-time monitoring and high-precision defect identification of the printing process.

Benefits of technology

It improves the real-time performance and accuracy of defect detection, enables real-time monitoring and defect identification of the printing process, and reduces damage to part performance and material waste.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118653A_ABST
    Figure CN123118653A_ABST
Patent Text Reader

Abstract

This application discloses a defect detection method, apparatus, electronic device, storage medium, and product. It relates to the field of image processing technology. The method includes: acquiring process images, each accompanied by corresponding time information and image type, i.e., toner spreading image or printing image; inputting an image group consisting of multiple consecutive process images and instruction text into a detection model to obtain defect detection results; the detection model includes a visual encoder and a text model, the visual encoder being used to extract visual features for each process image using corresponding convolutional channels, and the text model being used to output defect detection results based on the instruction text and visual features. This scheme also inputs image type and time information into the model, combining multiple process images for analysis, extracting defect features from different types of images, enabling independent expression of image features in the toner spreading and printing stages, improving the real-time performance and accuracy of detection, and realizing real-time monitoring and defect identification of the printing process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a defect detection method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] To meet the manufacturing demands of high-end equipment, additive manufacturing (AM) technology has experienced rapid development. Among these, Laser Powder Bed Fusion (LPBF) technology, with its unique layer-by-layer deposition mechanism, has found widespread application across various fields. Compared to traditional processing techniques, LPBF is not constrained by tooling models and can directly fabricate lightweight, integrated parts with complex internal flow channels and lattice structures. However, the entire LPBF forming process is accompanied by continuously changing temperature and stress fields. Disturbances such as powder material properties, laser parameter drift, and powder uniformity accumulate layer by layer, inducing various defects such as warping, spheroidization, powder residue contamination, doctor blade streaks, and incomplete powder deposition. Furthermore, these defects exhibit a significant time-series propagation effect; weak anomalies in a single layer can be amplified in subsequent printed layers. If these defects are not detected during the forming stage, they will continuously damage the performance of the part, and in severe cases, directly halt printing, resulting in a significant waste of raw materials and time.

[0003] Current mainstream industrial defect detection methods are unable to capture this time-series evolution characteristic, exhibiting significant limitations:

[0004] Destructive standard test: It indirectly assesses process stability by relying on the mechanical testing of standard specimens. It can only assess the batch process level and cannot carry out layer-by-layer dynamic monitoring of physical parts.

[0005] Industrial CT scanning: It can perform non-destructive testing on internal defects of finished parts, but it is an offline inspection of finished products. Not only is the inspection cost high and the inspection cycle long, but more importantly, it can only trace defects after the printing is completed. It cannot use interlayer timing information to identify abnormal trends in advance and does not have the ability to provide process early warning.

[0006] Against the backdrop of the rapid development of the Industrial Internet and artificial intelligence, how to effectively model the layer-sequence temporal correlation characteristics of the LPBF printing process, and use temporal intelligent detection and predictive reasoning technology to achieve real-time defect identification and early prediction during the forming process, has become a core problem that needs to be solved in the field of online monitoring of additive manufacturing. Summary of the Invention

[0007] This application provides a defect detection method, apparatus, electronic device, storage medium, and product to achieve real-time monitoring and high-precision defect identification of the printing process.

[0008] In a first aspect, embodiments of this application provide a defect detection method, including:

[0009] Collect process images during the additive manufacturing process. Each process image is accompanied by corresponding time information and image type, wherein the image type is either a powder spreading image or a printing image.

[0010] An image group consisting of multiple consecutive process images, along with the corresponding instruction text, is input into a trained detection model to obtain a defect detection result. The detection model includes a visual encoder and a text model. The visual encoder extracts visual features for each process image in the image group using a convolutional channel corresponding to the image type. The text model infers and outputs a defect description text as the defect detection result based on the instruction text and visual features corresponding to the image group. Secondly, embodiments of this application also provide a defect detection device, including:

[0011] The image acquisition module is used to acquire process images during the additive manufacturing process. Each process image is accompanied by corresponding time information and image type, wherein the image type is either a powder spreading image or a printing image.

[0012] The defect detection module is used to input an image group consisting of multiple consecutive process images and the corresponding instruction text of the image group into a trained detection model to obtain a defect detection result. The detection model includes a visual encoder and a text model. The visual encoder is used to extract visual features for each process image in the image group using a convolutional channel corresponding to the image type. The text model is used to infer and output a defect description text as a defect detection result based on the instruction text corresponding to the image group and the visual features.

[0013] Thirdly, embodiments of this application provide an electronic device, including:

[0014] One or more processors;

[0015] Storage device for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the defect detection method as described in the first aspect.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the defect detection method as described in the first aspect.

[0018] Fifthly, embodiments of this application also provide a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the defect detection method as described in any of the above embodiments.

[0019] This application provides a defect detection method, apparatus, electronic device, storage medium, and product. The defect detection method includes: acquiring process images during the additive manufacturing process, with each process image accompanied by corresponding time information and image type, wherein the image type is a powder spreading image or a printing image;

[0020] An image group consisting of multiple consecutive process images, along with the corresponding instruction text, is input into a trained detection model to obtain defect detection results. The detection model includes a visual encoder and a text model. The visual encoder extracts visual features for each process image in the image group using convolutional channels corresponding to the image type. The text model infers and outputs a defect description text as the defect detection result based on the instruction text and visual features corresponding to the image group. This technical solution also inputs image type and time information into the model, combining multiple process images for analysis to extract defect features from different image types. This allows for independent expression of image features in the powder spreading and printing stages, improving the real-time performance and accuracy of detection, and achieving real-time monitoring and defect identification of the printing process. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart illustrating a defect detection method provided in an embodiment of this application;

[0023] Figure 2 A schematic diagram of the internal structure of a TC-LoRA module provided in an embodiment of this application;

[0024] Figure 3 A schematic diagram of the internal structure of a time encoder provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0028] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0029] It should be noted that the concepts of "first" and "second" mentioned in the embodiments of this application are only used to distinguish different devices, modules, units or other objects, and are not used to limit the order of functions performed by these devices, modules, units or other objects or their interdependencies.

[0030] In the embodiments of this application, "multiple" refers to at least two unless otherwise specified.

[0031] In the embodiments of this application, "A / B" refers to A and / or B when there is no conflict, and can refer to A or B when there is a conflict.

[0032] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0033] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the relevant content of the solution.

[0034] Figure 1This is a flowchart illustrating a defect detection method provided in an embodiment of this application. This embodiment is applicable to situations where defects are detected using process images from the additive manufacturing process. Specifically, the defect detection method can be executed by a defect detection device, which can be implemented through software and / or hardware and integrated into an electronic device. The electronic device includes, but is not limited to, devices with data processing capabilities such as computers, smartphones, or servers, and can also be a central processing unit (CPU), a system-on-chip (SoC), a field-programmable gate array (FPGA), or a microcontroller (MCU), etc.

[0035] like Figure 1 As shown, the method specifically includes the following steps:

[0036] S110. Collect process images during the additive manufacturing process. Each process image is accompanied by corresponding time information and image type, wherein the image type is either a powder spreading image or a printing image.

[0037] In this embodiment, while acquiring process images, the time information and image type of the process images are also obtained. The time information mainly refers to the generation time of the process image, also known as a timestamp. The image type mainly refers to the process operation corresponding to the process image, such as a powder spreading image or a printing image. On the one hand, considering that the powder spreading and printing images in the additive manufacturing process have a temporal sequence, many defects may not be detectable at a single moment. For example, if the powder spreading time of the previous layer is too long and the amount is too large, it may lead to insufficient printing of the next layer under the same printing time, resulting in slag defects. Or, if the edge of the previous layer is over-melted due to a long printing time, it may not be observed in the current image, but the edge warping traces can be observed when the next layer is powder spread. Therefore, when acquiring process images in the additive manufacturing process, the corresponding time information of each process image is obtained and integrated into the defect detection process. The acquired process images can be arranged according to the generation time. On the other hand, considering that the powder spreading and printing images have obvious feature differences, the image type of the process image is also obtained when acquiring process images in the additive manufacturing process. Based on this, process images, time information, and image type are used as inputs and basis for subsequent defect detection. Process images from different times can be combined for joint analysis, and targeted processing can be performed on different image types when extracting image features to improve the quality of features.

[0038] S120. An image group consisting of multiple consecutive process images and the corresponding instruction text of the image group are input into a trained detection model to obtain a defect detection result. The detection model includes a visual encoder and a text model. The visual encoder is used to extract visual features for each process image in the image group using a convolutional channel corresponding to the image type. The text model is used to infer and output a defect description text as a defect detection result based on the instruction text corresponding to the image group and the visual features.

[0039] In this embodiment, after training, the detection model is capable of reasoning out the output (defect description text) based on the input (an image group containing multiple process images, instruction text, time information, and image type). Specifically, the input to the detection model includes not only the image pixels of the process images, but also the image type (e.g., powder spreading or printing) and time information (e.g., 2003.12.09 12:32:12) for each process image, i.e., the input content is a triplet (...). , <type> , <time>).

[0040] For example, the detection model is a multimodal large model, such as Qwen3-VL-8B. It mainly consists of two parts: a visual model (primarily a visual encoder) and a text model. The visual encoder is mainly used to extract visual features (i.e., image features) from a set of process images (i.e., a group of images, such as 3-5 consecutive process images) through convolutional processing. These visual features can be converted into a form understandable to the text model, such as being encoded as keywords representing anomaly location, anomaly size, anomaly type, and / or cause, used to describe the image content of the process images for subsequent analysis and processing by the text model. It is important to note that different convolutional channels can be used for different types of process images within the image group, such as setting different receptive fields, specifically different network depths, or different kernel sizes and strides for each convolutional layer.

[0041] The text model can be a Large Language Model (LLM), which is mainly used to understand the input text instructions, thereby analyzing defects based on the visual features of the image set and generating defect description text. The text instructions specify the tasks that the text model needs to complete, such as analyzing the location, size, type, cause, and / or remedial measures of the defect. The defect description text includes the corresponding analysis results.

[0042] Understandably, training the detection model requires a large amount of sample data, including sample input data (such as multiple sample groups, the image type and time information of the sample process images in each sample group, and sample instruction text) and sample output data (defect description text labels output by the text model). Through continuous training, the model parameters in the detection model can be iteratively updated. The trained detection model has the ability to output defect description text based on triples and text instructions. The visual encoder and text model can be trained collaboratively or independently. Understandably, during the training of the visual encoder and / or text model, if the loss between the output defect description text and the defect description text labels is sufficiently small, then the visual encoder and / or text model meet the requirements.

[0043] The defect detection method provided in this application uses triples of process images as input, takes into account the time information and image type of the process images, can perform joint analysis on multiple process images, and performs targeted convolution processing according to the image type to extract high-quality visual features, so that the image features of the powder spreading and printing stages are expressed independently, improving the real-time performance and accuracy of detection, and realizing real-time monitoring and defect identification of the printing process.

[0044] In one embodiment, the detection model is obtained through the following training process:

[0045] S10. Obtain the training set, which includes multiple sample groups. Each sample group includes multiple sample process images, corresponding image type labels and time information labels for each sample process image, sample instruction text, and defect description text labels.

[0046] The sample instruction text specifies the tasks the text model needs to perform on the sample process images, such as analyzing the location, size, type, cause, and / or remedial measures of defects. The defect description text label includes a description of the analysis results corresponding to the sample instruction text, which can be used to measure the accuracy of the visual features output by the final text model. The sample instruction text includes a time information label for each sample process image. The time information label for each sample process image is converted into a text marker and appended before the marker of the corresponding sample process image.

[0047] S20. In the first stage, the text model is frozen, and the visual encoder is fine-tuned based on the training set. The sample instruction text in the training set describes the image content in each sample group as the task objective.

[0048] For example, in the first stage of training, the text model does not participate in training, the model parameters of the text model remain unchanged, and some model parameters of the visual encoder are fine-tuned. The task goal is to make the visual features output by the visual encoder aligned with the semantics of the text.

[0049] Taking a single sample group as an example, it includes x sample process images (I1, I2, ..., I...). x (where x is an integer greater than 1), and x sample process images and their corresponding image type labels (Type1, Type2, ..., Type...). x ) and time information labels (time1, time2, ..., time x ), Defect description text label T out and sample instruction text T in1 Input to the detection model, in T in1 In the text, time1, time2, ..., time x They are converted into text tokens and concatenated into I1, I2, ..., I1 respectively. x Before the labeling. In the detection model, firstly, the visual encoder processes the process image I for each sample in the sample group. i (i=1,2,…,x), based on its image type label Type i (For powder-coated or printed images) Select the appropriate convolution channel to extract visual features C. i Then, the corresponding C1, C2, ..., C x And T in1 All inputs are fed into the text model, which then performs inference and outputs a defect description text T. out ', according to T out 'and T out The model parameters of the visual encoder can be updated by analyzing the loss between them.

[0050] S30. In the second stage, the fine-tuned visual encoder is frozen, and the text model is fine-tuned based on the training set. The instructions in the training set are aimed at analyzing the defect features of each sample group.

[0051] For example, in the second phase of training, the model parameters of the fine-tuned visual encoder are kept unchanged, while some model parameters of the text model are fine-tuned.

[0052] Taking a single sample group as an example, it includes x sample process images (I1, I2, ..., I...). x (where x is an integer greater than 1), and x sample process images and their corresponding image type labels (Type1, Type2, ..., Type...). x ) and time information labels (time1, time2, ..., time x ), Defect description text label T out and sample instruction text T in1 Input to the detection model, in T in1 In the text, time1, time2, ..., time x They are converted into text tokens and concatenated into I1, I2, ..., I1 respectively. x Before the labeling. In the detection model, firstly, the visual encoder processes the process image I for each sample in the sample group. i (i=1,2,…,x), based on its image type label Type i (For powder-coated or printed images) Select the appropriate convolution channel to extract visual features C. i Then, the corresponding C1, C2, ..., C x And T in1 All inputs are fed into the text model, which then performs inference and outputs a defect description text T. out ', according to T out 'and T out The model parameters of the text model can be updated by analyzing the loss between them.

[0053] It should be noted that the training sets for the two stages are different. The first stage mainly focuses on image description tasks, while the second stage mainly focuses on temporal diagnostic tasks such as cause analysis and defect analysis.

[0054] The following is a specific example of the training process:

[0055] In the first stage, a question-and-answer training set is generated for the sample group. This training set is for an image description task, with the unified question being "What do these images describe?" Each question-and-answer pair contains the following content:

[0056] Instruction: Task instruction;

[0057] Output: The model's expected answer;

[0058] Images: In relation to the Instruction The image path corresponding to the label;

[0059] Type: Similar to the one in Instruction Image classification corresponding to the label;

[0060] Time: In relation to Instruction The time information corresponding to the tag.

[0061] The time information is used in the following two parts:

[0062] The first part involves inputting the sample process images into the vision encoder in batches, where the vision encoder encodes the time information.

[0063] The second part involves concatenating the corresponding time before the marker of the corresponding sample process image in the sample text instruction, and sending it as a text marker for the image time sequence into the text module for direct encoding.

[0064] The process of generating the training set can be represented as: {

[0065] "instruction": " <time0> 2001:09:12 12:32:09< / time0>

[0066] <time1> 2001:09:12 12:32:19< / time1>

[0067] <time2> 2001:09:12 12:32:29< / time2>

[0068] What do these images depict?

[0069] "output": "The input consists of four process images, in the order of powder spreading, printing, powder spreading, and printing. The first process image shows the result of the first powder spreading, where obvious white areas are not completely covered by powder. There is a 10-second interval between the first powder spreading image and the next printing image; the printed image shows dark gray raised edges. The second powder spreading takes 10 seconds to complete, and the incomplete powder spreading at the edges is even more pronounced after the second spreading."

[0070] "images":["E: / xxx.jpg","E: / xxx.jpg","E: / xxx.jpg"]

[0071] "type":["Toner spreading","Printing","Toner spreading"]

[0072] "time":["2001:09:12 12:32:09","2001:09:12 12:32:19","2001:09:12 12:32:29"]

[0073] }

[0074] In the second stage, the text model (Decoder) of the detection model is fine-tuned and trained to enable the text model to have high-order temporal logic reasoning capabilities such as "cause analysis" and "defect diagnosis" for various process defects in 3D printing.

[0075] It should be noted that the dataset format is the same for both the first and second stages; the main difference lies in the type of problem. The first stage primarily involves image description tasks, while the second stage mainly involves time-series diagnostic tasks such as defect cause analysis and defect analysis.

[0076] In one embodiment, the visual encoder includes a backbone network, with each self-attention layer of the backbone network connected to a fine-tuning bypass module in parallel. This involves adding a "low-rank" branch next to each self-attention layer of the backbone network, training only this branch without modifying the model itself, thus achieving fine-tuning with minimal parameters. The fine-tuning bypass module can also be called a Temporal Consistency-Low Rank Adaptation (TC-LoRA) module. The fine-tuning bypass module incorporates convolutional layers, thereby integrating local texture extraction capabilities to accurately capture the spatial distribution features of various defects in sample process images, providing local image recognition capabilities. It also includes a temporal encoder to encode the temporal information of each sample process image into a corresponding temporal modulation vector.

[0077] In this embodiment, the fine-tuning bypass module specifically includes:

[0078] Dimensionality reduction layer is used to project the features of the input sample process image into a low-rank space to obtain dimensionality-reduced features;

[0079] The time encoder is used to generate a corresponding time modulation vector based on the time difference of each sample process image. For example, if the time information corresponding to the current sample process image is t, the time encoder can output a weight vector (time modulation vector) for t, which is directly multiplied by the dimensionality-reduced features to represent the influence of time on the sample process image.

[0080] The convolution module includes a layer-specific convolutional structure unique to each self-attention layer, as well as a shared convolutional structure shared by all self-attention layers. The layer-specific and shared convolutional structures are used to convert spatial arrangement features into two-dimensional convolutional features. The spatial arrangement features are obtained by reshaping the temporally modulated features, which are obtained by multiplying the temporally modulated vector and the dimensionality-reduced features element-wise through broadcasting. The shared convolutional structure is used to select the corresponding convolutional channel according to the image type to perform convolution operations and obtain two-dimensional convolutional features. The convolutional channel is the first channel corresponding to the powder-spreading image or the second channel corresponding to the printed image. The receptive field of the first channel is larger than that of the second channel.

[0081] The upscaling layer projects the flattened 2D convolutional features onto the original rank space, scales them, and uses them as bypass features. These features are then added to the output of the corresponding self-attention layer to obtain the visual features.

[0082] It should be noted that traditional (non-temporally modulated) Low Rank Adaptation (LoRA) modules are two-dimensional linear matrices with a simple structure. Input features undergo simple matrix multiplication, performing only a linear transformation of the channel dimension—a simple "dimensionality reduction and then dimensionality increase" linear projection. This is only suitable for general text question answering and simple image description tasks, and has limited ability to represent two-dimensional neighborhood relationships in images, such as warping, bulging, and sphericity. These are local spatial textures, and linear matrices cannot capture the adjacency relationships between pixels; they can only model global channel weights, resulting in weak ability to extract subtle defect features. However, in the embodiments of this application, to adapt to the characteristics of continuous imaging in the additive manufacturing process, such as... Figure 2 As shown, a time encoder (Time MLP) and two 2D convolutional structures (a private layer and a shared layer) are added to the traditional LoRA module to obtain the TC-LoRA module. The shared convolutional structure can select either the first channel (apply_conv) corresponding to the powder-spreading image or the second channel (print_conv) corresponding to the printed image, with the receptive field of the first channel being larger than that of the second channel. Optionally, the receptive field can be determined based on at least one of the following parameters: network depth, kernel size, and stride.

[0083] For example, the temporal encoder is a lightweight multilayer perceptron (MLP) structure. Its input shape is [B, 1], and its output shape is [B, r], where B is the batch size and r is the lora rank, maintaining the same dimension as the rank of the reduced feature vector. The internal structure of the temporal encoder is as follows... Figure 3 As shown, nn.Linear() represents a fully connected layer, and nn.GELUTanh() represents the tanh approximation of the Gaussian Error Linear Unit (GELU) activation function.

[0084] Optionally, the time information in the above embodiments mainly refers to the image generation time, i.e., the timestamp, usually expressed in hours, minutes, and seconds. During training, the timestamp needs to be converted into a normalized feature that has physical meaning for the visual encoder. Specifically, for the first sample process image in the sample group, the average time taken under standard 3D printing scenarios is used. As a time difference; for any other arbitrary (such as the first) (Number) sample process images, where the time difference is the difference in time information between the current sample process image and the previous sample process image, i.e. Furthermore, to normalize the representation, the maximum observation interval for the values ​​is pre-set. It is uniformly expressed as the ratio of the time difference to the maximum time difference, that is: Furthermore, if the time difference exceeds the maximum time interval, it is truncated to 1. The timing modulation vector generated by the time encoder is represented as follows: ,in, Indicates that the time encoder will Convert to The timing modulation coding operations and processing performed.

[0085] The process of generating a timing modulation vector using the above time coding can be represented by the following pseudocode:

[0086] dt1_norm = min(mean_interval / max_interval,1.0)

[0087] delta_times = [dt0_norm]

[0088] for i in range(1, len(times)):

[0089] dt = (times[i] - times[i-1]).total_seconds()

[0090] dt_norm = min(dt / max_interval,1.0)

[0091] delta_times.append(dt_norm)

[0092] Building on this, adding a time encoder to TC-LoRA allows subsequent convolutional modules to incorporate temporal physics knowledge such as "what stage of printing is currently in and how much heat has accumulated" during computation.

[0093] For example, the convolution module is designed as a two-layer convolutional structure. The first layer is a layer-specific convolutional structure, meaning each Transformer layer has its own independent convolutional structure, and the layers do not share parameters. The layer-specific convolutional structure is mainly responsible for extracting features specific to the current network depth. The second layer is a globally shared convolutional structure, which is a convolutional structure shared by all layers. It can be divided into two channels, which dynamically switch according to the image type: "applying powder" or "printing". Different channels have different parameter settings for their receptive fields. For example, the channel corresponding to the powder-applying image can be bound to a larger receptive field. This is because powder-applying defects (such as scraper marks) are usually very large, and a large receptive field can clearly see whether the powder is evenly spread globally. The channel corresponding to the printing image can be bound to a smaller receptive field. This is because defects such as warping and spheroidization are localized, and a small receptive field can focus extremely on microscopic details such as "local melt pool edges and size warping". If the fixed receptive field is set too large, the melt pool will be distorted; if it is too small, the scraper marks that span the entire image will not be visible.

[0094] For convolutional modules, a visual feature extraction MLP can be set after each attention layer. The output calculation formula for a single attention layer of the visual encoder is as follows: ; ;

[0095] in, This represents the output of the attention layer where TC-LoRA resides. This represents the output vector of the backbone network whose parameters have been frozen. This represents the output vector of the TC_LoRA module. This represents the input to the attention layer where TC-LoRA resides. This indicates the weights of the backbone network that are frozen.

[0096] After dimensionality reduction layer (dimensionality reduction matrix) The reduced-dimensional features are obtained after projection: ;

[0097] Element-wise broadcast multiplication of the dimensionality-reduced features with the temporal modulation vector yields the temporally modulated features: ;

[0098] The temporally modulated features are reshaped in two dimensions to obtain spatial arrangement features. Then, it enters a layer-specific convolutional structure to obtain two-dimensional convolutional features. ,in, Presentation layer-specific convolutional structure ( )right The convolution operations and processing performed;

[0099] In a shared convolutional structure, image type can be used as a hard classifier. Two convolutional channels are set, and different receptive fields are used to perform convolution processing based on the defects of different image types, resulting in two-dimensional convolutional features. ,in, Indicates the first channel ( )right The convolution operations and processing performed Indicates the second channel ( )right The convolutional operations and processing performed allow for better extraction of corresponding defect features, enabling the features of the powder-spreading and printing images to be expressed independently, thus avoiding mutual interference.

[0100] Finally, the fused two-dimensional convolutional features are flattened to obtain... The flattened features are projected back to the original space through an up-dimensional layer (up-dimensional matrix B), and multiplied by a low-rank adaptive scaling factor to obtain the bypass features: ;in, For LoRA hyperparameters, Let be the rank of the low-rank matrix. The scaling factor determines the fine-tuning intensity of the TC_LoRA module.

[0101] For example, the structure of a visual encoder can be represented as follows:

[0102] Qwen3VLVisionModel(

[0103] # Shared Convolution

[0104] global_print_conv: Conv2d(r,r,3,padding=1,dilation=1,bias=True)

[0105] global_apply_conv: Conv2d(r,r,3,padding=3,dilation=3,bias=True)

[0106] # Stay still

[0107] (patch_embed): Qwen3VLVisionPatchEmbed(

[0108] (proj): Conv3d(3,1152,kernel_size=(2,16,16),stride=(2,16,16)) )

[0110] (pos_embed): Embedding(2304,1152)

[0111] (rotary_pos_emb): Qwen3VLVisionRotaryEmbedding()

[0112] (blocks): ModuleList(

[0113] (0-26): 27 x Qwen3VLVisionBlock(

[0114] (norm1): LayerNorm((1152,),eps=1e-06,elementwise_affine=True,bias=True)

[0115] (norm2): LayerNorm((1152,),eps=1e-06,elementwise_affine=True,bias=True)

[0116] (attn): Qwen3VLVisionAttention(

[0117] # Replace the original qkv Linear module with the TC-LoRA timing modulation low-rank module

[0118] (qkv): TC_LoRA(

[0119] # Freeze the original weights of the backbone network

[0120] (base_layer): Linear(in_features=1152,out_features=3456,bias=True)

[0121] # LoRA Dimensionality Reduction Branch A

[0122] (lora_A): Linear(in_features=1152,out_features=r,bias=False)

[0123] # LoRA Upgrade Branch B

[0124] (lora_B): Linear(in_features=r,out_features=3456,bias=False)

[0125] # [Time-series modulation branch: Lightweight Time MLP time encoder]

[0126] (time_mlp): TimeMLP(

[0127] (fc1): Linear(in_features=1,out_features=r,bias=True)

[0128] (act): GELUTanh()

[0129] (fc2): Linear(in_features=r,out_features=r,bias=True) )

[0131] # Private Layer: Layer-specific convolutional structure (each layer is independent and not shared)

[0132] (layer_conv): nn.Conv2d(

[0133] in_channels=r,

[0134] out_channels=r,

[0135] kernel_size=3,

[0136] padding=1,

[0137] dilation=1,

[0138] bias=True ) )

[0141] # Output projection preserves native linear layers

[0142] (proj): Linear(in_features=1152,out_features=1152,bias=True) )

[0144] (mlp): Qwen3VLVisionMLP(

[0145] (linear_fc1): Linear(in_features=1152,out_features=4304,bias=True)

[0146] (linear_fc2): Linear(in_features=4304,out_features=1152,bias=True)

[0147] (act_fn): GELUTanh() ) ) ) )

[0152] In one embodiment, the text model is a large language model; a low-rank adaptation module is connected in parallel on the linear projection weight matrix of each self-attention layer of the large language model. The low-rank adaptation module is used to perform low-rank updates on the linear projection weights. The low-rank adaptation module includes a dimensionality reduction layer and a dimensionality increase layer; a multi-token prediction module is attached after the last decoder layer of the large language model. The multi-token prediction module includes a first prediction branch and a second prediction branch. The first prediction branch and the second prediction branch are respectively used to predict tokens at subsequent time steps based on the top-level temporal hidden representation of the large language model.

[0153] For example, in the second stage of training, the visual encoder (including the trained TC-LoRA module) that has been fine-tuned and spatiotemporally aligned in the first stage is frozen, and the feature extraction and alignment channels are locked. Furthermore, the main parameters of the large language model can also be locked, with low-rank adaptation modules inserted in parallel only within each layer of self-attention (self_attn). In this embodiment, the low-rank adaptation module can refer to a (traditional) LoRA bypass containing dimensionality reduction and expansion layers. The multi-token prediction (MTP) module can be attached after the last layer (e.g., Layer 35) of the LLM backbone network. The MTP module is a newly added, fully trainable prediction branch (also called an auxiliary decoding branch) used to force the low-rank adaptation module to learn future multi-step planning strategies during the training phase and to achieve lossless speculative sampling acceleration at the inference end. As a parallel inference branch, each branch of the MTP module has a built-in trainable MTP decoding layer (DecoderLayer).

[0154] For example, the LoRA bypass is connected in parallel next to the frozen linear projection layer. Its input features (features processed by the linear projection layer and aligned with the large language model) are projected onto the low-rank space through a low-rank dimensionality reduction matrix A to obtain low-rank features: Then, by projecting back to the original dimensional space using the increased-dimensional matrix B and multiplying by a scaling factor, the bypass features are obtained: ,in, This represents the input of the self-attention layer where the LoRA bypass is located. For LoRA hyperparameters, Let be the rank of the low-rank matrix. The scaling factor determines the fine-tuning intensity of the LoRa bypass (its value can differ from that of TC_LoRA). Based on this, large models can be aligned with knowledge in the semantic domain of 3D printing with minimal parameter updates.

[0155] In one embodiment, fine-tuning the text model based on the training set includes:

[0156] For any given time, the top-level temporal hidden representation:

[0157] The top-level temporal hidden representation is input into the first prediction branch and the second prediction branch. The first prediction branch is used to predict the first output word at the next time step based on the top-level temporal hidden representation. The second prediction branch is used to predict the second output word at the next time step after that based on the top-level temporal hidden representation.

[0158] Based on the first and second output terms and the term labels at the corresponding positions in the defect description text, update the parameters of the low-rank adaptation module and the multi-term prediction module.

[0159] For example, the fine-tuning process of a text model includes:

[0160] Obtain the top-level temporal hidden representation of the backbone at time t. , This represents the slice at time step t, i.e., the defect description text label T. out In the text model, at time t, the output label sequence ( The vector obtained after passing through the hidden layer excluding the MTP module contains information about the encoded preceding sequence. ) context information;

[0161] Will Simultaneously, two independent prediction branches (MTP1 and MTP2) are input for prediction: MTP1 and MTP2 are each processed through their respective dedicated decoding layers. Perform deep feature fusion and establish separate features for each feature. and The prediction task;

[0162] The output features of the two prediction branches reuse the normalization layer frozen in the backbone and the globally shared word head (lm_head, i.e. the model layer with frozen weights) to generate word probabilities, and calculate the corresponding multi-step prediction loss respectively.

[0163] The representation of the large language model and MTP structure is as follows:

[0164] Qwen3VLModel(

[0165] (vision_tower): Qwen3VLVisionModel(

[0166] # The vision module retains the original TC-LoRA structure, which is omitted here. )

[0168] (language_model): Qwen3VLTextModel(

[0169] (embed_tokens): Embedding(151936,4096) # Frozen, no LoRA, MTP shares this embedding layer

[0170] (layers): ModuleList(

[0171] (0-35): 36 x Qwen3VLTextDecoderLayer(

[0172] (self_attn): Qwen3VLTextAttention(

[0173] # Replace all of QKV / O with standard ordinary LoRA linear layers

[0174] (q_proj): LoRALinear(

[0175] base_layer=Linear(in_features=4096,out_features=4096,bias=False),

[0176] lora_A=Linear(4096,r,bias=False),

[0177] lora_B=Linear(r,4096,bias=False) )

[0179] (k_proj): LoRALinear(

[0180] base_layer=Linear(in_features=4096,out_features=1024,bias=False),

[0181] lora_A=Linear(4096,r,bias=False),

[0182] lora_B=Linear(r,1024,bias=False) )

[0184] (v_proj): LoRALinear(

[0185] base_layer=Linear(in_features=4096,out_features=1024,bias=False),

[0186] lora_A=Linear(4096,r,bias=False),

[0187] lora_B=Linear(r,1024,bias=False) )

[0189] (o_proj): LoRALinear(

[0190] base_layer=Linear(in_features=4096,out_features=4096,bias=False),

[0191] lora_A=Linear(4096,r,bias=False),

[0192] lora_B=Linear(r,4096,bias=False) )

[0194] (q_norm): Qwen3VLTextRMSNorm((128,),eps=1e-06)

[0195] (k_norm): Qwen3VLTextRMSNorm((128,),eps=1e-06) )

[0197] (mlp): Qwen3VLTextMLP(

[0198] # MLP has a slightly different range of LoRA

[0199] ( gate_proj ): LoRALinear (

[0200] base_layer=Linear(in_features=4096,out_features=12288,bias=False),

[0201] lora_A=Linear(4096,r,bias=False),

[0202] lora_B=Linear(r,12288,bias=False) )

[0204] ( up_proj ): LoRALinear (

[0205] base_layer=Linear(in_features=4096,out_features=12288,bias=False),

[0206] lora_A=Linear(4096,r,bias=False),

[0207] lora_B=Linear(r,12288,bias=False) )

[0209] ( down_proj ): LoRALinear (

[0210] base_layer=Linear(in_features=12288,out_features=4096,bias=False),

[0211] lora_A=Linear(12288,r,bias=False),

[0212] lora_B=Linear(r,4096,bias=False) )

[0214] ( act_fn ): SiLUActivation ( ) )

[0216] (input_layer_norm): Qwen3VLTextRMSNorm((4096,),eps=1e-06);

[0217] (post_attention_layer_norm): Qwen3VLTextRMSNorm((4096,),eps=1e-06); ) )

[0220] ( norm ): Qwen3VLTextRMSNorm ( ( 4096 , ), eps = 1e - 06 ) # MTP enabled , enabled

[0221] ( rotary_emb ): Qwen3VLTextRotaryEmbedding ( ) ;

[0222] # ===================== MTP Release Notes ================================

[0223] ( mtp_branch1 ): MTPBranch (

[0224] (mtp_decoder): Qwen3VLTextDecoderLayer(

[0225] ( self_attn ): Qwen3VLTextAttention ( self_attn ) .

[0226] (q_proj): Linear(4096,4096,bias=False)

[0227] (k_proj): Linear(4096,1024,bias=False)

[0228] (v_proj): Linear(4096,1024,bias=False)

[0229] (o_proj): Linear(4096,4096,bias=False)

[0230] (q_norm): Qwen3VLTextRMSNorm((128,),eps=1e-06)

[0231] (k_norm): Qwen3VLTextRMSNorm((128,),eps=1e-06) )

[0233] (mlp): Qwen3VLTextMLP(

[0234] (gate_proj): Linear(4096,12288,bias=False)

[0235] (up_proj): Linear(4096,12288,bias=False)

[0236] (down_proj): Linear(12288,4096,bias=False)

[0237] (act_fn): SiLUActivation() )

[0239] (input_layernorm): Qwen3VLTextRMSNorm((4096,),eps=1e-06)

[0240] (post_attention_layernorm): Qwen3VLTextRMSNorm((4096,),eps=1e-06) )

[0242] ),

[0243] (mtp_branch2): MTPBranch(

[0244] (mtp_decoder): Qwen3VLTextDecoderLayer(

[0245] (self_attn): Qwen3VLTextAttention(

[0246] (q_proj): Linear(4096,4096,bias=False)

[0247] (k_proj): Linear(4096,1024,bias=False)

[0248] (v_proj): Linear(4096,1024,bias=False)

[0249] (o_proj): Linear(4096,4096,bias=False)

[0250] (q_norm): Qwen3VLTextRMSNorm((128,),eps=1e-06)

[0251] (k_norm): Qwen3VLTextRMSNorm((128,),eps=1e-06) )

[0253] (mlp): Qwen3VLTextMLP(

[0254] (gate_proj): Linear(4096,12288,bias=False)

[0255] (up_proj): Linear(4096,12288,bias=False)

[0256] (down_proj): Linear(12288,4096,bias=False)

[0257] (act_fn): SiLUActivation() )

[0259] (input_layernorm): Qwen3VLTextRMSNorm((4096,),eps=1e-06)

[0260] (post_attention_layernorm): Qwen3VLTextRMSNorm((4096,),eps=1e-06) ) ) ) )

[0265] # Globally shared header: Shared by the trunk and MTP branches, completely frozen

[0266] (lm_head): Linear(in_features=4096,out_features=151936,bias=False)

[0267] In one embodiment, the loss function used in the second stage is expressed as: ;in, This represents the loss of the primary generation task, using autoregressive masked cross-entropy loss. This represents the auxiliary prediction loss, which is determined based on the losses of the two prediction branches of the multi-word prediction module. This is a balancing factor used to control the ratio of the main generation task loss to the auxiliary prediction loss; For the loss growth curve function, in the first part of the second stage, =0; in the second part of the second stage The value is smoothly increased according to the half-cycle cosine windowing curve; in the third part of the second stage, =1.

[0268] For example, the loss function in the second stage consists of the sum of two parts: the main generation task loss (NTP loss) and the auxiliary multi-term prediction loss (MTP loss). Due to the masking design, the loss calculation is performed only within the target answer (Output) range, on the true answer portion, and not on the prompt words and image portions.

[0269] For the main generation task loss, the classic autoregressive masked cross-entropy loss can be used to optimize the probability of predicting the next word at the current time step, and gradient backpropagation updates the LoRA parameters in the backbone; the main generation task loss can be expressed as... ;in, The target answer interval corresponds to the real token at position t (i.e., the word in Tout). Let V be the total number of tokens in the target answer interval, used as the normalized denominator. V is the visual input after the first stage alignment, and the loss is calculated only for the actual answer portion.

[0270] For auxiliary prediction loss The two prediction branches only receive the single-step hidden representation of the backbone network at time t. As input, the historical token sequence is no longer concatenated. );

[0271] MTP1 is based on Predicting words with a temporal shift of 1 step after feature enhancement The corresponding losses are: ;

[0272] MTP2 is based on the same Predicting temporal shifts in two steps after feature enhancement The corresponding losses are: ;

[0273] but .

[0274] Regarding the overall joint training losses By introducing a loss growth curve function A progressive growth control is applied to the MTP loss. In the early stages of training, the LoRA loss is updated first, and then the loss update of the MTP module is gradually increased in the later stages, completing end-to-end joint training optimization. The loss growth curve function can be expressed as:

[0275] ;

[0276] in, This is expressed as the percentage of the expected steps to the total steps. For example, if we expect to start adding MTP loss at 10% of the total steps, then... Set it to 0.1. Indicates the percentage of training progress: ;in, This represents the current number of training steps. This represents the total number of steps taken during training.

[0277] It should be noted that the loss growth curve is divided into the following three stages:

[0278] Pure LoRA convergence phase ( ): =0, MTP loss contributes nothing to the total loss, and the gradient is backpropagated to the LoRA adapter only by the single-step master generation loss. The detection model prioritizes fitting the basic temporal generation task to stabilize the multimodal text representation and avoid the non-converged backbone from oscillating due to multi-step prediction gradient interference;

[0279] Cosine smooth heating stage ( ): Employ a half-cycle cosine windowing curve to smoothly increase the MTP weights: at a threshold ( The derivative is 0 at point ), and the weights slowly increase after crossing the starting node. The weights rise steadily in the middle section, and the rate of increase slows down again when the training progress is close to 95%. There are no numerical jumps or gradient mutations throughout the process. While LoRA is continuously fine-tuned, the MTP module gradually participates in the update.

[0280] Steady-state joint optimization stage ( ): With the weights locked at a constant 1, the MTP loss weights no longer change, and the remaining 5% of training steps maintain a fixed gradient ratio. The cosine warming curve completes the weight increase ahead of schedule, and the weights remain stable at the end of training. The model converges stably to the optimal solution that balances single-step generation accuracy and multi-step causal programming capability.

[0281] In summary, the total joint loss can be expressed as: ;in, This is a balancing factor used to control the basic order of magnitude ratio between the MTP loss and the main generation loss. Under this joint loss constraint, the multi-step prediction gradient flow acts together on the trainable LoRA adapter, enabling it to have both extremely high temporal generation accuracy and the ability to predict the future.

[0282] The method in this application introduces a lightweight TC-LoRA temporal convolutional fusion module during the training of the detection model. This module abandons the traditional single linear LoRA structure and integrates a TC-LoRA bypass structure with temporal encoding and dual convolutional branches. A lightweight Time MLP dynamically modulates visual features based on printing time differences, incorporating process sequence and thermal accumulation physical information. A layered convolutional design is adopted, adaptively switching the receptive field size according to powder spreading and printing processes, balancing the extraction of large-area global defects and local micro-defects. The main branch is frozen, and the bypass is fine-tuned, achieving scene-specific feature optimization in a lightweight manner. Furthermore, a 3D printing temporal normalization quantization algorithm is provided: for continuous printing imaging scenarios, a self-developed differentiated temporal difference calculation and normalization strategy is used. The first frame uses the industry average printing time, and subsequent frames use the true time difference between adjacent frames, combined with maximum time interval truncation normalization. The original character timestamps are transformed into learnable physical temporal features, and bidirectional alignment of spatiotemporal features is achieved through text temporal marker concatenation and visual encoding modulation dual-path embedding. Furthermore, an MTP dual-branch progressive joint loss mechanism was adopted: a joint training strategy combining NTP main loss and dual-path MTP multi-step prediction auxiliary loss, predicting sequential words for the next 1 and 2 steps respectively, thereby enhancing the model's causal planning capability. A cosine warming weight function was introduced to progressively activate the MTP loss, avoiding early gradient oscillations.

[0283] Figure 4 This is a schematic diagram of a defect detection device provided in an embodiment of this application. Figure 4 As shown, the defect detection device provided in this embodiment includes:

[0284] Image acquisition module 210 is used to acquire process images during the additive manufacturing process. Each process image is accompanied by corresponding time information and image type, wherein the image type is a powder spreading image or a printing image.

[0285] The defect detection module 220 is used to input an image group consisting of multiple consecutive process images and the corresponding instruction text of the image group into a trained detection model to obtain a defect detection result. The detection model includes a visual encoder and a text model. The visual encoder is used to extract visual features for each process image in the image group using a convolutional channel corresponding to the image type. The text model is used to infer and output a defect description text as a defect detection result based on the instruction text corresponding to the image group and the visual features.

[0286] The device also inputs image type and time information into the model, analyzes multiple process images together, and extracts defect features from different types of images. This allows the image features of the powder spreading and printing stages to be expressed independently, improving the real-time performance and accuracy of detection, and enabling real-time monitoring and defect identification of the printing process.

[0287] Based on any of the above embodiments, the detection model is obtained through the following training process:

[0288] Obtain a training set, which includes multiple sample groups. Each sample group includes multiple sample process images, image type labels and time information labels for each sample process image, sample instruction text, and defect description text labels.

[0289] In the first stage, the text model is frozen, and the visual encoder is fine-tuned based on the training set, wherein the sample instruction text in the training set is used to describe the image content in each sample group as the task objective.

[0290] In the second stage, the fine-tuned visual encoder is frozen, and the text model is fine-tuned based on the training set, wherein the instructions in the training set are aimed at analyzing the defect features of each sample group.

[0291] The sample instruction text includes a time information tag for each sample process image. The time information tag for each sample process image is converted into a text marker and appended before the marker of the corresponding sample process image.

[0292] Based on any of the above embodiments, the visual encoder includes a backbone network, and each self-attention layer of the backbone network is connected in parallel with a fine-tuning bypass module; the fine-tuning bypass module includes:

[0293] Dimensionality reduction layer is used to project the features of the input sample process image into a low-rank space to obtain dimensionality-reduced features;

[0294] The time encoder is used to generate a corresponding timing modulation vector based on the time difference corresponding to each sample process image;

[0295] The convolution module includes a layer-specific convolutional structure unique to each self-attention layer, and a shared convolutional structure shared by all self-attention layers. Each layer-specific convolutional structure and the shared convolutional structure are used to convert spatial arrangement features into two-dimensional convolutional features. The spatial arrangement features are obtained by reshaping temporally modulated features, which are obtained by element-wise broadcast multiplication of the temporally modulated vector and the reduced-dimensional features. The shared convolutional structure is used to select the corresponding convolutional channel according to the image type to perform convolution operations, obtaining two-dimensional convolutional features. The convolutional channel is either the first channel corresponding to the powder-spreading image or the second channel corresponding to the printed image, where the receptive field of the first channel is larger than that of the second channel.

[0296] The upscaling layer projects the flattened 2D convolutional features onto the original rank space, scales them, and uses them as bypass features. These features are then added to the output of the corresponding self-attention layer to obtain the visual features.

[0297] Based on any of the above embodiments, the text model is a large language model;

[0298] A low-rank adaptation module is connected in parallel on the linear projection weight matrix of each self-attention layer of the large language model. The low-rank adaptation module is used to perform low-rank updates on the linear graph projection weights. The low-rank adaptation module includes a dimensionality reduction layer and a dimensionality increase layer.

[0299] The last layer decoder of the large language model is followed by a multi-lexical prediction module, which includes a first prediction branch and a second prediction branch. The first prediction branch and the second prediction branch are respectively used to predict the lexical units at subsequent time steps based on the top-level temporal hidden representation of the large language model.

[0300] Based on any of the above embodiments, the fine-tuning of the text model based on the training set includes:

[0301] For any given time, the top-level temporal hidden representation:

[0302] The top-level temporal hidden representation is input into the first prediction branch and the second prediction branch, wherein the first prediction branch is used to predict the first output word at the next time step based on the top-level temporal hidden representation; the second prediction branch is used to predict the second output word at the next time step after that based on the top-level temporal hidden representation.

[0303] The parameters of the low-rank adaptation module and the multi-term prediction module are updated based on the first output term, the second output term, and the term tags at corresponding positions in the defect description text.

[0304] Based on any of the above embodiments, the loss function used in the second stage is expressed as follows: ;in,

[0305] This represents the loss of the primary generation task, using autoregressive masked cross-entropy loss.

[0306] This represents the auxiliary prediction loss, which is determined based on the losses of the two prediction branches of the multi-word prediction module.

[0307] This is a balancing factor used to control the ratio of the main generation task loss to the auxiliary prediction loss;

[0308] For the loss growth curve function, in the first part of the second stage, =0; in the second part of the second stage The value is smoothly increased according to the half-cycle cosine windowing curve; in the third part of the second stage, =1.

[0309] The defect detection device provided in this application embodiment can be used to execute the defect detection method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0310] Figure 5 A schematic diagram of an electronic device 10, which can be used to implement embodiments of this application, is shown. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, user equipment, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0311] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0312] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks and wireless networks.

[0313] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above.

[0314] In some embodiments, the methods described above can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the methods of any of the embodiments described above by any other suitable means (e.g., by means of firmware).

[0315] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0316] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0317] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0318] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 10, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device 10. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0319] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0320] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0321] This application also provides a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the defect detection method as described in any of the above embodiments.

[0322] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0323] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.< / time> < / type>

Claims

1. A defect detection method characterized by, include: Collect process images during the additive manufacturing process. Each process image is accompanied by corresponding time information and image type, wherein the image type is either a powder spreading image or a printing image. An image group consisting of multiple consecutive process images, along with the corresponding instruction text, is input into a trained detection model to obtain a defect detection result. The detection model includes a visual encoder and a text model. The visual encoder is used to extract visual features for each process image in the image group using a convolutional channel corresponding to the image type. The text model is used to infer and output a defect description text as a defect detection result based on the instruction text corresponding to the image group and the visual features.

2. The method of claim 1, wherein, The detection model is obtained through the following training process: Obtain a training set, which includes multiple sample groups. Each sample group includes multiple sample process images, image type labels and time information labels for each sample process image, sample instruction text, and defect description text labels. In the first stage, the text model is frozen, and the visual encoder is fine-tuned based on the training set, wherein the sample instruction text in the training set is used to describe the image content in each sample group as the task objective. In the second stage, the fine-tuned visual encoder is frozen, and the text model is fine-tuned based on the training set, wherein the instructions in the training set are aimed at analyzing the defect features of each sample group. The sample instruction text includes a time information tag for each sample process image. The time information tag for each sample process image is converted into a text marker and appended before the marker of the corresponding sample process image.

3. The method of claim 2, wherein, The visual encoder includes a backbone network, and each self-attention layer of the backbone network is connected in parallel with a fine-tuning bypass module. The fine-tuning bypass module includes: Dimensionality reduction layer is used to project the features of the input sample process image into a low-rank space to obtain dimensionality-reduced features; The time encoder is used to generate a corresponding timing modulation vector based on the time difference corresponding to each sample process image; The convolution module includes a layer-specific convolutional structure unique to each self-attention layer, and a shared convolutional structure shared by all self-attention layers. Each layer-specific convolutional structure and the shared convolutional structure are used to convert spatial arrangement features into two-dimensional convolutional features. The spatial arrangement features are obtained by reshaping temporally modulated features, which are obtained by element-wise broadcast multiplication of the temporally modulated vector and the reduced-dimensional features. The shared convolutional structure is used to select the corresponding convolutional channel according to the image type to perform convolution operations, obtaining two-dimensional convolutional features. The convolutional channel is either the first channel corresponding to the powder-spreading image or the second channel corresponding to the printed image, where the receptive field of the first channel is larger than that of the second channel. The upscaling layer projects the flattened 2D convolutional features onto the original rank space, scales them, and uses them as bypass features. These features are then added to the output of the corresponding self-attention layer to obtain the visual features.

4. The method of claim 2, wherein, The text model is a large language model; A low-rank adaptation module is connected in parallel on the linear projection weight matrix of each self-attention layer of the large language model. The low-rank adaptation module is used to perform low-rank updates on the linear graph projection weights. The low-rank adaptation module includes a dimensionality reduction layer and a dimensionality increase layer. The last layer decoder of the large language model is followed by a multi-lexical prediction module, which includes a first prediction branch and a second prediction branch. The first prediction branch and the second prediction branch are respectively used to predict the lexical units at subsequent time steps based on the top-level temporal hidden representation of the large language model.

5. The method of claim 4, wherein, The fine-tuning of the text model based on the training set includes: For any given time, the top-level temporal hidden representation: The top-level temporal hidden representation is input into the first prediction branch and the second prediction branch, wherein the first prediction branch is used to predict the first output word at the next time step based on the top-level temporal hidden representation; the second prediction branch is used to predict the second output word at the next time step after that based on the top-level temporal hidden representation. The parameters of the low-rank adaptation module and the multi-term prediction module are updated based on the first output term, the second output term, and the term tags at corresponding positions in the defect description text.

6. The method of claim 4, wherein, The loss function employed by the second stage is represented as ; where, denotes the main generation task loss, and an autoregressive mask cross-entropy loss is adopted; denotes an auxiliary prediction loss determined from losses of two prediction branches of the multi- token prediction module; a balancing factor for controlling a proportion of the main generated task loss and the auxiliary prediction loss; for the loss growth curve function, in a first portion of the second phase, = 0; in a second portion of the second phase, the value of is smoothly ramped up according to a half-cycle cosine ramp curve; in a third portion of the second phase, = 1.

7. A defect detection apparatus characterized by comprising: include: The image acquisition module is used to acquire process images during the additive manufacturing process. Each process image is accompanied by corresponding time information and image type, wherein the image type is either a powder spreading image or a printing image. The defect detection module is used to input an image group consisting of multiple consecutive process images and the corresponding instruction text of the image group into a trained detection model to obtain a defect detection result. The detection model includes a visual encoder and a text model. The visual encoder is used to extract visual features for each process image in the image group using a convolutional channel corresponding to the image type. The text model is used to infer and output a defect description text as a defect detection result based on the instruction text corresponding to the image group and the visual features.

8. An electronic device, comprising: include: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the defect detection method as described in any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the defect detection method as described in any one of claims 1-6.

10. A computer program product comprising computer programs and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the defect detection method as described in any one of claims 1-6.