A method for visually inspecting the appearance of products printed by a 3D printer

Abstract

The present invention relates to a method for visually inspecting the appearance of products printed by a 3D printer, and relates to the field of computer processing technology. The method comprises the following steps: S01: collecting multi-angle surface image data of the printed product using a circularly arranged multispectral visual sensor array, and simultaneously acquiring three-dimensional point cloud data generated by a laser scanner; S02: performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; S03: fusing the first and second appearance models using a feature registration algorithm to generate a fused three-dimensional model, and employing an adaptive weight allocation strategy to process overlapping regions of the models, calculating the confidence weights of each pixel in the overlapping region to eliminate fusion errors. This invention enables real-time feedback, promptly identifying and addressing problems during the printing process, and effectively improving the overall quality of 3D printed products.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to a method for visually detecting the appearance of products printed by a 3D printer. Background Art

[0002] With the popularization of 3D printing technology, the detection of appearance defects of printed products has become a key link in quality control.

[0003] Traditional methods mainly rely on manual visual inspection or single-dimensional optical scanning (refer to Chinese patent publication number CN114062391A, which discloses an inspection method for automatic optical inspection equipment). Optical scanning uses visual sensors, such as high-definition cameras and 3D laser scanning sensors, to obtain images of the product in question. The image is then processed to determine whether the image has problems such as low efficiency, high missed detection rate, and difficulty in identifying internal structural abnormalities. This method is two-dimensional image analysis.

[0004] Due to the drawbacks of two-dimensional image analysis, multiple visual sensors are used to obtain images of multiple surfaces of a product, which are then fused to form a three-dimensional image (refer to Chinese patent publication number CN118781207B, which discloses a method, device, and apparatus for determining a target based on multiple cameras). This is equivalent to performing two-dimensional image analysis on multiple surfaces on the basis of the above-mentioned two-dimensional image analysis, and then combining the multiple two-dimensional images to form a three-dimensional image. However, the fusion part between different two-dimensional images, that is, the overlapping part, requires a lot of calculations to ensure that the fused image is equal to the three-dimensional appearance of the product. In actual operation, there are certain misjudgments, which mainly include errors in the fusion part and loss of features in the fusion part.

[0005] Therefore, how to effectively capture three-dimensional defects such as three-dimensional deformation, interlayer dislocation and uneven material distribution is expected to be well solved. Summary of the Invention

[0006] In response to the above technical problems, the technical solution adopted by the present invention is a method for visually inspecting the appearance of products printed by a 3D printer, comprising the following steps:

[0007] S01. Collect multi-angle surface image data of printed products through a circularly arranged multispectral visual sensor array, and simultaneously obtain three-dimensional point cloud data generated by a laser scanner;

[0008] S02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model;

[0009] S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, and processing the overlapping area of ​​the models using an adaptive weight distribution strategy, calculating the confidence weight of each pixel in the overlapping area to eliminate fusion errors;

[0010] S04, extracting the geometric deviation feature vector of the fused three-dimensional model, and generating a process feature vector by combining the interlayer temperature field data collected in real time during the printing process;

[0011] S05. Input the geometric deviation feature vector and the process feature vector into a pre-trained multimodal defect detection model, and output a deformation index, an interlayer dislocation probability, and a material distribution uniformity parameter;

[0012] S06. When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

[0013] Preferably, the multispectral vision sensor includes:

[0014] At least three groups of industrial cameras arranged in a 120° ring above the printing platform, each group of industrial cameras including at least one visible light camera and one near-infrared camera;

[0015] Install laser line scanners at the four corners of the printing platform with the collection end tilted 45° downward;

[0016] Furthermore, in the step S01 , the surface image data and the three-dimensional point cloud data are collected so as to complete a complete cycle of data collection within 2 seconds after each layer of the printed product is printed.

[0017] Preferably, the fusing of the first appearance model and the second appearance model by a feature registration algorithm in step S03 includes:

[0018] S31, using an improved ICP algorithm to perform coarse registration on the first appearance model and the second appearance model to establish an initial coordinate correspondence relationship;

[0019] S32, constructing overlapping area feature pyramids and calculating SIFT feature point matching degrees at multiple scales;

[0020] S33. Dynamically adjust the registration weight based on the spatial distribution density of feature points:

[0021] ;

[0022] Among them, D i is the point cloud density, C i is the image contrast, , is the adaptive adjustment coefficient, both are 3, D maxis the maximum point cloud density, C max To maximize the image contrast, the low confidence area is then locally re-scanned to fill in the gaps.

[0023] Preferably, the step S03 further includes coordinating the printing layer of the next printed product, which includes:

[0024] S34, extracting geometric deviation features of the fused three-dimensional model and performing correlation analysis with interlayer cooling rate data collected in real time;

[0025] S35. Based on the result of the association analysis, input the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level;

[0026] S36. Based on the obtained output defect type and severity level, a parameter compensation sequence is generated for the corresponding interlayer dislocation defect based on historical process data:

[0027] Print speed correction for the next layer: ;

[0028] Nozzle temperature adjustment curve: ;

[0029] Where T(t) is the nozzle temperature adjustment curve, Δv is the printing speed, δ is the offset, k, τ, ω are material-related correction coefficients, t is time, ΔT is the temperature fluctuation amplitude, ω is the angular frequency, e is 4, and T0 is the standard temperature of the wire.

[0030] Preferably, the input of the pre-trained spatiotemporal convolutional network in step S35 to obtain the output defect type and severity level includes:

[0031] S351, using 3D dilated convolution to extract 3D geometric features, the dilation rate increases from 2 to 6 with the network depth;

[0032] S352: Analyze the process parameters of each printing layer of the printed product using a bidirectional GRU network, create a comparison table of the process parameters of each printing layer, calculate the average value, and then subtract the process parameters of each printing layer, and mark the difference;

[0033] S353, dynamic integration of spatiotemporal features through the gated attention mechanism, the gate value calculation formula is:

[0034] ;

[0035] Among them, F s is the spatial feature, F t is the time feature, W g ,b gis a learnable parameter and σ is 3.

[0036] Preferably, the training method of the multimodal defect detection model in step S05 includes:

[0037] S51. Build a sample library containing three typical defects: surface cracks, interlayer delamination, and thermal deformation. Each sample includes multi-angle visible light, infrared images, high-precision 3D point cloud data, interlayer temperature curve data, and material extrusion pressure time series data.

[0038] S52, based on the dual-channel deep network, the first channel processes the image and point cloud data, and the second channel processes the process parameters;

[0039] S53. Feature interaction is achieved through the cross-channel attention mechanism, and the final output layer contains the three-dimensional deformation heat map and defect classification results.

[0040] Preferably, the dual-channel deep network in step S52 includes:

[0041] S521, using 3D convolution kernel to extract spatial features in the image point cloud channel, each convolution layer is followed by a SE attention module;

[0042] S522, using LSTM network to capture time series features in the process parameter channel;

[0043] S523. Implement hierarchical fusion of dual-channel features through a dynamic routing algorithm, wherein the hierarchical fusion of dual-channel features includes primary fusion for processing local detail features and high-level fusion for processing global semantic features.

[0044] Preferably, the execution basis of step S06 includes:

[0045] When 0.3<deformation index≤0.5, trigger an alarm and record the coordinates;

[0046] When 0.5<deformation index≤0.7, reduce printing speed and enhance local cooling;

[0047] When the deformation index is greater than 0.7, printing is paused and compensation processing is started;

[0048] The probability of inter-layer misalignment is detected using a sliding window. When the probability of misalignment of three consecutive layers is greater than 40%, it is determined to be a structural defect.

[0049] The present invention has at least the following beneficial effects:

[0050] 1. Through the circular arrangement of multi-spectral visual sensor arrays and laser scanners, the simultaneous collection of multi-angle, high-precision surface images and three-dimensional point cloud data of printed products is achieved.

[0051] 2. Through three-dimensional reconstruction, feature registration algorithm fusion, multimodal defect detection model and other steps, three-dimensional defects such as three-dimensional deformation, interlayer dislocation and uneven material distribution are effectively captured, improving the accuracy and efficiency of detection.

[0052] 3. Compared with the disadvantages of traditional manual visual inspection or single-dimensional optical scanning, it has a lower missed detection rate, higher recognition accuracy, and can adjust printing parameters in real time to prevent defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A distribution diagram of industrial cameras provided in Example 1 of the present invention;

[0055] Figure 2 A distribution diagram of the laser line scanner provided in Example 1 of the present invention;

[0056] Figure 3 This is a flow chart of a method for visually inspecting the appearance of products printed by a 3D printer, provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] Example 1:

[0060] This embodiment provides a method for visually inspecting the appearance of products printed by a 3D printer, the method comprising the following steps: Figure 3 As shown:

[0061] S01. Collect multi-angle surface image data of printed products through a circularly arranged multispectral visual sensor array, and simultaneously obtain three-dimensional point cloud data generated by a laser scanner;

[0062] Specifically, the multispectral vision sensor is composed of an industrial camera and a laser line scanner. The laser line scanner acquires three-dimensional point cloud data, while the industrial camera collects multi-angle surface image data of printed products.

[0063] At least three groups of industrial cameras are arranged in a 120° ring above the printing platform, and each group of industrial cameras includes at least one visible light camera and one near-infrared camera ( Figure 1 );

[0064] Set up laser line scanners at the four corners of the printing platform with the collection end tilted 45° downward ( Figure 2 );

[0065] The surface image data and 3D point cloud data acquisition process in step S01 is designed to complete a complete data acquisition cycle within 2 seconds after each layer of the printed product is printed. Specifically, the circularly arranged multispectral vision sensor array and laser scanner simultaneously acquire multi-angle surface image data and 3D point cloud data from the printed product within 2 seconds of each layer's completion, allowing for subsequent processing. This arrangement of the multispectral vision sensors and laser scanners, along with the required acquisition timeframe, ensures comprehensive and timely data acquisition, providing strong support for subsequent model generation and defect detection.

[0066] S02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model;

[0067] Specifically, an industrial camera captures multi-angle surface images, which are then reconstructed into a first appearance model through 3D reconstruction. A laser scanner then acquires submicron-level 3D point cloud data to generate a second appearance model. This 3D reconstruction involves fusing images captured by multiple industrial cameras, overlapping identical parts. This is a prior art technique and will not be elaborated on in detail.

[0068] S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, and using an adaptive weight distribution strategy to process the overlapping area of ​​the models, and calculating the confidence weight of each pixel in the overlapping area to eliminate fusion errors;

[0069] Specifically, the fusing of the first appearance model and the second appearance model by the feature registration algorithm in step S03 includes:

[0070] S31, using an improved ICP algorithm to perform coarse registration on the first appearance model and the second appearance model to establish an initial coordinate correspondence relationship;

[0071] S32, constructing overlapping area feature pyramids and calculating SIFT feature point matching degrees at multiple scales;

[0072] S33. Dynamically adjust the registration weight based on the spatial distribution density of feature points:

[0073] ;

[0074] Among them, D i is the point cloud density, C i is the image contrast, , is the adaptive adjustment coefficient, both are 3, D max is the maximum point cloud density, C max To maximize the image contrast, the low confidence area is then locally re-scanned to fill in the gaps.

[0075] In this approach, the first appearance model obtained by 3D reconstruction based on surface image data and the second appearance model obtained by conversion of 3D point cloud data are fused. A coarse registration is performed using an improved ICP algorithm. A feature pyramid is constructed in the overlapping areas to calculate the matching degree of SIFT feature points. The registration weights are dynamically adjusted based on the spatial distribution density of the feature points. Low-confidence areas are then locally re-scanned to fill in gaps. This improves the accuracy and efficiency of model fusion and reduces fusion errors and feature loss.

[0076] S04, extracting the geometric deviation feature vector of the fused 3D model, and generating a process feature vector by combining the interlayer temperature field data collected in real time during the printing process;

[0077] Specifically, a fused 3D model (known in the prior art) is generated by processing multi-angle surface image data and 3D point cloud data. This fused 3D model then extracts geometric deviation feature vectors, which describe the deviations between the model and the ideal shape, such as geometric features like protrusions, depressions, and distortions. During the printing process, interlayer temperature field data is collected in real time. This data reflects the temperature distribution and variations within each layer during the printing process. Temperature is a key factor affecting 3D printing quality, primarily influencing the material's melting, flow, and solidification processes, thereby affecting the shape and performance of the printed product. The geometric deviation feature vectors and interlayer temperature field data are combined to generate a process feature vector. This process feature vector contains both geometric deviation information and temperature information during the printing process. Comprehensive analysis of the geometric deviation feature vectors and process feature vectors provides a more comprehensive understanding of the quality and potential defects of the printed product. This provides a foundation for subsequently inputting the geometric deviation feature vectors and process feature vectors into a pre-trained multimodal defect detection model to output key metrics such as deformation index, interlayer misalignment probability, and material distribution uniformity parameters.

[0078] S05. Input the geometric deviation feature vector and the process feature vector into a pre-trained multimodal defect detection model, and output a deformation index, an interlayer dislocation probability, and a material distribution uniformity parameter;

[0079] In summary, based on the geometric deviation feature vector and process feature vector provided in step S04, they are imported into a pre-trained multimodal defect detection model to process data from two different modalities, namely, geometry and process, to more comprehensively evaluate the quality of the printed product.

[0080] Once these two eigenvectors are input into the model, it performs a series of complex calculations and analyses, ultimately outputting three key parameters: deformation index, interlayer misalignment probability, and material distribution uniformity parameter. The deformation index reflects the degree of deviation from the ideal shape of the printed part; the interlayer misalignment probability reveals whether there are defects such as misalignment between layers; and the material distribution uniformity parameter assesses the uniformity of material distribution within the printed part, determining whether there are areas of excess or insufficient material.

[0081] Specifically, further, the training method of the multimodal defect detection model in step S05 includes:

[0082] S51. Build a sample library containing three typical defects: surface cracks, interlayer delamination, and thermal deformation. Each sample includes multi-angle visible light, infrared images, high-precision 3D point cloud data, interlayer temperature curve data, and material extrusion pressure time series data.

[0083] S52, based on the dual-channel deep network, the first channel processes the image and point cloud data, and the second channel processes the process parameters;

[0084] S53. Feature interaction is achieved through the cross-channel attention mechanism, and the final output layer contains the three-dimensional deformation heat map and defect classification results.

[0085] The training method for the multimodal defect detection model in the above embodiment involves constructing a sample library containing three typical defect categories. Using a dual-channel deep network, the model processes image and point cloud data, as well as process parameters, respectively. This model then interacts with the process through a cross-channel attention mechanism, ultimately outputting a 3D deformation heat map and defect classification results. This enhances the model's generalization and recognition accuracy, providing robust support for defect detection in printed products.

[0086] It should be noted that the dual-channel deep network in step S52 above includes:

[0087] S521, using 3D convolution kernel to extract spatial features in the image point cloud channel, each convolution layer is followed by a SE attention module;

[0088] S522, using LSTM network to capture time series features in the process parameter channel;

[0089] S523. Implement hierarchical fusion of dual-channel features through a dynamic routing algorithm, wherein the hierarchical fusion of dual-channel features includes primary fusion for processing local detail features and high-level fusion for processing global semantic features.

[0090] In the dual-channel deep network described above, the image point cloud channel uses a 3D convolution kernel to extract spatial features, with each convolution layer followed by an SE attention module. The process parameter channel uses an LSTM network to capture time series features. A dynamic routing algorithm is used to achieve hierarchical fusion of dual-channel features. This improves the model's ability to process images and process parameters, further enhancing the accuracy and efficiency of defect detection.

[0091] S06. When any parameter exceeds a preset threshold, a three-dimensional defect is determined to exist and an early warning is issued. The specific identification strategy includes:

[0092] When 0.3<deformation index≤0.5, trigger an alarm and record the coordinates;

[0093] When 0.5<deformation index≤0.7, reduce printing speed and enhance local cooling;

[0094] When the deformation index is greater than 0.7, printing is paused and compensation processing is started;

[0095] The probability of inter-layer misalignment is detected using a sliding window. When the probability of misalignment of three consecutive layers is greater than 40%, it is determined to be a structural defect.

[0096] The above embodiment triggers different warning and response measures based on thresholds set for deformation index and interlayer misalignment probability, such as recording coordinates, reducing printing speed, enhancing local cooling, pausing printing, and initiating compensatory processing. This enables precise control of the printing process and timely handling of defects, effectively ensuring the quality of printed products.

[0097] In summary, this first embodiment aims to achieve the simultaneous acquisition of multi-angle, high-precision surface images and three-dimensional point cloud data of printed products through a ring-shaped multispectral visual sensor array and laser scanner. Furthermore, through steps such as three-dimensional reconstruction, feature registration algorithm fusion, and a multimodal defect detection model, it effectively captures three-dimensional defects such as three-dimensional deformation, interlayer misalignment, and uneven material distribution, thereby improving detection accuracy and efficiency. Furthermore, compared with traditional manual visual inspection or single-dimensional optical scanning methods, this invention has a lower missed detection rate, higher recognition accuracy, and can adjust printing parameters in real time to prevent defects.

[0098] Example 2:

[0099] This embodiment aims to provide a coordination method for a printing device based on the first embodiment. Its purpose is to perform coordination when the first embodiment detects an error in a printed layer of a printed product, or to coordinate the second layer, thereby modifying the printing parameters back to the correct range. If an error still occurs in the third layer, the processing of steps S04 to S06 is executed. Specifically:

[0100] Step S03 also includes coordination of the printing layer of the next printed product, which includes:

[0101] S34, extracting geometric deviation features of the fused 3D model and performing correlation analysis with interlayer cooling rate data collected in real time;

[0102] S35. Based on the result of the association analysis, input the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level;

[0103] S36. Based on the acquired output defect type and severity level, a parameter compensation sequence is generated for the corresponding interlayer dislocation defect based on historical process data:

[0104] Print speed correction for the next layer: ;

[0105] Nozzle temperature adjustment curve: ;

[0106] Where T(t) is the nozzle temperature adjustment curve, Δv is the printing speed, δ is the offset, k, τ, ω are material-related correction coefficients, t is time, ΔT is the temperature fluctuation amplitude, ω is the angular frequency, e is 4, and T0 is the standard temperature of the wire.

[0107] Based on the fusion of the 3D model, geometric deviation features are extracted and correlated with real-time interlayer cooling rate data. This information is then fed into a pre-trained spatiotemporal convolutional network to determine defect type and severity. A parameter compensation sequence is then generated based on historical process data to coordinate the printing of the next layer. This enables real-time monitoring and adjustment of the printing process, effectively preventing defects and improving print quality.

[0108] Furthermore, in step S35, the pre-trained spatiotemporal convolutional network is input to obtain the output defect type and severity level, including:

[0109] S351, using 3D dilated convolution to extract 3D geometric features, the dilation rate increases from 2 to 6 with the network depth;

[0110] S352: Use a bidirectional GRU network to analyze the process parameters of each printing layer of the printed product, and create a comparison table of the process parameters of each printing layer. After obtaining the average value, make a difference between the process parameters of each printing layer, and mark the difference.

[0111] S353. Dynamically integrate spatiotemporal features through the gated attention mechanism. The gate value calculation formula is:

[0112] ;

[0113] Among them, F s is the spatial feature, F t is the time feature, W g ,b g is a learnable parameter and σ is 3.

[0114] The above-mentioned spatiotemporal convolutional network extracts 3D geometric features through 3D dilated convolution, uses a bidirectional GRU network to analyze process parameters, and dynamically fuses spatiotemporal features through a gated attention mechanism to output defect types and severity levels. This improves the accuracy and efficiency of defect identification and provides a reliable basis for subsequent parameter compensation and printing adjustments.

[0115] Example 3:

[0116] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps:

[0117] The multi-spectral visual sensor array arranged in a ring collects multi-angle surface image data of the printed product and simultaneously obtains the three-dimensional point cloud data generated by the laser scanner;

[0118] Performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model;

[0119] The first appearance model and the second appearance model are fused through a feature registration algorithm to generate a fused 3D model. An adaptive weight distribution strategy is used to process the overlapping areas of the models, and the confidence weight of each pixel in the overlapping area is calculated to eliminate fusion errors.

[0120] Extract the geometric deviation feature vector of the fused 3D model and generate the process feature vector by combining the interlayer temperature field data collected in real time during the printing process;

[0121] The geometric deviation feature vector and the process feature vector are input into the pre-trained multimodal defect detection model to output the deformation index, interlayer misalignment probability and material distribution uniformity parameters;

[0122] When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0124] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0125] Example 4:

[0126] An embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the following steps:

[0127] The multi-spectral visual sensor array arranged in a ring collects multi-angle surface image data of the printed product and simultaneously obtains the three-dimensional point cloud data generated by the laser scanner;

[0128] Performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model;

[0129] The first appearance model and the second appearance model are fused through a feature registration algorithm to generate a fused 3D model. An adaptive weight distribution strategy is used to process the overlapping areas of the models, and the confidence weight of each pixel in the overlapping area is calculated to eliminate fusion errors.

[0130] Extract the geometric deviation feature vector of the fused 3D model and generate the process feature vector by combining the interlayer temperature field data collected in real time during the printing process;

[0131] The geometric deviation feature vector and the process feature vector are input into the pre-trained multimodal defect detection model to output the deformation index, interlayer misalignment probability and material distribution uniformity parameters;

[0132] When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

[0133] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for visually inspecting the appearance of products printed by a 3D printer, characterized in that: The following steps are involved: S01. Collect multi-angle surface image data of printed products through a circularly arranged multispectral visual sensor array, and simultaneously obtain three-dimensional point cloud data generated by a laser scanner; S02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, and processing the overlapping area of ​​the models using an adaptive weight distribution strategy, calculating the confidence weight of each pixel in the overlapping area to eliminate fusion errors; S04, extracting the geometric deviation feature vector of the fused three-dimensional model, and generating a process feature vector by combining the interlayer temperature field data collected in real time during the printing process; S05. Input the geometric deviation feature vector and the process feature vector into a pre-trained multimodal defect detection model, and output a deformation index, an interlayer dislocation probability, and a material distribution uniformity parameter; S06. When any output parameter in step S05 exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued; The execution basis of step S06 includes: When 0.3<deformation index≤0.5, trigger an alarm and record the coordinates; When 0.5<deformation index≤0.7, reduce printing speed and enhance local cooling; When the deformation index is greater than 0.7, printing is paused and compensation processing is started; The probability of inter-layer misalignment is detected using a sliding window. When the probability of misalignment of three consecutive layers is greater than 40%, it is determined to be a structural defect. The step S03 also includes coordinating the printing layer of the next printed product, which includes: S34, extracting geometric deviation features of the fused three-dimensional model and performing correlation analysis with interlayer cooling rate data collected in real time; S35. Based on the result of the association analysis, input the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level; S36. Based on the obtained output defect type and severity level, a parameter compensation sequence is generated for the corresponding interlayer dislocation defect based on historical process data: Print speed correction for the next layer: ; Nozzle temperature adjustment curve: ; in, is the nozzle temperature adjustment curve, Δv is the printing speed, δ is the offset, k, τ are material-related correction coefficients, t is time, ΔT is the temperature fluctuation amplitude, ω is the angular frequency, e is 4, and T0 is the standard temperature of the wire.

2. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, characterized in that: The multispectral vision sensor comprises: At least three groups of industrial cameras arranged in a 120° ring above the printing platform, each group of industrial cameras including at least one visible light camera and one near-infrared camera; Install laser line scanners at the four corners of the printing platform with the collection end tilted 45° downward; Furthermore, in the step S01 , the surface image data and the three-dimensional point cloud data are collected so as to complete a complete cycle of data collection within 2 seconds after each layer of the printed product is printed.

3. The method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, wherein: The fusing of the first appearance model and the second appearance model by the feature registration algorithm in step S03 includes: S31, using an improved ICP algorithm to perform coarse registration on the first appearance model and the second appearance model to establish an initial coordinate correspondence relationship; S32, constructing overlapping area feature pyramids and calculating SIFT feature point matching degrees at multiple scales; S33. Dynamically adjust the registration weight based on the spatial distribution density of feature points: ； Among them, D i is the point cloud density, C i is the image contrast, is the adaptive adjustment coefficient, both are 3, D max is the maximum point cloud density, C max To maximize the image contrast, the low confidence area is then locally re-scanned to fill in the gaps.

4. The method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, wherein: The step S35 of inputting the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level includes: S351, using 3D dilated convolution to extract 3D geometric features, the dilation rate increases from 2 to 6 with the network depth; S352: Analyze the process parameters of each printing layer of the printed product using a bidirectional GRU network, create a comparison table of the process parameters of each printing layer, calculate the average value, and then subtract the process parameters of each printing layer, and mark the difference; S353. Dynamically integrate spatiotemporal features through the gated attention mechanism. The gate value calculation formula is: ； Among them, F s is the spatial feature, F t is the time feature, W g ,b g is a learnable parameter and σ is 3.

5. The method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, wherein: The training method of the multimodal defect detection model in step S05 includes: S51. Build a sample library containing three typical defects: surface cracks, interlayer delamination, and thermal deformation. Each sample includes multi-angle visible light, infrared images, high-precision 3D point cloud data, interlayer temperature curve data, and material extrusion pressure time series data. S52, based on the dual-channel deep network, the first channel processes the image and point cloud data, and the second channel processes the process parameters; S53. Feature interaction is achieved through the cross-channel attention mechanism, and the final output layer contains the three-dimensional deformation heat map and defect classification results.

6. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 5, characterized in that: The dual-channel deep network in step S52 includes: S521, using 3D convolution kernel to extract spatial features in the image point cloud channel, each convolution layer is followed by a SE attention module; S522, using LSTM network to capture time series features in the process parameter channel; S523. Implement hierarchical fusion of dual-channel features through a dynamic routing algorithm, wherein the hierarchical fusion of dual-channel features includes primary fusion for processing local detail features and high-level fusion for processing global semantic features.

7. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the steps of the method for visually inspecting the appearance of products printed by a 3D printer as described in any one of claims 1 to 6.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the method for visually inspecting the appearance of a product printed by a 3D printer as described in any one of claims 1 to 6.

Images