Visual inspection method for appearance of product printed by 3D printer

Abstract

The invention provides a 3D printer printed product appearance visual detection method, and relates to the technical field of computer processing, and the method comprises the following steps: S01, collecting multi-angle surface image data of a printed product through an annularly arranged multispectral visual sensor array, and synchronously obtaining three-dimensional point cloud data generated by a laser scanner; s02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; and S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, processing a model overlapping region by adopting a self-adaptive weight distribution strategy, and calculating a confidence coefficient weight of each pixel point in the overlapping region to eliminate a fusion error. Real-time feedback can be achieved, problems in the printing process can be found and processed in time, and the overall quality of a 3D printing product is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to a method for visually detecting the appearance of products printed by a 3D printer. Background Art

[0002] With the popularization of 3D printing technology, the detection of appearance defects of printed products has become a key link in quality control.

[0003] Traditional methods mainly rely on manual visual inspection or single-dimensional optical scanning (refer to Chinese patent, publication number CN114062391A, which discloses a detection method of an automatic optical detection device), while optical scanning uses visual sensors, such as high-definition cameras, 3D laser scanning sensors and other devices to obtain images of the product in this application, and then uses image processing to determine whether the image has problems such as low efficiency, high missed detection rate, and difficulty in identifying internal structural abnormalities. This method is two-dimensional image analysis.

[0004] Due to the drawbacks of two-dimensional image analysis, multiple visual sensors are used to obtain images of multiple surfaces of the product, which are fused to form a three-dimensional image (refer to Chinese patent, publication number CN118781207B, which discloses a method, device and equipment for determining targets based on multiple cameras). This is equivalent to performing two-dimensional image analysis on multiple surfaces on the basis of the above-mentioned two-dimensional image analysis, and then combining multiple two-dimensional images to form a three-dimensional image. However, the fusion part between different two-dimensional images, that is, the overlapping part, requires a lot of calculations to ensure that the fused image is equal to the three-dimensional appearance of the product. There are certain misjudgments in the actual operation process, and the misjudgments mainly include errors in the fusion part, loss of features in the fusion part, and other problems.

[0005] Therefore, how to effectively capture three-dimensional defects such as three-dimensional deformation, interlayer dislocation and uneven material distribution is expected to be well solved. Summary of the invention

[0006] In view of the above technical problems, the technical solution adopted by the present invention is a method for visually inspecting the appearance of products printed by a 3D printer, comprising the following steps: S01, collecting multi-angle surface image data of printed products through a multi-spectral visual sensor array arranged in a ring, and synchronously acquiring three-dimensional point cloud data generated by a laser scanner; S02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, and using an adaptive weight allocation strategy to process the overlapping area of ​​the models, and calculating the confidence weight of each pixel in the overlapping area to eliminate fusion errors; S04, extracting the geometric deviation feature vector of the fused three-dimensional model, and generating a process feature vector in combination with the interlayer temperature field data collected in real time during the printing process; S05, inputting the geometric deviation feature vector and the process feature vector into a pre-trained multimodal defect detection model, and outputting a deformation index, an interlayer dislocation probability, and a material distribution uniformity parameter; S06. When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

[0007] Preferably, the multispectral vision sensor comprises: At least three groups of industrial cameras are arranged in a 120° ring above the printing platform, each group of industrial cameras includes at least one visible light camera and one near infrared camera; A laser line scanner with the collection end tilted 45° downward is installed at the four corners of the printing platform; In the step S01, the surface image data and the three-dimensional point cloud data are collected so as to complete a complete cycle of data collection within 2 seconds after each layer of the printed product is printed.

[0008] Preferably, the fusing of the first appearance model and the second appearance model by a feature registration algorithm in step S03 includes: S31, using an improved ICP algorithm to roughly align the first appearance model with the second appearance model to establish an initial coordinate correspondence relationship; S32, constructing a feature pyramid of overlapping regions and calculating the matching degree of SIFT feature points at multiple scales; S33. Dynamically adjust the registration weight based on the spatial distribution density of feature points: ; Among them, D i is the point cloud density, C i is the image contrast, , is the adaptive adjustment coefficient, both are 3, D max is the maximum point cloud density, C max To maximize the image contrast, the low confidence area is then locally rescanned to fill in the gaps.

[0009] Preferably, the step S03 also includes coordinating the printing layer of the next printed product, and the steps include: S34, extracting the geometric deviation features of the fused three-dimensional model, and performing correlation analysis with the interlayer cooling rate data collected in real time; S35, based on the result of the association analysis, input the pre-trained spatiotemporal convolutional network to obtain output defect types and severity levels; S36. Based on the obtained output defect type and severity level, for the corresponding interlayer dislocation defect, generate a parameter compensation sequence based on historical process data: Print speed correction for the next layer: ; Nozzle temperature adjustment curve: ; Where T(t) is the nozzle temperature adjustment curve, Δv is the printing speed, δ is the offset, k, τ, ω are material-related correction coefficients, t is time, ΔT is the temperature fluctuation amplitude, ω is the angular frequency, e is 4, T 0 is the standard temperature of the wire.

[0010] Preferably, the input of the pre-trained spatiotemporal convolutional network in step S35 to obtain the output defect type and severity level includes: S351, 3D dilated convolution is used to extract 3D geometric features, and the dilation rate increases from 2 to 6 with the network depth; S352, using a bidirectional GRU network to analyze the process parameters of each printing layer of the printed product, and making a comparison table of the process parameters of each printing layer, calculating the average value and then subtracting the process parameters of each printing layer, and marking the difference; S353, through the gated attention mechanism, the spatiotemporal features are dynamically integrated, and the gated value calculation formula is: ; Among them, F s is the spatial feature, F t is the time feature, W g ,b g is a learnable parameter and σ is 3.

[0011] Preferably, the training method of the multimodal defect detection model in step S05 includes: S51. Construct a sample library containing three typical defects: surface cracks, interlayer peeling, and thermal deformation. Each sample includes multi-angle visible light, infrared images, high-precision three-dimensional point cloud data, interlayer temperature curve data, and material extrusion pressure time series data; S52, based on the dual-channel deep network, the first channel processes the image and point cloud data, and the second channel processes the process parameters; S53. Feature interaction is achieved through the cross-channel attention mechanism, and the final output layer contains the three-dimensional deformation heat map and defect classification results.

[0012] Preferably, the dual-channel deep network in step S52 includes: S521, using 3D convolution kernel to extract spatial features in the image point cloud channel, each convolution layer is followed by a SE attention module; S522, using LSTM network to capture time series features in the process parameter channel; S523. Implement hierarchical fusion of dual-channel features through a dynamic routing algorithm, wherein the hierarchical fusion of dual-channel features includes primary fusion for processing local detail features and advanced fusion for processing global semantic features.

[0013] Preferably, the execution basis of step S06 includes: When 0.3<deformation index≤0.5, an early warning is triggered and the coordinates are recorded; When 0.5<deformation index≤0.7, reduce printing speed and enhance local cooling; When the deformation index is > 0.7, printing is paused and compensation processing is started; The probability of inter-layer misalignment is detected by sliding window, and it is judged as a structural defect when the probability of misalignment of three consecutive layers is greater than 40%.

[0014] The present invention has at least the following beneficial effects: 1. Through the circularly arranged multi-spectral visual sensor array and laser scanner, the simultaneous collection of multi-angle, high-precision surface images and three-dimensional point cloud data of printed products is achieved.

[0015] 2. Through three-dimensional reconstruction, feature registration algorithm fusion, multimodal defect detection model and other steps, three-dimensional defects such as three-dimensional deformation, interlayer dislocation and uneven material distribution are effectively captured, improving the accuracy and efficiency of detection.

[0016] 3. Compared with the disadvantages of traditional manual visual inspection or single-dimensional optical scanning, it has a lower missed detection rate, higher recognition accuracy, and can adjust printing parameters in real time to prevent defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A distribution diagram of industrial cameras provided in Embodiment 1 of the present invention; Figure 2 A distribution diagram of a laser line scanner provided in Embodiment 1 of the present invention; Figure 3A flow chart of a method for visually inspecting the appearance of products printed by a 3D printer provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] Embodiment 1:

[0022] This embodiment provides a method for visually inspecting the appearance of a product printed by a 3D printer, the method comprising the following steps: Figure 3 As shown: S01, collecting multi-angle surface image data of printed products through a multi-spectral visual sensor array arranged in a ring, and synchronously acquiring three-dimensional point cloud data generated by a laser scanner; Specifically, the multispectral vision sensor is composed of an industrial camera and a laser line scanner, wherein the laser line scanner acquires three-dimensional point cloud data, and the industrial camera acquires multi-angle surface image data of the printed product. In detail: At least three groups of industrial cameras are arranged in a 120° ring above the printing platform, each group of industrial cameras includes at least one visible light camera and one near infrared camera ( Figure 1 ); Set up laser line scanners at the four corners of the printing platform with the collection end tilted 45° downward ( Figure 2 ); And in the step S01, the surface image data and three-dimensional point cloud data are collected to complete a complete cycle of data collection within 2 seconds after each layer of the printed product is printed. That is, through the multi-spectral visual sensor array and laser scanner arranged in a ring, the multi-angle surface image data and three-dimensional point cloud data of the printed product are collected synchronously within 2 seconds after each layer is printed and processed in subsequent steps. Through the arrangement of the multi-spectral visual sensor and laser scanner and the time requirements of the collection behavior, the comprehensiveness and timeliness of data collection are ensured, which provides strong support for subsequent model generation and defect detection.

[0023] S02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; Specifically, an industrial camera is used to collect multi-angle surface images, and a first appearance model is generated through 3D reconstruction. Then, a laser scanner is used to obtain sub-micron 3D point cloud data to generate a second appearance model. The above 3D reconstruction refers to the fusion of images collected by multiple industrial cameras, that is, overlapping the same parts. It belongs to the existing technology and will not be elaborated in detail.

[0024] S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, and using an adaptive weight allocation strategy to process the overlapping area of ​​the models, and calculating the confidence weight of each pixel in the overlapping area to eliminate fusion errors; Specifically, in step S03, fusing the first appearance model and the second appearance model by using a feature registration algorithm includes: S31, using an improved ICP algorithm to roughly align the first appearance model with the second appearance model to establish an initial coordinate correspondence relationship; S32, constructing a feature pyramid of overlapping regions and calculating the matching degree of SIFT feature points at multiple scales; S33. Dynamically adjust the registration weight based on the spatial distribution density of feature points: ; Among them, D i is the point cloud density, C i is the image contrast, , is the adaptive adjustment coefficient, both are 3, D max is the maximum point cloud density, C max To maximize the image contrast, the low confidence area is then locally rescanned to fill in the gaps.

[0025] In the above, after fusing the first appearance model obtained by 3D reconstruction based on surface image data and the second appearance model obtained by conversion of 3D point cloud data, the improved ICP algorithm is used for rough registration, the overlapping area feature pyramid is constructed to calculate the SIFT feature point matching degree, and the registration weight is dynamically adjusted based on the spatial distribution density of the feature points, and the low confidence area is locally scanned again to fill in the gaps. This can improve the accuracy and efficiency of model fusion and reduce the problems of fusion error and feature loss.

[0026] S04, extracting the geometric deviation feature vector of the fused three-dimensional model, and generating a process feature vector in combination with the interlayer temperature field data collected in real time during the printing process; Specifically, by processing multi-angle surface image data and three-dimensional point cloud data, a fused three-dimensional model (belonging to the prior art) is generated. Then, the fused three-dimensional model extracts the geometric deviation feature vector, and the geometric deviation feature vector describes the deviation between the model and the ideal shape, such as geometric features such as convexity, concaveness, distortion, etc. And during the printing process, the interlayer temperature field data will be collected in real time. The above data reflects the temperature distribution and change of each layer during the printing process. Temperature is one of the important factors affecting the quality of 3D printing. It mainly affects the melting, flow and solidification process of the material, thereby affecting the shape and performance of the printed product. The geometric deviation feature vector and the interlayer temperature field data are combined to generate a process feature vector. The process feature vector contains both the geometric deviation information of the printed product and the temperature information during the printing process. By comprehensively analyzing the information of the geometric deviation feature vector and the process feature vector, the quality and possible defects of the printed product can be more comprehensively understood. It provides a basis for the subsequent input of the geometric deviation feature vector and the process feature vector into the pre-trained multimodal defect detection model to output key indicators such as deformation index, interlayer dislocation probability and material distribution uniformity parameter.

[0027] S05. Input the geometric deviation feature vector and the process feature vector into a pre-trained multimodal defect detection model, and output a deformation index, an interlayer dislocation probability, and a material distribution uniformity parameter; In summary, based on the geometric deviation feature vector and the process feature vector provided in step S04, they are imported into a pre-trained multi-modal defect detection model to process data of two different modalities, geometry and process, so as to more comprehensively evaluate the quality of the printed product.

[0028] When these two eigenvectors are input into the model, the model will perform a series of complex calculations and analyses, and finally output three key parameters: deformation index, interlayer misalignment probability, and material distribution uniformity parameter. The deformation index reflects the degree of deviation of the shape of the printed product from the ideal shape; the interlayer misalignment probability reveals whether there are defects such as misalignment between layers of the printed product; and the material distribution uniformity parameter evaluates whether the distribution of materials in the printed product is uniform, and whether there is too much or too little material in some parts.

[0029] Specifically, further, the training method of the multimodal defect detection model in step S05 includes: S51. Construct a sample library containing three typical defects: surface cracks, interlayer peeling, and thermal deformation. Each sample includes multi-angle visible light, infrared images, high-precision three-dimensional point cloud data, interlayer temperature curve data, and material extrusion pressure time series data; S52, based on the dual-channel deep network, the first channel processes the image and point cloud data, and the second channel processes the process parameters; S53. Feature interaction is achieved through the cross-channel attention mechanism, and the final output layer contains the three-dimensional deformation heat map and defect classification results.

[0030] The training method of the multimodal defect detection model in the above embodiment includes building a sample library containing three types of typical defects, processing image and point cloud data and process parameters based on a dual-channel deep network, and realizing feature interaction through a cross-channel attention mechanism, and finally outputting a three-dimensional deformation heat map and defect classification results. This enhances the generalization ability and recognition accuracy of the model, providing a guarantee for defect detection of printed products.

[0031] It should be noted that the dual-channel deep network in the above step S52 includes: S521, using 3D convolution kernel to extract spatial features in the image point cloud channel, each convolution layer is followed by a SE attention module; S522, using LSTM network to capture time series features in the process parameter channel; S523. Implement hierarchical fusion of dual-channel features through a dynamic routing algorithm, wherein the hierarchical fusion of dual-channel features includes primary fusion for processing local detail features and advanced fusion for processing global semantic features.

[0032] The image point cloud channel in the dual-channel deep network in the above embodiment uses a 3D convolution kernel to extract spatial features, and each convolution layer is followed by an SE attention module; the process parameter channel uses an LSTM network to capture time series features; and the hierarchical fusion of dual-channel features is achieved through a dynamic routing algorithm. This improves the model's ability to process images and process parameters, and further improves the accuracy and efficiency of defect detection.

[0033] S06. When any parameter exceeds a preset threshold, it is determined that there is a three-dimensional defect and an early warning is issued. The specific resolution strategy includes: When 0.3<deformation index≤0.5, an early warning is triggered and the coordinates are recorded; When 0.5<deformation index≤0.7, reduce printing speed and enhance local cooling; When the deformation index is > 0.7, printing is paused and compensation processing is started; The probability of inter-layer misalignment is detected by sliding window, and it is judged as a structural defect when the probability of misalignment of three consecutive layers is greater than 40%.

[0034] The above embodiment triggers different warnings and response measures according to the threshold settings of deformation index and interlayer misalignment probability, such as recording coordinates, reducing printing speed, enhancing local cooling, pausing printing and starting compensation processing, etc. It realizes precise control of the printing process and timely processing of defects, effectively ensuring the quality of printed products.

[0035] In summary, the first embodiment of the present invention aims to achieve the simultaneous acquisition of multi-angle, high-precision surface images and three-dimensional point cloud data of printed products through a ring-shaped multi-spectral visual sensor array and laser scanner. And through the steps of three-dimensional reconstruction, feature registration algorithm fusion, multi-modal defect detection model, etc., three-dimensional defects such as three-dimensional deformation, interlayer dislocation and uneven material distribution are effectively captured, and the accuracy and efficiency of detection are improved. Compared with traditional manual visual inspection or single-dimensional optical scanning methods, the present invention has a lower missed detection rate, higher recognition accuracy, and can adjust printing parameters in real time to prevent the occurrence of defects.

[0036] Embodiment 2:

[0037] This embodiment aims to provide a coordination method for a printing device based on the first embodiment, and its purpose is to coordinate when the first embodiment detects that an error occurs in the printing layer of the printed product, or coordinate the second layer, so as to modify the printing parameters back to the correct range. If an error still occurs in the third layer, the processing of step S04 to step S06 is executed. Specifically: Step S03 also includes coordination of the printing layer of the next printed product, and the steps include: S34, extracting geometric deviation features of the fused three-dimensional model, and performing correlation analysis with interlayer cooling rate data collected in real time; S35. Based on the result of the association analysis, input the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level; S36. Based on the acquired output defect type and severity level, a parameter compensation sequence is generated for the corresponding interlayer dislocation defect based on historical process data: Print speed correction for the next layer: ; Nozzle temperature adjustment curve: ; Where T(t) is the nozzle temperature adjustment curve, Δv is the printing speed, δ is the offset, k, τ, ω are material-related correction coefficients, t is time, ΔT is the temperature fluctuation amplitude, ω is the angular frequency, e is 4, T 0 is the standard temperature of the wire.

[0038] Based on the fusion of the 3D model, the geometric deviation features are extracted and correlated with the interlayer cooling rate data collected in real time. The pre-trained spatiotemporal convolutional network is input to obtain the defect type and severity level, and the parameter compensation sequence is generated based on the historical process data to coordinate the printing of the next layer. Real-time monitoring and adjustment of the printing process is achieved, which effectively prevents the occurrence of defects and improves the printing quality.

[0039] Furthermore, in step S35, the pre-trained spatiotemporal convolutional network is input to obtain output defect types and severity levels, including: S351, 3D dilated convolution is used to extract 3D geometric features, and the dilation rate increases from 2 to 6 with the network depth; S352, using a bidirectional GRU network to analyze the process parameters of each printing layer of the printed product, and making a comparison table of the process parameters of each printing layer, calculating the average value and then subtracting the process parameters of each printing layer, and marking the difference; S353, through the gated attention mechanism, the spatiotemporal features are dynamically integrated, and the gated value calculation formula is: ; Among them, F s is the spatial feature, F t is the time feature, W g ,b g is a learnable parameter and σ is 3.

[0040] The above-mentioned spatiotemporal convolutional network extracts 3D geometric features through 3D hole convolution, uses a bidirectional GRU network to analyze process parameters, and dynamically fuses spatiotemporal features through a gated attention mechanism to obtain the output defect type and severity level. This improves the accuracy and efficiency of defect identification and provides a reliable basis for subsequent parameter compensation and printing adjustment.

[0041] Embodiment three:

[0042] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps: The multi-spectral visual sensor array arranged in a ring collects multi-angle surface image data of the printed product and simultaneously obtains the three-dimensional point cloud data generated by the laser scanner; Performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; The first appearance model and the second appearance model are fused through a feature registration algorithm to generate a fused 3D model, and an adaptive weight allocation strategy is used to process the overlapping area of ​​the models, and the confidence weight of each pixel in the overlapping area is calculated to eliminate the fusion error; Extract the geometric deviation feature vector of the fused 3D model, and generate the process feature vector by combining the interlayer temperature field data collected in real time during the printing process; The geometric deviation feature vector and the process feature vector are input into the pre-trained multimodal defect detection model to output the deformation index, interlayer dislocation probability and material distribution uniformity parameters; When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

[0043] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0044] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0045] Embodiment 4:

[0046] An embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps: The multi-spectral visual sensor array arranged in a ring collects multi-angle surface image data of the printed product and simultaneously obtains the three-dimensional point cloud data generated by the laser scanner; Performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; The first appearance model and the second appearance model are fused through a feature registration algorithm to generate a fused 3D model, and an adaptive weight allocation strategy is used to process the overlapping area of ​​the models, and the confidence weight of each pixel in the overlapping area is calculated to eliminate the fusion error; Extract the geometric deviation feature vector of the fused 3D model, and generate the process feature vector by combining the interlayer temperature field data collected in real time during the printing process; The geometric deviation feature vector and the process feature vector are input into the pre-trained multimodal defect detection model to output the deformation index, interlayer dislocation probability and material distribution uniformity parameters; When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

[0047] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for visually inspecting the appearance of a product printed by a 3D printer, characterized in that: The following steps are involved: S01, collecting multi-angle surface image data of printed products through a multi-spectral visual sensor array arranged in a ring, and synchronously acquiring three-dimensional point cloud data generated by a laser scanner; S02, performing three-dimensional reconstruction on the surface image data to generate a first appearance model, and converting the three-dimensional point cloud data into a second appearance model; S03, fusing the first appearance model and the second appearance model through a feature registration algorithm to generate a fused three-dimensional model, and using an adaptive weight allocation strategy to process the overlapping area of ​​the models, and calculating the confidence weight of each pixel in the overlapping area to eliminate fusion errors; S04, extracting the geometric deviation feature vector of the fused three-dimensional model, and generating a process feature vector in combination with the interlayer temperature field data collected in real time during the printing process; S05, inputting the geometric deviation feature vector and the process feature vector into a pre-trained multimodal defect detection model, and outputting a deformation index, an interlayer dislocation probability, and a material distribution uniformity parameter; S06. When any parameter exceeds a preset threshold, it is determined that a three-dimensional defect exists and an early warning is issued.

2. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, characterized in that: The multispectral vision sensor comprises: At least three groups of industrial cameras are arranged in a 120° ring above the printing platform, each group of industrial cameras includes at least one visible light camera and one near infrared camera; A laser line scanner with the collection end tilted 45° downward is installed at the four corners of the printing platform; In the step S01, the surface image data and the three-dimensional point cloud data are collected so as to complete a complete cycle of data collection within 2 seconds after each layer of the printed product is printed.

3. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, characterized in that: The fusion of the first appearance model and the second appearance model by the feature registration algorithm in step S03 includes: S31, using an improved ICP algorithm to roughly align the first appearance model with the second appearance model to establish an initial coordinate correspondence relationship; S32, constructing a feature pyramid of overlapping regions and calculating the matching degree of SIFT feature points at multiple scales; S33. Dynamically adjust the registration weight based on the spatial distribution density of feature points: ； Among them, D i is the point cloud density, C i is the image contrast, , is the adaptive adjustment coefficient, both are 3, D max is the maximum point cloud density, C max To maximize the image contrast, the low confidence area is then locally rescanned to fill in the gaps.

4. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, characterized in that: The step S03 also includes coordinating the printing layer of the next layer of the printed product, and the steps include: S34, extracting the geometric deviation features of the fused three-dimensional model, and performing correlation analysis with the interlayer cooling rate data collected in real time; S35, based on the result of the association analysis, input the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level; S36. Based on the obtained output defect type and severity level, a parameter compensation sequence is generated for the corresponding interlayer dislocation defect based on historical process data: Print speed correction for the next layer: ; Nozzle temperature adjustment curve: ; Among them, T(t) is the nozzle temperature adjustment curve, Δv is the printing speed, δ is the misalignment, k, τ, ω are material-related correction coefficients, t is time, ΔT is the temperature fluctuation amplitude, ω is the angular frequency, e is 4, and T0 is the standard temperature of the wire.

5. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 4, characterized in that: The step S35 of inputting the pre-trained spatiotemporal convolutional network to obtain the output defect type and severity level includes: S351, 3D dilated convolution is used to extract 3D geometric features, and the dilation rate increases from 2 to 6 with the network depth; S352, using a bidirectional GRU network to analyze the process parameters of each printing layer of the printed product, and making a comparison table of the process parameters of each printing layer, calculating the average value and then subtracting the process parameters of each printing layer, and marking the difference; S353, through the gated attention mechanism, the spatiotemporal features are dynamically integrated, and the gated value calculation formula is: ； Among them, F s is the spatial feature, F t is the time feature, W g ,b g is a learnable parameter and σ is 3.

6. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, characterized in that: The training method of the multimodal defect detection model in step S05 includes: S51. Construct a sample library containing three typical defects: surface cracks, interlayer peeling, and thermal deformation. Each sample includes multi-angle visible light, infrared images, high-precision three-dimensional point cloud data, interlayer temperature curve data, and material extrusion pressure time series data; S52, based on the dual-channel deep network, the first channel processes the image and point cloud data, and the second channel processes the process parameters; S53. Feature interaction is achieved through the cross-channel attention mechanism, and the final output layer contains the three-dimensional deformation heat map and defect classification results.

7. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 6, characterized in that: The dual-channel deep network in step S52 includes: S521, using 3D convolution kernel to extract spatial features in the image point cloud channel, each convolution layer is followed by a SE attention module; S522, using LSTM network to capture time series features in the process parameter channel; S523. Implement hierarchical fusion of dual-channel features through a dynamic routing algorithm, wherein the hierarchical fusion of dual-channel features includes primary fusion for processing local detail features and advanced fusion for processing global semantic features.

8. A method for visually inspecting the appearance of a product printed by a 3D printer according to claim 1, characterized in that: The execution basis of step S06 includes: When 0.3<deformation index≤0.5, an early warning is triggered and the coordinates are recorded; When 0.5<deformation index≤0.7, reduce printing speed and enhance local cooling; When the deformation index is > 0.7, printing is paused and compensation processing is started; The probability of inter-layer misalignment is detected by sliding window, and it is judged as a structural defect when the probability of misalignment of three consecutive layers is greater than 40%.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the method for visually inspecting the appearance of products printed by a 3D printer as described in any one of claims 1 to 8.

10. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the method for visually inspecting the appearance of products printed by a 3D printer as described in any one of claims 1 to 8.

Images