Methods and apparatus for inspecting the print quality of 3D printers; 3D printers

Abstract

A method for detecting print quality of a 3D printer is provided. The method comprises: obtaining a model reference image; generating a scanning path; causing a print head to print a first layer of a 3D model on a hot bed; causing a color camera to move along the scanning path and take pictures at the plurality of different positions to generate a global image corresponding to the model reference image; determining a print quality result based on the model reference image and the global image.

Description

Technical Field

[0001] This disclosure relates to the field of 3D printing technology, and more specifically to methods for detecting the print quality of a 3D printer, apparatus for detecting the print quality of a 3D printer, a 3D printer, a computer-readable storage medium, and a computer program product. Background Technology

[0002] 3D printing technology, also known as additive manufacturing, is a technology that uses digital model files as a basis and employs adhesive materials to construct objects layer by layer. 3D printing is typically achieved using 3D printers. 3D printers, also called three-dimensional printers or stereoprinters, are a type of rapid prototyping equipment. 3D printers are commonly used in mold making, industrial design, and other fields to manufacture models or parts. A typical 3D printing technology is fused deposition modeling (FDM), which builds objects by selectively depositing molten material layer by layer along a predetermined path. The material used is thermoplastic polymer in filament form. Currently, there is still significant room for improvement in the print quality of 3D printers.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] This disclosure provides a method for detecting the print quality of a 3D printer, an apparatus for detecting the print quality of a 3D printer, a 3D printer, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, a method for detecting the print quality of a 3D printer is provided, wherein the 3D printer includes: a heated bed; a print head movable relative to the heated bed; a color camera disposed on the print head for capturing images of a portion of the heated bed; and at least one processor for controlling the print head to move relative to the heated bed based on control code generated by slicing software to print a 3D model layer by layer, the method comprising: acquiring a model reference map, wherein the model reference map is generated by parsing control information generated by the slicing software, the model reference map representing an area occupied by at least a portion of the first layer of the 3D model on the heated bed; generating a scan path based on the model reference map, wherein the scan path is generated such that when the color camera moves along the scan path as the print head moves relative to the heated bed, the color camera sequentially captures images of a plurality of different locations of the occupied area; causing the color camera to move along the scan path under the load of the print head and during the movement... Multiple first partial images are acquired by taking pictures at multiple different locations, each first partial image indicating a corresponding image of the heated bed at the multiple different locations; the print head prints the first layer of the 3D model on the heated bed; the color camera moves along the scanning path under the support of the print head and takes pictures at multiple different locations during the movement to acquire multiple second partial images, each second partial image indicating a corresponding image of the heated bed at the multiple different locations after the first layer of the 3D model has been printed on the heated bed; multiple local result images are determined based on the corresponding difference between each second partial image and the corresponding first partial image in the multiple first partial images; the multiple local result images are stitched together according to the scanning path to generate a global image corresponding to the model reference image; and a print quality result is determined based on the model reference image and the global image, the print quality result indicating the print quality of at least a portion of the first layer of the 3D model.

[0006] According to another aspect of this disclosure, an apparatus for detecting the print quality of a 3D printer is provided, wherein the 3D printer includes: a heated bed; a print head movable relative to the heated bed; a color camera disposed on the print head for capturing images of a portion of the heated bed; and at least one processor for controlling the print head to move relative to the heated bed based on control code generated by slicing software to print a 3D model layer by layer, the apparatus comprising: a first module for acquiring a model reference image, wherein the model reference image is generated by parsing control information generated by the slicing software, the model reference image representing an area occupied by at least a portion of the first layer of the 3D model on the heated bed; a second module for generating a scan path based on the model reference image, wherein the scan path is generated such that when the color camera moves along the scan path as the print head moves relative to the heated bed, the color camera sequentially captures images of a plurality of different locations of the occupied area; and a third module for causing the color camera to move along the scan path under the load of the print head and, during the movement, capture images of the plurality of different locations of the 3D model. The system comprises: a first module for capturing images at a location to obtain multiple first partial images, each indicating a corresponding image of the heated bed at a multiple different location; a fourth module for causing the print head to print the first layer of the 3D model on the heated bed; a fifth module for causing the color camera, carried by the print head, to move along the scanning path and capture images at multiple different locations during the movement to obtain multiple second partial images, each indicating a corresponding image of the heated bed at a multiple different location after the first layer of the 3D model has been printed on the heated bed; a sixth module for determining multiple local result images based on the corresponding difference between each second partial image and the corresponding first partial image in the multiple first partial images; a seventh module for stitching the multiple local result images according to the scanning path to generate a global image corresponding to the model reference image; and an eighth module for determining a printing quality result based on the model reference image and the global image, the printing quality result indicating the printing quality of at least a portion of the first layer of the 3D model.

[0007] According to another aspect of this disclosure, a 3D printer is provided, comprising: a heated bed; a print head movable relative to the heated bed; a depth sensor disposed on the print head for measuring a distance of a portion of the heated bed relative to the depth sensor; and at least one processor configured to generate a local depth map of the portion of the heated bed based on the measurement results of the depth sensor, and to control the print head to move relative to the heated bed based on control code generated by slicing software to print a 3D model layer by layer, wherein the at least one processor is further configured to execute instructions to implement the method described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing instructions, wherein the instructions, when executed by the at least one processor of the 3D printer described above, implement the method described above.

[0009] According to another aspect of this disclosure, a computer program product is provided, including instructions that, when executed by the at least one processor of the 3D printer described above, implement the method described above.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0012] Figure 1 A schematic diagram of a 3D printer according to an example embodiment is shown;

[0013] Figure 2 A flowchart of a method for detecting the print quality of a 3D printer according to an example embodiment is shown;

[0014] Figure 3 An example graphical representation of the first layer of a 3D model on a heated bed is shown;

[0015] Figure 4 It shows the relationship with Figure 3 The model reference diagram corresponding to the example;

[0016] Figure 5 Showing the target Figure 4 Example of a scan path for a model reference graph;

[0017] Figure 6 A flowchart illustrating the steps of determining multiple local result images according to an example embodiment is shown;

[0018] Figure 7 A flowchart illustrating the steps for determining print quality results according to an example embodiment is shown;

[0019] Figure 8 A flowchart of another method for detecting the print quality of a 3D printer, according to an example embodiment, is shown;

[0020] Figure 9a An example is shown of a global image stitched together from multiple local result images without illumination compensation;

[0021] Figure 9b It shows Figure 9a An example of a global image after illumination compensation;

[0022] Figure 10 A structural block diagram of an apparatus for detecting the print quality of a 3D printer according to an example embodiment is shown. Detailed Implementation

[0023] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present disclosure, including various details of these embodiments to aid understanding; however, these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0024] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0025] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof. The term "based on" should be interpreted as "at least partially based on".

[0026] 3D printing technology constructs objects by printing layer by layer. In 3D printing, the quality of the first layer is crucial to the success of the print run. Poor quality in the first layer will severely impact the quality of the final 3D model. Therefore, to ensure successful printing, the quality of the first layer needs to be checked. If the first layer's quality does not meet requirements, printing can be stopped immediately to avoid final print failure. Currently, 3D printers lack the function to check the quality of the first layer, making it impossible to know its condition in real time.

[0027] The inventors recognized that the printing quality of the first layer could be detected by using an optical camera to capture images and then processing those images. Moreover, compared to other quality inspection techniques (such as using depth detection techniques to detect the presence of printing voids), using an optical camera and image processing requires less equipment; ordinary consumer color cameras are sufficient, and therefore it can be easily integrated into various types of 3D printing equipment.

[0028] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 A schematic diagram of a 3D printer 100 according to an embodiment of the present disclosure is shown. (As...) Figure 1 As shown, the 3D printer 100 includes a heated bed 110, a print head 120 movable relative to the heated bed 110, and a color camera 130 arranged on the print head 120 for capturing images of a portion of the heated bed 110. Here, the phrase "print head movable relative to the heated bed" can refer to any of the following: (1) the heated bed remains stationary while the print head moves; (2) the heated bed moves while the print head remains stationary; (3) both the heated bed and the print head move. Examples of the color camera 130 include, but are limited to, common consumer 2D color cameras. In this document, for ease of description, embodiments of the present disclosure are illustrated by example of such a common consumer 2D color camera, but the present disclosure is not limited in this respect.

[0030] The 3D printer 100 also includes at least one processor (not shown). The at least one processor controls the movement of the print head 120 relative to the heated bed 110 based on control code generated by slicing software to print the 3D model layer by layer. Figure 1As shown, at least one processor can drive a motor (not shown), which in turn drives an extrusion wheel 150 to feed printing material 170 from a feed pan 160 into a print head 120. During the movement of the print head 120, printing material is extruded from the print head 120 and deposited onto a heated bed 110. Typically, slicing software runs on a computing device communicatively connected to the 3D printer 100 and operates to generate control information for controlling the printing process. For example, the slicing software can provide a graphical user interface (GUI) to allow users to select or adjust layout information representing the position and orientation of the 3D model on the heated bed 110. The slicing software can slice the 3D graphical representation of the 3D model to generate slice data (e.g., number of slices, height of each slice layer, etc.), and then convert the slice data into control code for controlling the print head 120 of the 3D printer 100 to move along the printing path to print the individual slice layers. Such control code is typically in the form of gcode. The control code is downloaded to the 3D printer 100 for execution by at least one processor. For this purpose, the 3D printer 100 may also include at least one memory (not shown) for storing programs and / or data.

[0031] At least one processor is also used to implement the various functions described below. In the example, the processor includes a microcontroller or computer that executes instructions stored in firmware and / or software (not shown). The processor may be programmable to perform the functions described herein. As used herein, the term computer is not limited to these integrated circuits referred to in the art as computers, but broadly refers to computers, processors, microcontrollers, microcomputers, programmable logic controllers, application-specific integrated circuits, and other programmable circuits, and these terms are used interchangeably herein. The computers and / or processors discussed herein may each take the form of computer-readable media or machine-readable media, referring to any medium that participates in providing instructions to the processor for execution. The memory discussed above constitutes a computer-readable medium. Such media may take many forms, including but not limited to non-volatile media, volatile media, and transmission media.

[0032] It will be understood that exemplary embodiments of the present disclosure are described below in conjunction with an FDM printer, but the present disclosure is not limited to an FDM printer. In embodiments, the printhead 120 may be configured to extrude any material suitable for 3D printing, including, for example, thermoplastics, alloys, metal powders, ceramic materials, ceramic powders, polymers, etc.

[0033] Figure 2 This is a flowchart illustrating a method 200 for detecting the print quality of a 3D printer according to an example embodiment. For discussion purposes, the following is combined with... Figure 1The method 200 is described using the 3D printer 100 shown. In the example, method 200 can be implemented by at least one processor in the 3D printer 100.

[0034] In step 210, a model reference diagram is obtained. The model reference diagram represents the area occupied by at least a portion of the first layer of the 3D model on the heated bed 110.

[0035] The following is combined Figure 3 and Figure 4 To illustrate the model reference diagram. Figure 3 An example graphical representation of the first layer of the 3D model on the heated bed 310 is shown, and Figure 4 It shows the relationship with Figure 3 The example corresponds to the model reference diagram. In this example, the first layer of the 3D model occupies an area on the heated bed 310 comprising four discrete regions 340a, 340b, 340c, and 340d. These regions can be formed from the same printing material or different printing materials. Although discrete regions 340a, 340b, 340c, and 340d are... Figure 3 The area shown is depicted as having a rectangular shape, but this is merely illustrative; in other examples, the area occupied by the first layer of a 3D model may have other shapes or configurations (e.g., a single connected region), and this disclosure is not limited in this respect. Figure 3 The corresponding graphic representation, Figure 5 The model reference figure 410 shown includes four pixel regions 440a, 440b, 440c, and 440d. Generally, the model reference figure 410 can be in the thermocline coordinate system oxyz( Figure 3 The coordinates of pixel regions 440a, 440b, 440c, and 440d in model reference figure 410 correspond one-to-one with the coordinates of discrete regions 340a, 340b, 340c, and 340d on the heated bed 310. It will be understood that although model reference figure 410 is generated under [a specific condition], the coordinates of pixel regions 440a, 440b, 440c, and 440d in model reference figure 410 correspond one-to-one with the coordinates of discrete regions 340a, 340b, 340c, and 340d on the heated bed 310. Figure 4 The middle is shown as having the same Figure 3 The dimensions of the heated bed 310 correspond to the dimensions of the heated bed, but this is not mandatory. In other examples, the model reference figure 410 may only have dimensions corresponding to the occupied area ( Figure 3 In the process, the dimensions of the bounding boxes of the discrete regions 340a, 340b, 340c and 340d as a whole are corresponding to the dimensions of the bounding boxes, thereby saving storage space.

[0036] It will also be understood that the model reference diagram does not necessarily need to represent the entire first layer of the 3D model, but only a portion of it. This is because in some cases, it may only be necessary to inspect the print quality of a portion of the first layer of the 3D model. For example, since the temperature distribution of the heated bed may be uneven, with some areas being hotter and others colder, the printing material may not form properly in the colder areas, resulting in printing defects. In such cases, print quality can be inspected only in the colder areas, thereby improving inspection efficiency.

[0037] A model reference map can be generated by parsing control information generated by slicing software. In some embodiments, the control information generated by the slicing software includes control code (e.g., gcode) for printing the first layer of the 3D model. In such an embodiment, obtaining the model reference map (step 210) may include receiving the model reference map from a computing device communicatively connected to the 3D printer 100. The model reference map is generated by the slicing software running on the computing device by parsing the control code for printing the first layer of the 3D model. Alternatively, obtaining the model reference map (step 210) may include reading the model reference map locally from the 3D printer 100. The model reference map is generated by at least one processor by parsing the control code for printing the first layer of the 3D model, or extracted directly from slices of the model. Since the control code specifies the movement path of the print head, the occupied area of ​​the first layer of the 3D model on the heated bed can be recovered from it.

[0038] In some embodiments, the control information generated by the slicing software includes layout information representing the position and orientation of the 3D model on the heated bed 110. In such an embodiment, obtaining a model reference image (step 210) may include receiving the model reference image from a computing device communicatively connected to the 3D printer 100. The model reference image is generated by the slicing software running on the computing device by parsing the layout information. Since the layout information defines the position and orientation of the 3D model on the heated bed, the area occupied by the first layer of the 3D model on the heated bed can be recovered from it.

[0039] Return to reference Figure 2 In step 220, a scan path is generated based on the model reference diagram. The scan path is generated such that as the color camera 130 moves along the scan path as the printhead 120 moves relative to the heated bed 110, the color camera 130 sequentially captures images of multiple different locations within the occupied area.

[0040] Figure 5 Showing the target Figure 4An example of a scan path for a model reference map. In some embodiments, the occupied area includes at least one discrete region spaced apart from each other, the model reference map includes at least one pixel region representing each of the at least one discrete region, and generating the scan path (step 220) may include the following operations:

[0041] (1a) Determine the bounding box of each of the at least one pixel region to obtain at least one bounding box corresponding to each of the at least one pixel region. Figure 5 In the example, the bounding boxes of pixel regions 440a, 440b, 440c and 440d can be determined, thus obtaining four bounding boxes.

[0042] (1b) In the model reference diagram, a scan path is determined, along which a virtual bounding box representing the field of view (FOV) of the color camera moves to cover part of the at least one bounding box as a whole each time, and ultimately traverses the entire area of ​​the at least one bounding box. Figure 5 In the example, a virtual box representing the FOV of the color camera 130 is shown, and the determined scan path is indicated by a hollow arrow. In this example, the scan path is a Zig-Zag path, but this is illustrative and not limiting.

[0043] It will be understood that generating bounding boxes is not mandatory. In some embodiments, the scan path can be generated based on the original shape of the pixel region representing the occupied area of ​​the first layer of the 3D model on the heated bed in the model reference diagram. In other embodiments, any other suitable method can be used to generate the scan path, as long as the depth sensor can measure multiple target locations of the occupied area of ​​the first layer of the 3D model on the heated bed.

[0044] In some embodiments, the occupied area includes at least one discrete region spaced apart from each other, the model reference map includes at least one pixel region representing the at least one discrete region, and generating the scan path (step 220) may include the following operations:

[0045] (2a) Determine the connected components of each of the at least one pixel region to obtain at least one connected component corresponding to each of the at least one pixel region. Figure 5 In the example, the connected components of pixel regions 440a, 440b, 440c and 440d can be determined, thus obtaining four connected components.

[0046] (2b) For each connected component, a movement path is determined in the model reference graph, along which the virtual bounding box representing the field of view of the color camera moves to cover a portion of the connected component each time, and eventually traverses the entire region of the connected component. This can be similar to operation (1b) described above, and will not be repeated here.

[0047] (2c) The movement paths for all connected components are merged into a single merged path as the scan path. By generating separate scan paths for each discrete region of the occupied area and merging these separate scan paths into the final scan path, the number of scans for non-target regions (e.g., Figure 5 Scanning the blank areas in the image improves detection efficiency.

[0048] It will be understood that, in the embodiments, the scanning path is generated for the field of view of the color camera 130, and the scanning path of the color camera 130 may not necessarily coincide with the movement path of the printhead 120, because there may be rotation and / or translation between the orientation of the color camera 130 and the orientation of the printhead 120. Extrinsic parameter calibration can be used to pre-calibrate the rotation and / or translation of the printhead 120 and the color camera 130 in a three-dimensional coordinate system (e.g., a heated bed coordinate system), and the scanning path for the color camera 130 can be converted into a movement path for the printhead 120. Corresponding control code can then be generated to control the movement of the printhead 120, causing the color camera 130 to move along the scanning path under the load of the printhead 120. Extrinsic parameter calibration is a known technique and will not be described in detail here to avoid obscuring the subject matter of this disclosure.

[0049] Return to reference Figure 2 In step 230, the color camera 130 is moved along the scanning path under the support of the printhead 120, and the color camera 130 takes pictures at multiple different locations during the movement to obtain multiple first partial images. The multiple first partial images respectively indicate corresponding images of the heated bed 110 at multiple different locations.

[0050] In some implementations, each of the multiple first partial images is numbered and stored in memory according to the physical coordinates of the printhead 120 in the heated bed coordinate system when the color camera 130 captures each first partial image along the scanning path. The purpose of numbering is to ensure that each first partial image corresponds to one of the multiple different positions on the heated bed 110. It will be understood that the multiple first partial images can be stored in the camera coordinate system or converted to the image coordinate system for storage. The following explanation uses storage in the camera coordinate system as an example.

[0051] In step 240, the print head 120 prints the first layer of the 3D model on the heated bed 110.

[0052] In step 250, the color camera 130, carried by the print head 120, moves along the scanning path and captures images at the aforementioned multiple different locations during this movement to obtain multiple second partial images. These multiple second partial images respectively indicate corresponding images of the heated bed 110 at multiple different locations after the first layer of the 3D model has been printed onto the heated bed 110. This step causes the color camera 130 to capture images again at the same multiple different locations along the same scanning path as in step 230.

[0053] Similarly, in some implementations, each of the multiple second partial images can be numbered according to the physical coordinates of the printhead 120 in the heated bed coordinate system when the color camera 130 captures each optical image along the scanning path, and stored in memory in the camera coordinate system. This allows each second partial image to correspond to the multiple different positions on the heated bed 110, and therefore also to the first partial images stored in step 230.

[0054] In step 260, multiple local result images are determined based on the corresponding differences between each of the multiple second local images and the corresponding first local images in the multiple first local images. These multiple local result images represent corresponding images at multiple different locations after subtracting the hotbed background. See further... Figure 6 Determining multiple local result images (step 260) may include the following operations:

[0055] In step 610, multiple difference images are obtained by calculating the difference between each of the multiple second local images and the corresponding first local image in the multiple first local images. For example, if the second local image is represented by a pixel matrix I and the corresponding first local image (hotbed image) is represented by a pixel matrix J, then the pixel matrix D of the difference image is D = abs(IJ).

[0056] In step 620, for each difference image, a threshold for determining valid pixels is determined, where each valid pixel indicates that the pixel belongs to the occupied region. In some embodiments, the average variance of each pixel in a first local image (hotbed image) corresponding to the difference image is calculated, and the threshold is determined based on the average variance. For example, the average variance of each pixel is calculated as mean_std. The threshold for determining valid pixels is defined as diff_thresh = clip(2*mean_std, 30, 80), that is, the threshold diff_thresh is equal to twice mean_std, but does not exceed the range [30, 80].

[0057] In step 630, a masking operation is performed on each pixel in each difference image. The masking operation includes: comparing the pixel value of each pixel with a determined threshold; in response to the pixel value being greater than the threshold, determining that the pixel is a valid pixel and replacing the pixel value of the pixel with the pixel value of the corresponding pixel in the second local image; otherwise, retaining the pixel value.

[0058] In step 640, the multiple difference images after the masking operation are used as multiple local result images to obtain multiple local result images.

[0059] Return to reference Figure 2 In step 270, multiple local result images are stitched together according to the scanning path to generate a global image corresponding to the model reference image. In some embodiments, the stitching operation includes: in response to determining that the multiple local result images overlap each other, averaging or linearly interpolating the pixel values ​​of pixels at the overlapping positions between the multiple local result images.

[0060] In step 280, based on the model reference image and the global image, a print quality result is determined, indicating the print quality of at least a portion of the first layer of the 3D model. It should be understood that the model reference image indicates target print images at multiple different locations, and the global image indicates actual print images at multiple different locations. Therefore, the presence or severity of print defects can be detected based on the error between the target print image and the actual print image.

[0061] See further Figure 7 Determining the print quality result (step 280) may include the following operations:

[0062] In step 710, the target printed images at multiple different locations are compared with the actual printed images at the corresponding locations in multiple different locations.

[0063] In step 720, the print quality result is determined based on the comparison operation.

[0064] It will also be understood that the error between the target value and the actual value can be measured in various possible ways, thereby determining the print quality result. Some illustrative implementations are provided below and should not be considered restrictive.

[0065] In some embodiments, the comparison step 710 described above may include:

[0066] (3a) Determine the normal printing threshold for the first layer of the 3D model. The upper and lower bounds of the normal printing threshold are related to the printing height and printing material set for the first layer.

[0067] (3b) Based on the determined normal printing threshold, the global image and model reference are... Figure 2 Binarization. In some embodiments, the Lab algorithm is used to convert the global image and the model reference image to the Lab color space, and the luminance component is extracted and binarized to obtain a binarized image B of the global image and a binarized image R of the model reference image. The binarization threshold can be the threshold diff_thresh / 2 used above for background subtraction. Binarization is a known technique, and will not be described in detail here to avoid obscuring the subject matter of this disclosure.

[0068] (3c) By comparing the pixel values ​​of corresponding pixels in the binarized global image and the model reference image, erroneous pixels in the global image are identified. Erroneous pixels indicate that the first layer of the 3D model was not printed correctly. Continuing the previous example (obtaining the binarized image B of the global image and the binarized image R of the model reference image), the binarized image R of the model reference image is traversed. If R(x,y)>0&B(x,y)=0 at a certain position, it indicates that there is a hole and is marked as an erroneous pixel. After traversing, the error map errmap is obtained.

[0069] (3d) Identify at least one pixel region in the model reference graph that represents the occupied area. Each pixel region is typically in the form of a connected component, where each pixel represents a corresponding location on the heated bed that is occupied by the first layer of the 3D model. It should be understood that different pixel regions representing the occupied area can have different areas. For detection efficiency purposes, detection can be performed only on a subset of larger pixel regions (e.g., larger than a threshold T1) instead of all pixel regions.

[0070] (3e) For at least one of at least one pixel region:

[0071] (3e-1) Count the number of erroneous pixels in each pixel corresponding to the pixel region in the global image. For example, for each connected region C with an area greater than the threshold T1, count the number of pixels n in the global image within that connected region C.

[0072] (3e-2) Compare the number of erroneous pixels with the corresponding threshold, and / or compare the ratio of the number of erroneous pixels to the number of pixels in the global image corresponding to the pixel region with the corresponding threshold.

[0073] As mentioned earlier, the error between the target and actual values ​​of the first-layer printed image can be measured in various ways. Here, the number of normal pixels, the number of error pixels, and the relative number of normal and error pixels are all metrics reflecting the error between the target and actual values ​​of the first-layer printed image. In one example, the error level l can be defined as follows:

[0074]

[0075] Here, T2, T3, and T4 are all thresholds, and n is the number of pixels in the global image within the connected component C. These thresholds can be pre-defined or adaptive. For example, as a function of the number of pixels in connected component C, these thresholds can adaptively change with different numbers of pixels in connected component C. In this example, the error level l can take the value 0, 1, or 2. It will be understood that such error levels are illustrative rather than restrictive.

[0076] Based on the comparison results in step 710, the print quality result can be determined in step 720. Continuing with the example above regarding error level l, the following decision logic can be defined:

[0077] If any connected component C has an error level of 2, the print quality result is determined to be "error", and the final error level is 2.

[0078] Otherwise, if there are two or more connected components with an error level of 1, and the total number of error pixels is greater than the threshold T4 or the proportion of error pixels is greater than the threshold T5, the print quality result is also determined to be "error", and the final error level 2 is output.

[0079] Otherwise, if there are more than one connected component with an error level of 1, the print quality result is determined to be "warning", and the final error level is 1.

[0080] Otherwise, the print quality result is judged as "normal", with a final error level of 0.

[0081] It will be understood that such judgment logic is illustrative rather than limiting. Other judgment logic may be applied in other embodiments. For example, the print quality result may be judged based on the cumulative absolute value of the error between the target value and the actual value of the first-layer printed image. It should be understood that if the error between the target value and the actual value of the first-layer printed image is known, various possible judgment criteria can be designed to detect print quality. This disclosure cannot exhaust all judgment criteria, but this does not affect the fact that these other judgment criteria also fall within the scope of this disclosure.

[0082] In some embodiments, the print quality result includes a confidence level indicating the reliability of the detection, which is a function of the total number of valid pixels and the total number of pixels in the model reference image. For example, the confidence level is the ratio of the total number of valid pixels to the total number of pixels in the model reference image. In other embodiments, the confidence level is another suitable function of the total number of valid pixels and the total number of pixels in the model reference image.

[0083] Due to various systematic errors, the actual printed first-layer pattern may not be perfectly aligned with the first-layer pattern of the model reference image. Therefore, it may be necessary to register the model reference image and the global image to find the error at the optimal matching position. In some embodiments, before operation (3e) (i.e., for at least one of at least one pixel region, counting the number of erroneous pixels in each pixel in the global image corresponding to that pixel region), the global image may be registered with the model reference image so that the global image and the model reference image are aligned according to the registration criteria.

[0084] In the embodiments, various registration methods can be used, such as:

[0085] (1) Gray-scale based template matching algorithm: Find a sub-image similar to the template image in another image based on the known template image. For example, binarize the global image and the model reference image, and perform template matching on the binarized global image and the model reference image.

[0086] (2) Feature-based matching algorithm: First, extract the features of the image, then generate feature descriptors, and finally match the features between the two images based on the similarity of the descriptors. Image features can include points, lines (edges), regions (areas), etc., and can also be divided into local features and global features.

[0087] (3) Relationship-based matching algorithm: using machine learning algorithms to match images.

[0088] In one example, a brute-force search can be used to find the optimal matching position between the global image and the model reference image, and then the error is calculated at the optimal matching position. Specifically, the global image is moved in both the x and y directions, and the error is calculated at the new position. If the number of error pixels at the new position is less than the number of error pixels at the previously recorded optimal position, then the new position is updated to the optimal matching position. To reduce computational cost, the search range can be limited to a window (e.g., 20 pixels, corresponding to 2mm in physical coordinates). Furthermore, if the number of error pixels at the new position is significantly greater than the number of error pixels at the previous position (e.g., 20%), the search in the current direction is stopped.

[0089] As previously mentioned, the error between the target value and the actual value can be measured in various possible ways, thereby determining the print quality result. In some embodiments, determining the print quality result may include inputting a model reference image, a print image set by the slicing software, and a global image into a trained machine learning algorithm (e.g., a classification neural network) to obtain the print quality result output by the trained machine learning algorithm. As previously mentioned, the model reference image and the print image set by the slicing software indicate the target print image at multiple different locations on the heated bed, while the global image indicates the actual print image at said multiple different locations. The machine learning algorithm can be applied to scenarios such as determining the error between the target value and the actual value of the print image. With a large number of training samples, the machine learning algorithm can be trained to detect the presence of print defects.

[0090] Figure 8 This is a flowchart illustrating another method 800 for detecting the print quality of a 3D printer according to an example embodiment. The steps of method 800 (steps 810, 820, 830, 840, 850, 860, 88, 890) are... Figure 2 The corresponding steps of the illustrated method are the same, the difference being that after subtracting the heated bed background (step 860), method 800 further includes step 870: determining a brightness field indicating the uniformity of the illumination captured by the color camera, and performing illumination compensation on multiple local result images based on the brightness field. The reason for adding the illumination compensation step is that because the supplementary light is on one side of the camera, the illumination of the image is non-uniform, requiring illumination compensation; otherwise, the stitched images will have obvious seams, affecting the detection effect. Figure 9a and Figure 9b The images show a global image 900, which is a stitched image of multiple local result images without illumination compensation, and a global image 910, which is a stitched image of multiple local result images with illumination compensation.

[0091] In some embodiments, determining a luminance field that indicates the uniformity of illumination during shooting includes determining a luminance field L1 based on predetermined calibration information. The calibration information indicates the default uniformity of illumination in the shooting field of view of the color camera.

[0092] In other embodiments, determining the brightness field indicating the uniformity of the captured illumination includes: determining an average image of a plurality of first local images. The pixel value of each pixel in the average image corresponds to the average of the pixel values ​​of corresponding pixels in the plurality of first local images, and the brightness field L2 is determined based on the average image.

[0093] In other embodiments, the total luminance field can be determined based on the average of the luminance fields L1 and L2 obtained above. For example, the total luminance field L = (L1 + L2) / 2.

[0094] It should be understood that the method for determining the brightness field can be determined based on the specific circumstances and application scenarios.

[0095] After determining the luminance field, illumination compensation is performed on multiple local result images based on this luminance field. In some embodiments, dividing each pixel of the multiple local result images by the luminance field L can essentially eliminate the effect of luminance non-uniformity. It is worth noting that either L1 or L2 can be used directly with acceptable results.

[0096] Figure 10 A structural block diagram of an apparatus 1000 for detecting the print quality of a 3D printer according to an example embodiment is shown. The apparatus 1000 includes a first module 1100, a second module 1200, a third module 1300, a fourth module 1400, a fifth module 1500, a sixth module 1600, a seventh module 1700, and an eighth module 1800. For discussion purposes, the following is in conjunction with… Figure 1 The 3D printer 100 describes the device 1000.

[0097] The first module 1100 is used to obtain a model reference image. The model reference image is generated by parsing the control information generated by the slicing software, and the model reference image represents the occupied area of ​​at least a portion of the first layer of the 3D model on the heated bed 110.

[0098] The second module 1200 is used to generate a scan path based on a model reference map, wherein the scan path is generated such that when the color camera 130 moves along the scan path as the print head 120 moves relative to the heated bed 110, the color camera 130 sequentially captures images of multiple different locations in the occupied area.

[0099] The third module 1300 is used to move the color camera 130 along the scanning path under the support of the printhead 120 and to capture images at the plurality of different locations during the movement to obtain a plurality of first partial images, the plurality of first partial images respectively indicating corresponding images of the heated bed 110 at the plurality of different locations.

[0100] The fourth module 1400 is used to enable the print head 120 to print the first layer of the 3D model on the heated bed 110.

[0101] The fifth module 1500 is used to move the color camera 130 along the scanning path under the support of the print head 120 and to take pictures at the plurality of different locations during the movement to obtain a plurality of second partial images, the plurality of second partial images respectively indicating the corresponding images of the heated bed at the plurality of different locations after the first layer of the 3D model is printed on the heated bed 110.

[0102] The sixth module 1600 is used to determine multiple local result images based on the corresponding difference between each of the multiple second local images and the corresponding first local image in the multiple first local images.

[0103] The seventh module 1700 is used to stitch together the multiple local result images according to the scanning path to generate a global image corresponding to the model reference image.

[0104] The eighth module 1800 is used for the model reference image and the global image to determine the print quality result, which indicates the print quality of at least a portion of the first layer of the 3D model.

[0105] It should be understood that Figure 10 The various modules of the device 1000 shown can be connected to the reference. Figure 2 Method 200 described and Figure 8 The steps in method 800 are described in the same way. Therefore, the operations, features, and advantages described above for methods 200 and 800 also apply to apparatus 1000 and its included modules. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0106] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0107] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 10The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, in some embodiments, one or more of these modules can be implemented together in a system-on-a-chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0108] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium storing instructions for causing the 3D printer 100 as described above to perform the methods as described in any embodiment of the present disclosure is also provided.

[0109] According to embodiments of the present disclosure, a computer program product is also provided, including instructions for causing the 3D printer 100 as described above to perform the methods as described in any embodiment of the present disclosure.

[0110] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0111] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of this disclosure is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for detecting print quality of a 3D printer, wherein, The 3D printer comprises a hot bed, a print head movable relative to the hot bed, a color camera arranged on the print head for capturing images of a portion of the hot bed, and at least one processor for controlling the print head to move relative to the hot bed based on control codes generated by slicing software to print a 3D model layer by layer, and the method comprises: obtaining a model reference map, wherein the model reference map is generated by analyzing control information generated by the slicing software, and the model reference map represents an occupied area of at least a portion of a first layer of the 3D model on the hot bed; generating a scanning path based on the model reference map, wherein the scanning path is generated such that when the color camera moves along the scanning path as the print head moves relative to the hot bed, the color camera sequentially captures images of a plurality of different positions of the occupied area; causing the print head to print the first layer of the 3D model on the hot bed; causing the color camera to move along the scanning path under the bearing of the print head and capture images at the plurality of different positions during the movement to obtain a plurality of second partial images, each of the plurality of second partial images indicating a corresponding image of the hot bed at the plurality of different positions after the first layer of the 3D model is printed on the hot bed, to generate a global image corresponding to the model reference map; and based on the model reference map and the global image, determining a print quality result indicating the print quality of the at least a portion of the first layer of the 3D model.

2. The method of claim 1, wherein, The method further comprises: causing the color camera to move along the scanning path under the bearing of the print head and capture images at the plurality of different positions to obtain a plurality of first partial images, each of the plurality of first partial images indicating a corresponding image of the hot bed at the plurality of different positions; the causing the color camera to move along the scanning path under the bearing of the print head and capture images at the plurality of different positions to obtain a plurality of second partial images to generate a global image corresponding to the model reference map comprises: causing the color camera to move along the scanning path under the bearing of the print head and capture images at the plurality of different positions to obtain a plurality of second partial images; determining a plurality of local result images based on a respective difference between each of the plurality of second partial images and a corresponding one of the plurality of first partial images; stitching the plurality of local result images according to the scanning path to generate a global image corresponding to the model reference map.

3. The method of claim 2, wherein, The determining a plurality of local result images comprises: obtaining a plurality of difference images by calculating a difference between each of the plurality of second partial images and a corresponding one of the plurality of first partial images; for each difference image, determining a threshold value for judging valid pixels, each valid pixel indicating that the pixel belongs to the occupied area; performing a masking operation for each pixel in each difference image, the masking operation comprising: comparing the pixel value of the pixel with the threshold value, and in response to the pixel value of the pixel being greater than the threshold value, determining that the pixel belongs to the valid pixels and replacing the pixel value of the pixel with a pixel value of a pixel of a corresponding second local image; and outputting the plurality of difference images after the masking operation as the plurality of local result images.

4. The method of claim 3, wherein, The determining the threshold value for judging the valid pixels comprises: determining an average variance of each pixel in a first local image corresponding to the difference image; based on the average variance, determining the threshold value.

5. The method of any one of claims 2 to 4, further comprising: determining a brightness field indicating a uniformity of a shooting light of the color camera; and based on the brightness field, performing light compensation on the plurality of local result images. The determining the brightness field indicating the uniformity of the shooting light comprises:

6. The method of claim 5, wherein, determining the brightness field according to predetermined calibration information, wherein the calibration information indicates a default uniformity of a shooting light in a field of view of the color camera. The determining the brightness field indicating the uniformity of the shooting light comprises:

7. The method of claim 5, wherein, determining an average image of the plurality of first local images, wherein a pixel value of each pixel of the average image corresponds to an average value of pixel values of corresponding pixels of the plurality of first local images; and based at least in part on the average image, determining the brightness field. The model reference image indicates target printed images at the plurality of different positions, and the global image indicates actual printed images at the plurality of different positions, wherein the determining the print quality result comprises:

8. The method of any one of claims 1 to 7, wherein, comparing the target printed images at the plurality of different positions with the actual printed images at corresponding positions of the plurality of different positions; and based on the comparison, determining the print quality result. The comparing comprises:

9. The method of claim 8, wherein, determining a normal printing threshold value of a first layer of the 3D model; based on the normal printing threshold value, binarizing the global image and the model reference image; identifying an error pixel in the global image by comparing pixel values of corresponding pixels of the binarized global image and the model reference image, wherein the error pixel indicates that the first layer of the 3D model is not normally printed; determining at least one pixel region of the model reference image representing the occupied region; for at least one of the at least one pixel region: counting a number of error pixels in each pixel of the global image corresponding to the pixel region; and comparing the number of error pixels with a corresponding threshold value, and / or comparing a ratio of the number of error pixels to a number of pixels of the global image corresponding to the pixel region with a corresponding threshold value. The print quality result comprises a confidence indicating a detection reliability, which is a function of a total number of the valid pixels and a total number of pixels of the model reference image.

10. The method of claim 3, wherein, The splicing the plurality of local result images according to the scan path comprises:

11. The method of any one of claims 2-4, wherein, in response to determining that the plurality of local result images overlap with each other, averaging or linearly interpolating pixel values of pixels at overlapping positions between the plurality of local result images.

12. A 3D printer, comprising: a hot bed; ​ a print head movable relative to the hotbed; at least one processor configured to control the print head to move relative to the hotbed based on control code generated by slicing software to print the 3D model layer by layer, wherein the at least one processor is further configured to execute instructions to implement the method of any one of claims 1 to 11.

13. A non-transitory computer readable storage medium having stored therein instructions, wherein, The instructions, when executed by the at least one processor of the 3D printer of claim 12, implement the method of any one of claims 1 to 11.

14. A computer program product comprising instructions which, when executed by the at least one processor of the 3D printer of claim 12, implement the method of any one of claims 1 to 11.

Images