A method and apparatus for calibrating image acquisition devices for 3D printers

Abstract

This application provides a method and apparatus for calibrating an image acquisition device used in a 3D printer. The image acquisition device is fixed to the nozzle of the 3D printer. The method includes: acquiring a first coordinate of at least one target point in a world coordinate system; a second coordinate of the printing nozzle in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including at least one target point; and a third coordinate of at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in the calibration pattern printed by the printing nozzle; and determining extrinsic parameters of the image acquisition device based on the first, second, and third coordinates to calibrate the image acquisition device, wherein the extrinsic parameters include a translation vector between the image acquisition device and the printing nozzle. The image acquisition device calibration method and apparatus provided in this application can accurately calibrate the image acquisition device.

Description

Technical Field

[0001] This application relates to the field of 3D printing technology, and more specifically, to a method and apparatus for calibrating an image acquisition device for a 3D printer. Background Technology

[0002] Hot bed leveling, flow calibration, and first-layer inspection are essential processes before three-dimensional (3D) printing, directly determining print quality and success rate. Currently, the effectiveness of hot bed leveling, flow calibration, and first-layer inspection in most 3D printers is generally poor. For example, hot bed leveling using pressure sensors and manual buttons can only measure information at a limited number of locations on the hot bed, resulting in poor leveling performance and a time-consuming process.

[0003] By installing an image acquisition device, such as a 3D laser profilometer, on the printing nozzle and accurately calibrating the image acquisition device, hot bed leveling, flow calibration, and first-layer detection can be performed. For example, using the image acquisition device to acquire point cloud data of the hot bed for hot bed leveling can improve the effect of hot bed leveling, reduce the hot bed leveling time, improve the flow calibration accuracy of the printing nozzle, and improve the accuracy of first-layer detection, thereby improving the quality and success rate of 3D printing.

[0004] Therefore, how to accurately calibrate the image acquisition device of a 3D printer is one of the urgent problems to be solved. Summary of the Invention

[0005] This application provides a method and apparatus for calibrating an image acquisition device for a 3D printer, which can accurately calibrate the image acquisition device, thereby improving the quality of 3D printing.

[0006] In a first aspect, a calibration method for an image acquisition device applied to a 3D printer is provided. The image acquisition device is fixed to the print head of the 3D printer. The method includes: acquiring the first coordinates of at least one target point in the world coordinate system; the second coordinates of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including at least one target point; and the third coordinates of at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in the calibration graphic printed by the print head; and determining the extrinsic parameters of the image acquisition device based on the first coordinates, the second coordinates, and the third coordinates to calibrate the image acquisition device, wherein the extrinsic parameters include the translation vector between the image acquisition device and the print head.

[0007] In this embodiment, the external parameters of the image acquisition device can be accurately calibrated by using the first coordinate of the target point in the calibration graphic printed by the print head in the world coordinate system, the second coordinate of the print head in the world coordinate system when the image acquisition device acquires the two-dimensional image or three-dimensional point cloud of the target point, and the third coordinate of the image coordinate system of the two-dimensional image of the target point or the camera coordinate system corresponding to the three-dimensional point cloud. This allows for the improvement of the effects of heated bed leveling, flow calibration, and first-layer detection, thereby improving the quality of 3D printing.

[0008] On the other hand, by accurately calibrating the external parameters of the image acquisition device, the installation accuracy requirements of the image acquisition device can be reduced, and the installation efficiency of the image acquisition device can be improved.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the extrinsic parameters of the image acquisition device are determined based on the first coordinate, the second coordinate, and the third coordinate, including: determining the fourth coordinate of at least one target point in a coordinate system with the print head as the origin based on the first coordinate and the second coordinate when acquiring the 3D point cloud; and determining the extrinsic parameters based on the fourth coordinate and the third coordinate of at least one target point in the camera coordinate system.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the extrinsic parameters of the image acquisition device are determined based on the first coordinate, the second coordinate, and the third coordinate, including: T = (X... W ,Y W Z W )-(X C ,Y C Z C )-(P_X W ,P_Y W ,P_Z W ), where T is the translation vector, (X W ,Y W Z W (X) is the first coordinate, (X) C ,Y C Z C (P_X) represents the third coordinate of at least one target point in the camera coordinate system. W ,P_Y W ,P_Z W () represents the second coordinate of the printing nozzle when acquiring a 3D point cloud, and at least one target point includes one target point.

[0011] In the embodiments of this application, the external parameters of the image acquisition device can be accurately determined based on the coordinates of the target point in the world coordinate system, the coordinates of the printing nozzle in the world coordinate system when acquiring the three-dimensional point cloud including the target point, and the coordinates of the target point in the camera coordinate system. That is, by using the coordinates of the target point in the world coordinate system and the relevant information of the acquired three-dimensional point cloud of the target point, the external parameters of the image acquisition device can be accurately determined.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the extrinsic parameters also include the rotation matrix between the camera coordinate system and the world coordinate system.

[0013] In the embodiments of this application, the image acquisition device can be calibrated based on multiple degrees of freedom of translation vectors and rotation matrices, thereby improving the accuracy of image acquisition device calibration.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the extrinsic parameters of the image acquisition device are determined based on the first coordinate, the second coordinate, and the fourth coordinate, including: determining the rotation matrix and the translation vector using the following formula, (X... Wi ,Y Wi Z Wi ) = R c (X Ci ,Y Ci Z Ci )+T c +(P_X Wi ,P_Y Wi ,P_Z Wi ), where (X Wi ,Y Wi Z Wi (X) represents the first coordinate of the i-th target point. Ci ,Y Ci Z Ci Let (P_X) be the third coordinate of the i-th target point in the camera coordinate system. Wi ,P_Y Wi ,P_Z Wi R represents the second coordinate of the printing nozzle when acquiring the 3D point cloud including the i-th target point. c Let T be the rotation matrix. c It is a translation vector.

[0015] In conjunction with the first aspect, in certain implementations of the first aspect, before determining the extrinsic parameters of the image acquisition device based on the first, second, and third coordinates, the method further includes: obtaining a first offset ΔZ between at least one target point and the camera optical center in the world coordinate system Z-axis when acquiring a two-dimensional image, wherein the camera optical center is the optical center of the camera corresponding to the image acquisition device, and at least one target point is located on the surface of the calibration pattern on one side of the image acquisition device; obtaining the focal length of the image acquisition device when acquiring a two-dimensional image; determining the extrinsic parameters of the image acquisition device based on the first, second, and third coordinates, including: determining a second offset (ΔX, ΔY) between at least one target point and the camera optical center in the world coordinate system X-axis and Y-axis when acquiring a two-dimensional image based on the third coordinate of at least one target point in the image coordinate system, the first offset, and the focal length; determining a fourth coordinate of the camera optical center in the world coordinate system when acquiring a two-dimensional image based on the first offset, the second offset, and the first coordinates; and determining a translation vector based on the second and fourth coordinates.

[0016] In this embodiment, the offset between the target point and the camera's optical center in the world coordinate system can be determined based on the target point's coordinates in the world coordinate system, the offset between the image acquisition device and the target point on the Z-axis of the world coordinate system when acquiring the 2D image, and the focal length of the image acquisition device when acquiring the 2D image. Therefore, the coordinates of the camera's optical center in the world coordinate system can be determined based on this offset and the target point's coordinates. Furthermore, the translation vector between the image acquisition device and the print head can be accurately determined based on the coordinates of the camera's optical center and the print head in the world coordinate system. In other words, the solution provided in this application can accurately calibrate the image acquisition device using the target point's coordinates in the world coordinate system and relevant information from the acquired 2D image of the target point.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, determining the second offset of at least one target point from the camera optical center in the X and Y axes of the world coordinate system when acquiring a two-dimensional image, based on the third coordinate of at least one target point in the image coordinate system, the first offset, and the focal length, includes: determining the third offset (Δx, Δy) of at least one target point from the principal point of the two-dimensional image in each coordinate axis of the image coordinate system based on the third coordinate; and determining the second offset based on the first offset, the third offset, and the focal length.

[0018] In conjunction with the first aspect, in certain implementations of the first aspect, the second offset is determined based on the third offset, the first offset, and the focal length, including: Where, ΔX m ΔY m ΔZ m These represent the offsets of the camera optical center relative to the m-th target point in the world coordinate system along the X, Y, and Z axes, respectively, when acquiring a 2D image including the m-th target point.m The focal length Δx of the image acquisition device when acquiring a two-dimensional image including the m-th target point is given by... m Δy m These are the offsets of the principal point of the two-dimensional image including the m-th target point along the x-axis and y-axis of the image coordinate system, respectively, where the m-th target point is one of at least one target point.

[0019] In this embodiment, the offset between the target point and the camera optical center in the X and Y axes of the world coordinate system can be accurately determined by using similar triangles based on the offset between the target point and the principal point of the two-dimensional image in the image coordinate system.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, before acquiring the first coordinates of at least one target point in the world coordinate system, the second coordinates of the printing nozzle in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including at least one target point, and the third coordinates of at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, the method further includes: controlling the printing nozzle to print a calibration graphic on a heated bed, the 3D printer including the heated bed; and controlling the image acquisition device to acquire a two-dimensional image or a three-dimensional point cloud.

[0021] Secondly, a calibration device for an image acquisition device applied to a 3D printer is provided. The image acquisition device is fixed to the print head of the 3D printer. The device includes: an acquisition unit, configured to acquire a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in a world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including at least one target point, and a third coordinate of at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in a calibration graphic printed by the print head; and a processing unit, configured to determine the extrinsic parameters of the image acquisition device based on the first, second, and third coordinates to calibrate the image acquisition device, wherein the extrinsic parameters include a translation vector between the image acquisition device and the print head.

[0022] Thirdly, a 3D printer is provided, comprising: a heated bed; a print head for printing a target graphic on the heated bed; an image acquisition device for acquiring a two-dimensional image or a three-dimensional point cloud including target points of the target graphic, the image acquisition device being mounted on the print head; and a main control chip for controlling the print head and the image acquisition device, and further for acquiring the two-dimensional image or three-dimensional point cloud acquired by the image acquisition device and processing the two-dimensional image or three-dimensional point cloud according to the method described in the first aspect or any implementation thereof to obtain external parameters between the image acquisition device and the print head.

[0023] Fourthly, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the methods described in the first aspect or any implementation thereof.

[0024] The beneficial effects of the second to fourth aspects mentioned above can be referred to the relevant description in the first aspect, and will not be repeated here. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure of the 3D printer provided in the embodiments of this application.

[0026] Figure 2 This is a schematic diagram illustrating the working principle of the image acquisition device provided in the embodiments of this application.

[0027] Figure 3 This is a schematic diagram of the image acquisition device fixed to the print head according to an embodiment of this application.

[0028] Figure 4 This is a flowchart illustrating the calibration method for an image acquisition device provided in an embodiment of this application.

[0029] Figure 5 This is a schematic diagram of the calibration pattern provided in the embodiments of this application.

[0030] Figure 6 This is a schematic diagram of a two-dimensional image acquired by the image acquisition device provided in the embodiments of this application.

[0031] Figure 7 This is a flowchart illustrating the calibration method for an image acquisition device provided in an embodiment of this application.

[0032] Figure 8 This is a schematic diagram illustrating the principle of determining the second offset provided in the embodiments of this application.

[0033] Figure 9 This is a schematic block diagram of the calibration device for the image acquisition equipment provided in the embodiments of this application.

[0034] Figure 10 This is a schematic block diagram of the 3D printer provided in the embodiments of this application. Detailed Implementation

[0035] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0036] The terms “comprising” and “having”, and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0037] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0038] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0039] Figure 1 This is a schematic diagram of the structure of a 3D printer applicable to the embodiments of this application.

[0040] like Figure 1 As shown, the fused deposition modeling (FDM) 3D printer 100 includes a filament roll 101, a feeding device 102, an extruder 103, a print head 104, and a heated bed 105. The feeding device 102 can be connected to the filament roll 101. During the printing process, the feeding device 102 can obtain filamentous material from the filament roll 101, and after the filamentous material is melted by the extruder 103, it is ejected from the print head 104 and hardened and deposited on the heated bed 105. The materials used in 3D printing are generally thermoplastic. For example, the printing materials of a 3D printer include polymers, low-melting-point metals, and other materials that can be formulated into a flowable paste (such as paste-like ceramics, mixtures of high-melting-point metal powders, cement, etc.).

[0041] Before printing with a 3D printer, it is generally necessary to perform heated bed leveling, flow calibration, and first-layer testing. The effectiveness of heated bed leveling, flow calibration, and first-layer testing directly determines the print quality and success rate.

[0042] Currently, the performance of most 3D printers in terms of heated bed leveling, flow calibration, and first-layer detection is generally poor. For example, heated bed leveling is achieved using pressure sensors combined with manual or automatic compensation methods. However, this method can only measure information from a limited number of locations on the heated bed, resulting in mediocre leveling effectiveness and a time-consuming and labor-intensive process. Furthermore, this method may fail to detect localized unevenness on the heated bed, thus failing to provide effective compensation in subsequent printing and affecting print quality. Alternatively, flow calibration can be achieved by comprehensively considering the changes in nozzle flow rate and printing material properties over time. This flow calibration method requires extensive data collection on nozzle flow rate and printing material properties beforehand, which is time-consuming and complex.

[0043] To address the aforementioned issues, this application improves the effectiveness of heated bed leveling, flow calibration, and first-layer detection by fixing an image acquisition device, such as a 3D laser profilometer, to the print head of a 3D printer and accurately calibrating the image acquisition device. The calibrated image acquisition device is then used for heated bed leveling, flow calibration, and first-layer detection.

[0044] Specifically, using image acquisition equipment to obtain point cloud data of the heated bed for 3D reconstruction to level the heated bed can improve the leveling effect and reduce the leveling time. The 3D reconstructed point cloud can also detect whether there are unevennesses in the heated bed, so that effective compensation can be made in the subsequent printing process. Using image acquisition equipment to obtain point cloud data of the print head's ejection flow rate and perform 3D reconstruction can quickly and accurately determine whether the print head's flow rate meets the requirements based on the 3D reconstructed point cloud data, so as to perform flow rate calibration.

[0045] The image acquisition device provided in this application embodiment may include a light source and a camera. The image acquisition device may be a depth camera, a laser profilometer, or a single-line LiDAR, etc. For simplicity, Figure 2 The working principle of image acquisition equipment will be introduced by taking a laser profilometer as an example.

[0046] The laser profilometer 200 may include a laser emitter 210 (light source) and a camera 220. The optical axis of the laser emitter 210 and the optical axis of the camera 220 may form a certain angle or be parallel. Based on the principle of laser triangulation, the laser emitter 210 projects a line laser onto the object being measured. The center line of the line laser is extracted from the image captured by the camera 220. Then, using the light plane equation and the calibrated intrinsic and extrinsic parameters of the camera 210, the point cloud of the object being measured in the world coordinate system is calculated.

[0047] Specifically, laser emitter 210 emits a line laser beam towards the object under test. This line laser beam forms a scalpel plane, and each scalpel plane corresponds to a scalpel plane equation, which can be obtained through calibration. Camera 220 generates a line image on the imaging plane based on the beam reflected from the object under test. Any point on the line laser beam in the line image is extracted, and a ray is drawn from the optical center of camera 220 to this point. The intersection of this ray with the scalpel plane determines the three-dimensional coordinates of this point in the camera coordinate system. This allows the acquisition of the point cloud of the object under test in the camera coordinate system. Furthermore, based on the extrinsic parameters of camera 220, the point cloud information of the object under test in the world coordinate system is obtained.

[0048] In some embodiments of this application, to obtain accurate point cloud information, a centerline extraction algorithm can be used to extract the centerline of the line laser in the line image. Simultaneously, a ray is formed from the optical center of the camera 220 and any center point of the line laser. By determining the intersection of this ray and the laser blade plane, the three-dimensional coordinates of the corresponding center point can be obtained, thus acquiring the point cloud data of the object under test in the camera coordinate system. Compared to directly calculating the point cloud of the object under test from any point on the line laser, obtaining the sub-pixel coordinates of the corresponding center point by calculating the centerline on the line laser can improve the accuracy of the point cloud data.

[0049] In some embodiments of this application, the accuracy of point cloud data can also be improved by filtering the point cloud data of the object under test in the camera coordinate system.

[0050] In some embodiments of this application, the camera 220 of the laser profilometer 200 can also be used independently to acquire two-dimensional images of the object under test, thereby obtaining information such as line width, shape, and texture of the object under test.

[0051] Figure 3 The image shown is a top view of the image acquisition device fixed to the print head. Figure 3 As shown, the image acquisition device 310 is fixed together with the print head 104 so that the image acquisition device 310 can move with the print head 104. The image acquisition device 310 can be installed next to the print head 104, such as by mechanical connection, or it can be installed inside the print head 104. This application does not limit the installation position of the image acquisition device 310. When the two are installed, the relative pose of the image acquisition device 310 and the print head 104 is fixed, that is, the relative deviation between the two in the movement directions of the three drive axes of the 3D printer (X, Y, and Z) is fixed.

[0052] Camera lens distortion can be corrected using the camera intrinsic parameters calibrated by the image acquisition device. Specifically, a calibration plate 320 parallel to the heated bed 105 can be set on the side of the heated bed. The image acquisition device 310 acquires calibration images including those of the calibration plate 320, thereby obtaining the camera's intrinsic parameters according to a preset calibration algorithm to correct the camera's lens distortion. The preset calibration algorithm may include Zhang Zhengyou's calibration algorithm, etc.

[0053] The point cloud data acquired by the image acquisition device 310 is represented in the camera coordinate system of the image acquisition device 310, where the origin of the camera coordinate system is the optical center of the camera. Assume a point cloud is represented in the camera coordinate system as (X... C Y C Z C Because the camera coordinate system, with its origin at the camera optical center, shifts relative to the world coordinate system during the acquisition of multiple point clouds by the moving image acquisition device 310, the point clouds obtained solely by the image acquisition device 310 cannot be transformed from the camera coordinate system to the world coordinate system, with a corner of the heated bed as its origin, to achieve point cloud stitching. However, the printing nozzle needs to perform printing operations on the heated bed under appropriate control, meaning its coordinates in the world coordinate system are known. Therefore, each frame of point cloud can be mapped to the printing nozzle's coordinate system first, and then the point clouds in the printing nozzle's coordinate system can be transformed to the world coordinate system to achieve point cloud stitching. It should be noted that the origin of the world coordinate system can be defined as any corner point on the heated bed 105, and the X and Y axes of the world coordinate system are parallel to the movement directions of the 3D printer's X and Y drive axes, respectively.

[0054] However, when the image acquisition device 310 is installed onto the print head 104, a certain installation tolerance is introduced, and this tolerance varies from 3D printer to 3D printer. To ensure the normal operation of the image acquisition device 310, its external parameters need to be calibrated before use.

[0055] During the scanning process of the 3D printer, the print head 104 can move according to a preset printing trajectory. Since the image acquisition device 310 is fixed to the print head 104, the print head 104 can drive the image acquisition device 310 to move together. Therefore, by acquiring the coordinates of the print head 104 in the world coordinate system in real time, and the relative pose relationship between the image acquisition device 310 and the print head 104, the point cloud of the camera coordinate system of the image acquisition device 310 can be mapped to the world coordinate system.

[0056] Formula (1) can be used to map a point cloud of the object under test in the camera coordinate system corresponding to the image acquisition device 310 to the world coordinate system.

[0057] (X W ,Y W Z W )=R(X C ,Y C Z C )+T+(P_X W ,P_Y W ,P_Z W ) Formula (1)

[0058] Among them, (X) W ,Y W Z W Let X be the coordinates of a point on the object being measured in the world coordinate system, and R be the rotation matrix between the camera coordinate system and the world coordinate system. C ,Y C Z C Let P_X be the coordinates of a point on the object being measured in the camera coordinate system, and T be the translation vector between the image acquisition device 310 and the print head 104. W ,P_Y W ,P_Z W The coordinates of the printing nozzle 104 in the world coordinate system when the image acquisition device acquires an image of a point on the object under test. In some embodiments of this application, the origin of the world coordinate system can be defined as one of the corner points of the heated bed, with its X-axis and Y-axis parallel to the movement directions of the X and Y drive axes of the 3D printer, respectively.

[0059] R can also be viewed as the rotation matrix between the coordinate system with the print head 104 as the origin and the camera coordinate system, and T can be viewed as the translation vector between the coordinate system with the print head 104 as the origin and the camera coordinate system. That is, R can be the rotation matrix (rotation amount) between the image acquisition device 310 and the print head 104, and T can be the translation vector (translation amount) between the image acquisition device 310 and the print head 104.

[0060] It should be understood that, in the embodiments of this application, the amount of rotation and translation between the image acquisition device 310 and the print head 104 refers to the amount of rotation and translation between the camera of the image acquisition device 310 and the print head 104.

[0061] Typically, the rotation matrix R is relatively small, and the offset introduced by R is much smaller than the offset introduced by the translation vector T. In this case, R can be ignored. That is, R is a 3*3 identity matrix with a value of 1 on the main diagonal and 0 on the others. For example, if the structural installation tolerances can ensure that the angle between the image acquisition device 310 and the print head 104 on each axis of the world coordinate system is less than 5°, the rotation matrix R can be considered negligible.

[0062] When the rotation matrix R can be ignored, Equation (1) can be simplified to Equation (2).

[0063] (X W ,Y W Z W )=(X C ,Y C Z C )+T+(P_X W ,P_Y W ,P_Z W ) Formula (2)

[0064] Therefore, in this embodiment of the application, at least the translation vector T between the image acquisition device 310 and the print head 104 is obtained to calibrate the image acquisition device 310. This allows the point cloud in the camera coordinate system corresponding to the image acquisition device 310 to be transformed into the world coordinate system, thereby realizing the stitching of multiple frames of point cloud acquired during the movement of the image acquisition device 310.

[0065] Figure 4 This is a schematic flowchart illustrating the calibration method for the image acquisition device provided in an embodiment of this application. Figure 4 The method can be applied to Figure 3 The image acquisition device shown.

[0066] 410. Obtain the first coordinates of at least one target point in the world coordinate system, the second coordinates of the printing nozzle in the world coordinate system when the image acquisition device acquires a two-dimensional image or three-dimensional point cloud including at least one target point, and the third coordinates of at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud. Wherein, at least one target point is a point in the calibration pattern printed by the printing nozzle.

[0067] In some embodiments of this application, before acquiring the first, second, and third coordinates, the print head is controlled to print a calibration pattern on the heated bed according to a preset trajectory; and the image acquisition device is controlled to acquire an image including at least one target point.

[0068] In some embodiments of this application, due to the limitation of the camera's field of view in the image acquisition device, the image acquisition device may be unable to acquire a clear 2D image / 3D point cloud or may be unable to acquire a 2D image / 3D point cloud including the target point. Therefore, before controlling the image acquisition device to acquire a 2D image or 3D point cloud including the target point, the print head can be moved to move the image acquisition device, so that the image acquisition device can clearly acquire a 2D image / 3D point cloud including the target point.

[0069] It should be understood that, in the embodiments of this application, when the camera of the image acquisition device can clearly acquire a two-dimensional image including the target point, it can be considered that the image acquisition device can clearly acquire a three-dimensional point cloud including the target point. That is, the coordinates of the printing nozzle in the world coordinate system are consistent when acquiring a two-dimensional image and when acquiring a three-dimensional point cloud.

[0070] The calibration graphics can be in various styles, such as Figure 5 As shown, from a top-down view, calibration graphics can include single-point graphics, circles, rings, squares, triangles, etc. The design of calibration graphics mainly considers the following aspects: ease of rapid printing, high printing accuracy, and ease of extracting the coordinates of the target point, etc. The target point can be the geometric center of the calibration graphic, the center of the upper surface of the calibration graphic, or a corner point of a triangular calibration graphic, etc.

[0071] In some embodiments of this application, a calibration pattern may include one target point, or it may include multiple target points. For example, in the calibration pattern of the triangle described above, the three corner points on the upper surface can be used as three target points respectively.

[0072] In some embodiments of this application, a frame of two-dimensional image acquired by the image acquisition device may include one target point, or it may include multiple target points. For example, taking the center of a circular calibration image as a target point, a frame of two-dimensional image may include multiple target points of circular calibration images.

[0073] In this embodiment of the application, since it is necessary to control the print head to print on the heated bed, the coordinates of the print head in the world coordinate system can be obtained in real time. Thus, the coordinates of the print head in the world coordinate system can be obtained when the image acquisition device acquires two-dimensional images or three-dimensional point clouds.

[0074] Since the printhead prints the calibration graphic according to a preset printing trajectory, the coordinates of the printhead in the world coordinate system and the coordinates of the target point of the calibration graphic in the world coordinate system can be obtained in real time. In one embodiment, the position of the edge contour of the calibration graphic in the world coordinate system can be obtained based on the movement trajectory of the printhead during printing, and the coordinates of the geometric center of the calibration graphic in the world coordinate system can be calculated based on the edge contour. In another embodiment, the position of the edge contour of the calibration graphic in the world coordinate system can also be obtained based on the movement trajectory of the printhead during printing, and the center coordinates of the calibration graphic on the plane formed by the X and Y axes of the world coordinate system can be calculated based on the edge contour. In addition, when the printhead prints the calibration graphic, the bottom of the printhead is exactly in contact with the calibration graphic; therefore, the height of the calibration graphic can be obtained based on the height of the printhead on the Z axis of the world coordinate system. Thus, the coordinates of the center point of the upper surface of the calibration graphic in the world coordinate system can be obtained.

[0075] In this embodiment, after one calibration pattern is printed, an image including the target points of that calibration pattern can be acquired, and then the next calibration pattern can be printed, and an image including the target points of that next calibration pattern can be acquired, and so on. Alternatively, images including the target points of all calibration patterns can be acquired after all calibration patterns have been printed. The images can be two-dimensional images of the object being measured, or they can be three-dimensional point clouds of the object being measured.

[0076] Figure 6 The image shown is a two-dimensional image of a calibration graphic target point acquired using an image acquisition device. In some embodiments of this application, the two-dimensional image includes a complete calibration graphic. The calibration graphic in the two-dimensional image can be identified using an image recognition algorithm, and the position of the target point of the calibration graphic in the image coordinate system can be further calculated. For example, the calibration graphic in the two-dimensional image can be identified using an edge detection algorithm or a deep learning algorithm, and the center coordinates (Px, Py) of the calibration graphic can be calculated based on its contour. In other embodiments of this application, the two-dimensional image of the target point acquired above includes a partial image of the calibration graphic. The coordinates of the target point in the image coordinate system can be calculated by identifying the local features of the calibration graphic in the two-dimensional image.

[0077] In some embodiments of this application, the image acquisition device acquires a 3D point cloud including a target point, thereby obtaining the coordinates of the target point in the camera coordinate system corresponding to the 3D point cloud. For example, on one hand, the 3D point cloud acquired by the image acquisition device may include point cloud information of a complete calibration graphic; based on the acquired 3D point cloud of the calibration graphic, the position of the geometric center of the calibration graphic is calculated, thereby determining the coordinates of the geometric center in the camera coordinate system. On the other hand, the 3D point cloud acquired by the image acquisition device may include a point cloud of a certain local area of ​​the calibration image, so as to calculate the position of the geometric center of the graphic using the local point cloud.

[0078] 420. Based on the first coordinate, the second coordinate, and the third coordinate, determine the extrinsic parameters of the image acquisition device to calibrate the image acquisition device; wherein, the extrinsic parameters of the image acquisition device include the translation vector between the image acquisition device and the print head.

[0079] In some embodiments of this application, at least one target point can be determined in a fourth coordinate system with the print head as the origin based on the first coordinate and the second coordinate when acquiring the 3D point cloud; and the external parameters of the image acquisition device can be determined based on the fourth coordinate and the third coordinate of at least one target point in the camera coordinate system.

[0080] For example, by calculating the difference between the target point's coordinates in the world coordinate system and the printhead's coordinates in the world coordinate system, the target point's coordinates in a coordinate system with the printhead as the origin can be obtained. The axes of the coordinate system with the printhead as the origin are aligned with those of the world coordinate system. Then, based on the target point's coordinates in the coordinate system with the printhead as the origin (which can be considered a movable world coordinate system) and its coordinates in the camera coordinate system, the extrinsic parameters of the image acquisition device can be determined.

[0081] In some embodiments of this application, the external parameters of the image acquisition device can be determined according to formula (3).

[0082] T = (X W ,Y W Z W )-(X C ,Y C Z C )-(P_X W ,P_Y W ,P_Z W ) Formula (3)

[0083] Where T is the translation vector, (X W ,Y W Z W (X) is the first coordinate, (X) C ,Y C Z C(P_X) represents the third coordinate of at least one target point in the camera coordinate system. W ,P_Y W ,P_Z W () represents the second coordinate of the printing nozzle when acquiring a 3D point cloud, and at least one target point includes one target point.

[0084] In other words, the translation vector T between the image acquisition device and the print head can be determined by the coordinates of a target point in the world coordinate system, the coordinates of the print head in the world coordinate system when acquiring a 3D point cloud including the target point, and the coordinates of the target point in the camera coordinate system corresponding to the 3D point cloud.

[0085] Optionally, using the above formula (3), multiple translation vectors can be calculated based on the coordinates of each target point in the world coordinate system, the coordinates of the printing nozzle in the world coordinate system when acquiring the 3D point cloud of each target point, and the coordinates of each target point in the camera coordinate system corresponding to its 3D point cloud. Then, the translation vector T between the image acquisition device and the printing nozzle is determined based on these multiple translation vectors. For example, the average value of these multiple translation vectors can be used as the translation vector T between the image acquisition device and the printing nozzle.

[0086] In some embodiments of this application, the extrinsic parameters of the image acquisition device may also include the rotation matrix R between the camera coordinate system and the world coordinate system.

[0087] Formula (4) can be used to first determine the initial values ​​of the rotation matrix and the translation vector.

[0088] (X Wi ,Y Wi Z Wi ) = R c (X Ci ,Y Ci Z Ci )+T c +(P_X Wi ,P_Y Wi ,P_Z Wi ) Formula (4)

[0089] Among them, (X) Wi ,Y Wi Z Wi (X) represents the first coordinate of the i-th target point. Ci ,Y Ci Z Ci Let (P_X) be the third coordinate of the i-th target point in the camera coordinate system. Wi ,P_Y Wi ,P_Z Wi R represents the second coordinate of the printing nozzle when acquiring the 3D point cloud including the i-th target point.c Let T be the rotation matrix. c The translation vector is denoted by . Since the rotation matrix and translation vector are six-degree-of-freedom parameters, the rotation matrix and translation vector can be obtained using the first, second, and fourth coordinates corresponding to the six target points.

[0090] Furthermore, the rotation matrix and translation vector obtained based on the six target points are considered as initial values. Using the first coordinates of one or more target points, the second coordinates of the printing nozzle when acquiring the 3D point cloud of one or more target points, and the third coordinates of one or more target points in the camera coordinate system, along with a loss function, the initial values ​​of the translation vector and rotation matrix are updated to obtain the optimal translation vector and rotation matrix. It should be noted that the target points used in obtaining the initial values ​​of the translation vector and rotation matrix can be the same set of target points or different sets of target points used in updating the initial values ​​of the translation and rotation matrices; this application does not impose any restrictions on this.

[0091] For example, based on the second coordinates of the print head when acquiring a 3D point cloud including one or more target points, the third coordinates of one or more target points in the camera coordinate system corresponding to their 3D point cloud, the initial values ​​of the translation vector and the rotation matrix, the predicted coordinates of one or more target points in the world coordinate system are calculated; then, based on the measured coordinates of one or more target points in the world coordinate system (i.e., the first coordinates) and the predicted coordinates, the initial values ​​of the translation vector and the rotation matrix are updated using a loss function such as the least squares method.

[0092] For example, the initial values ​​of the translation vector and the rotation matrix can be updated using formula (5) to obtain the translation vector T and the rotation matrix R.

[0093]

[0094] in, Let X be the loss function, representing the combination of variables in the function at the minimum of the square of the norm of * (loss value), (X) Cj ,Y Cj Z Cj Let (P_X) be the coordinates of the j-th target point in the camera coordinate system. wj ,P_Y Wj ,P_Z Wj (X) represents the coordinates of the printing nozzle in the world coordinate system when acquiring a 3D point cloud including the j-th target point. Wj ,Y wj Z Wj ) represents the coordinates of the j-th target point in the world coordinate system, and n represents the number of target points used.

[0095] Using the above formula (5), the initial values ​​of the rotation matrix and translation vector are iteratively updated until the loss function is minimized. Then, T is calculated when the loss function is minimized. c and R c The external parameters of the image acquisition device are the translation vector T and the rotation matrix R.

[0096] In this embodiment, the external parameters of the image acquisition device can also be determined based on the first coordinate of the target point in the world coordinate system, the second coordinate of the printing nozzle in the world coordinate system when acquiring the two-dimensional image of the target point, and the third coordinate of the target point in the image coordinate system corresponding to the two-dimensional image, as follows: Figure 7 This paper provides a detailed introduction to the method for determining the extrinsic parameters of the image acquisition device.

[0097] Figure 7 This is a flowchart illustrating a method for determining the extrinsic parameters of an image acquisition device according to an embodiment of this application. It is applied in a 3D printer. The device includes a heated bed, a print head, and an image acquisition device. The image acquisition device is mounted on the print head and can move synchronously with the print head. The specific method is as follows:

[0098] 710, controls the printhead to print the calibration pattern onto the heated bed.

[0099] 720 controls the movement of the print head to move the image acquisition device.

[0100] Due to the limited field of view of the camera in the image acquisition device, it may not be able to clearly capture a 2D image including the target point. Therefore, before controlling the image acquisition device to capture a 2D image including the target point, the print head can be moved to move the image acquisition device until it can capture a clear image including the target point. It should be noted that if the image acquisition device can clearly capture a 2D image including the target point, it is not necessary to control the print head movement.

[0101] 730, Control the image acquisition device to acquire a two-dimensional image including at least one target point.

[0102] 740, when acquiring a two-dimensional image, at least one target point is offset by a first offset ΔZ from the optical center of the camera of the image acquisition device on the Z-axis of the world coordinate system; at least one target point is located on the surface of the calibration pattern on one side of the image acquisition device.

[0103] Specifically, the image acquisition device can measure the offset ΔZ between the optical center of the camera and the upper surface of the calibration pattern in the world coordinate system Z-axis. For example, the calibration pattern on the heated bed can be scanned by the line laser and camera of a laser profilometer to obtain the point cloud of the calibration pattern in the camera coordinate system, and then the first offset ΔZ can be obtained based on the coordinates of the point cloud in the camera coordinate system.

[0104] 750, the focal length of the image acquisition device when acquiring a two-dimensional image.

[0105] 760, acquiring the first coordinates of at least one target point in the world coordinate system, the second coordinates of the printing nozzle in the world coordinate system when the image acquisition device acquires a two-dimensional image including at least one target point, and the third coordinates of at least one target point in the image coordinate system corresponding to the two-dimensional image; wherein, at least one target point is a point in the calibration pattern printed by the printing nozzle.

[0106] It should be noted that the description of step 760 can refer to the relevant content in step 410, and will not be repeated here.

[0107] 770. Based on the third coordinate, the first offset, and the focal length, determine the second offset (ΔX, ΔY) between at least one target point and the camera optical center in the world coordinate system X and Y axes when acquiring a two-dimensional image.

[0108] In some embodiments of this application, a third offset (Δx, Δy) between at least one target point and the principal point of the two-dimensional image on each coordinate axis of the image coordinate system can be determined based on the third coordinate; and a second offset (ΔX, ΔY) can be determined based on the third offset, the first offset, and the focal length of the image acquisition device.

[0109] like Figure 6 As shown, the coordinates (Px, Py) of the target point of the calibration graphic in the image coordinate system of the two-dimensional image can be calculated based on the position of the outline of the calibration graphic. Furthermore, the offset (Δx, Δy) between the coordinates (Px, Py) of the target point and the principal point (Cx, Cy) of the two-dimensional image on each coordinate axis in the image coordinate system can be calculated.

[0110] The aforementioned offsets (Δx, Δy), mapped onto the heated bed plane, represent the offsets (ΔX, ΔY) between the target point of the calibration pattern and the camera's optical center along the X and Y axes of the world coordinate system. For example... Figure 8 As shown, based on similar triangles, we can obtain... f is the focal length of the image acquisition device when acquiring a two-dimensional image including the target point. Given ΔZ, f, and (Δx, Δy), (ΔX, ΔY) can be determined based on ΔZ, f, and (Δx, Δy).

[0111] That is, the offsets (ΔX, ΔY) of at least one target point from the optical center on the X and Y axes of the world coordinate system can be determined using formulas (6) and (7).

[0112]

[0113]

[0114] Where, ΔX m ΔY m ΔZ m These represent the offsets of the camera optical center relative to the m-th target point in the world coordinate system along the X, Y, and Z axes, respectively, when acquiring a 2D image including the m-th target point. m The focal length Δx of the image acquisition device when acquiring a two-dimensional image including the m-th target point is given by... m Δy m These are the offsets of the principal point of the two-dimensional image including the m-th target point along the x-axis and y-axis of the image coordinate system, respectively, where the m-th target point is one of at least one target point.

[0115] 780. Based on the first offset, the second offset, and the first coordinate, determine the fourth coordinate of the camera optical center in the world coordinate system when acquiring a two-dimensional image. That is, the coordinates of the camera optical center in the world coordinate system can be determined based on the coordinates of the target point in the world coordinate system and the offsets of the target point from the camera optical center on each coordinate axis of the world coordinate system.

[0116] 790. Based on the second and fourth coordinates, determine the translation vector between the image acquisition device and the print head.

[0117] Based on the coordinates of the print head in the world coordinate system and the coordinates of the image acquisition device in the world coordinate system when acquiring a two-dimensional image including the target point, the translation vector between the image acquisition device and the print head is determined.

[0118] In some embodiments of this application, the aforementioned at least one target point can be a single target point. That is, a translation vector can be determined based on the coordinates and other parameters corresponding to a target point and used as the translation vector between the image acquisition device and the print head.

[0119] In some embodiments of this application, the aforementioned at least one target point can be multiple target points. That is, multiple translation vectors can be determined based on the coordinates and other parameters corresponding to the multiple target points, and then the translation vector T between the image acquisition device and the print head can be determined based on the multiple translation vectors. For example, the average value of the multiple translation vectors can be used as the translation vector T between the image acquisition device and the print head.

[0120] In the embodiments of this application, the order of the above processes does not imply the order of execution, and not all of the above processes need to be executed. The execution order of each process and whether or not it is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0121] The above text combined Figures 4 to 7The calibration method of the image acquisition device provided in the embodiments of this application is described below, in conjunction with Figures 9 to 10 This describes the apparatus embodiments of the present application.

[0122] Figure 9 A schematic block diagram of a calibration device 2000 for an image acquisition device provided in an embodiment of this application is shown. The image acquisition device is fixed to the print head of a 3D printer. The calibration device 2000 includes an acquisition unit 2010 and a processing unit 2020. The acquisition unit 2010 is used to acquire the first coordinates of at least one target point in the world coordinate system, the second coordinates of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or three-dimensional point cloud including at least one target point, and the third coordinates of at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud. The at least one target point is a point in the calibration pattern printed by the print head. The processing unit 2020 is used to determine the extrinsic parameters of the image acquisition device based on the first, second, and third coordinates to calibrate the image acquisition device. The extrinsic parameters include the translation vector between the image acquisition device and the print head.

[0123] In some embodiments of this application, the processing unit 2020 is configured to: determine a fourth coordinate of at least one target point in a coordinate system with the print head as the origin, based on the first coordinate and the second coordinate when acquiring the three-dimensional point cloud; and determine extrinsic parameters based on the fourth coordinate and the third coordinate of at least one target point in the camera coordinate system.

[0124] In some embodiments of this application, the processing unit 2020 is used to calculate the translation vector according to the following formula, T = (X W ,Y W Z W )-(X C ,Y C Z C )-(P_X W ,P_Y W ,P_Z W ), where T is the translation vector, (X W ,Y W Z W (X) is the first coordinate, (X) C ,Y C Z C (P_X) represents the third coordinate of at least one target point in the camera coordinate system. W ,P_Y W ,P_Z W () represents the second coordinate of the printing nozzle when acquiring 3D point clouds.

[0125] In some embodiments of this application, the extrinsic parameters also include a rotation matrix between the camera coordinate system and the world coordinate system.

[0126] In some embodiments of this application, the processing unit 2020 is configured to: determine the initial value of the rotation matrix and the initial value of the translation vector using the following formula, (X Wi ,Y Wi Z Wi ) = R c (X Ci ,Y Ci Z Ci )+T c +(P_X Wi ,P_Y Wi ,P_Z Wi ), where (X Wi ,Y Wi Z Wi Let (X) be the first coordinate of the i-th target point in the target points. Ci ,Y Ci Z Ci Let (P_X) be the third coordinate of the i-th target point in the camera coordinate system. Wi ,P_Y Wi ,P_Z Wi R represents the second coordinate of the printing nozzle when acquiring the 3D point cloud including the i-th target point. c Let T be the initial value of the rotation matrix. c The initial values ​​of the translation vector and rotation matrix are obtained by using the first coordinate of the target point, the second coordinate of the printing nozzle when acquiring the 3D point cloud including the target point, the third coordinate of the target point in the camera coordinate system, and the loss function.

[0127] In some embodiments of this application, the acquisition unit 2010 is used to acquire a first offset ΔZ between at least one target point and the camera optical center in the world coordinate system Z-axis when acquiring a two-dimensional image, wherein the camera optical center is the camera optical center corresponding to the image acquisition device, and at least one target point is located on the surface of the calibration pattern on one side of the image acquisition device; and to acquire the focal length of the image acquisition device when acquiring the two-dimensional image; the processing unit 2020 is used to: determine a second offset (ΔX, ΔY) between at least one target point and the camera optical center in the world coordinate system X-axis and Y-axis when acquiring the two-dimensional image based on the third coordinate of at least one target point in the image coordinate system, the first offset, and the focal length; determine a fourth coordinate of the camera optical center in the world coordinate system when acquiring the two-dimensional image based on the first offset, the second offset, and the first coordinate; and determine a translation vector based on the second coordinate and the fourth coordinate.

[0128] In some embodiments of this application, the processing unit 2020 is configured to: determine, based on the third coordinates, a third offset (Δx, Δy) between at least one target point and the principal point of the two-dimensional image on each coordinate axis of the image coordinate system; and determine a second offset based on the first offset, the third offset, and the focal length.

[0129] In some embodiments of this application, the processing unit 2020 is configured to: calculate a second offset according to the following formula, Where, ΔX m ΔY m ΔZ m These represent the offsets of the camera optical center relative to the m-th target point in the world coordinate system along the X, Y, and Z axes, respectively, when acquiring a 2D image including the m-th target point. m The focal length Δx of the image acquisition device when acquiring a two-dimensional image including the m-th target point is given by... m Δy m These are the offsets of the principal point of the two-dimensional image including the m-th target point along the x-axis and y-axis of the image coordinate system, respectively, where the m-th target point is one of at least one target point.

[0130] In some embodiments of this application, the processing unit 2020 is also used to: control the print head to print the calibration pattern on the heated bed; and control the image acquisition device to acquire two-dimensional images or three-dimensional point clouds.

[0131] In some embodiments of this application, the processing unit 2020 is further configured to: control the movement of the printing nozzle to drive the image acquisition device to move, thereby acquiring a two-dimensional image or a three-dimensional point cloud.

[0132] Figure 10 A schematic block diagram of a 3D printer 5000 provided in an embodiment of this application is shown. The 3D printer 5000 includes a heated bed 5100, a print head 5200, an image acquisition device 5300, and a main control chip 5400. The print head 5200 is used to print calibration graphics on the heated bed 5100. The image acquisition device 5300 is used to acquire a two-dimensional image or a three-dimensional point cloud including target points of the calibration graphics, and the image acquisition device 5300 is mounted on the print head. The main control chip 5400 is used to control the print head and the image acquisition device, and also to acquire the two-dimensional image or three-dimensional point cloud acquired by the image acquisition device and process the two-dimensional image or three-dimensional point cloud according to the calibration method provided in one or more embodiments of this application to obtain external parameters of the image acquisition device 5300 and the print head 5200.

[0133] In one embodiment, the main control chip 5400 includes a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to call and run the computer program stored in the memory to perform the various steps of the calibration method of the embodiments of this application.

[0134] This application also provides a computer-readable medium storing program code for execution by a device, the program code including a calibration method for performing the image acquisition device described in this application.

[0135] This application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface and executes the image acquisition device calibration method of this application.

[0136] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the image acquisition device calibration method in the embodiments of this application.

[0137] It should be noted that although only some components, such as the processor and memory, are shown in the above-described apparatus or system, those skilled in the art should understand that in specific implementations, the apparatus or system may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the above-described apparatus or system may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the above-described apparatus or system may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figures 9 to 10 All the devices shown.

[0138] It should be understood that the processor in any of the above embodiments of this application can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0139] Furthermore, the memory in any of the above embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM). By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0140] It should be understood that the formulas in the embodiments of this application are merely examples and are not intended to limit the scope of the embodiments of this application. The formulas can be modified, and these modifications should also fall within the scope of protection of this application.

[0141] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0142] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of this application are not limited in this respect.

[0143] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items.

[0144] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A calibration method for an image acquisition device used in a 3D printer, characterized in that, The image acquisition device is fixed to the print head of the 3D printer, and the method includes: The image acquisition device acquires the first coordinates of at least one target point in the world coordinate system, the second coordinates of the printing nozzle in the world coordinate system when acquiring a two-dimensional image or a three-dimensional point cloud including the at least one target point, and the third coordinates of the at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in the calibration graphic printed by the printing nozzle. Based on the first coordinate, the second coordinate, and the third coordinate, the extrinsic parameters of the image acquisition device are determined to calibrate the image acquisition device. The extrinsic parameters include the translation vector between the image acquisition device and the print head.

2. The method according to claim 1, characterized in that, Determining the extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate includes: Based on the first coordinates and the second coordinates when acquiring the 3D point cloud, determine the fourth coordinates of the at least one target point in the coordinate system with the print head as the origin; The extrinsic parameters are determined based on the fourth coordinate and the third coordinate of the at least one target point in the camera coordinate system.

3. The method according to claim 1 or 2, characterized in that, Determining the extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate includes: T=(X W ,Y W ,Z W )-(X C ,Y C ,Z C )-(P-X W ,P-Y W ,P-Z W ), Where T is the translation vector, (X) W ,Y W Z W (X) is the first coordinate, (X) C ,Y C Z C (P_X) is the third coordinate of the at least one target point in the camera coordinate system. W ,P_Y W ,P_Z W The second coordinate of the printing nozzle is used when acquiring the three-dimensional point cloud, and the at least one target point includes a target point.

4. The method according to claim 1 or 2, characterized in that, The extrinsic parameters also include the rotation matrix between the camera coordinate system and the world coordinate system.

5. The method according to claim 4, characterized in that, Determining the extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate includes determining the rotation matrix and the translation vector using the following formulas. (X Wi ,Y Wi ,Z Wi )=R c (X Ci ,Y Ci ,Z Ci )+T c +(P_X Wi ,P_Y Wi ,P_Z Wi ), Among them, (X) Wi ,Y Wi Z Wi (X) represents the first coordinate of the i-th target point. Ci ,Y Ci Z Ci (P_X) represents the third coordinate of the i-th target point in the camera coordinate system. Wi ,P_Y Wi ,P_Z Wi R represents the second coordinate of the printing nozzle when acquiring the 3D point cloud including the i-th target point. c Let T be the rotation matrix. c Let be the translation vector.

6. The method according to claim 1, characterized in that, Before determining the extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate, the method further includes: When acquiring the two-dimensional image, the first offset ΔZ between the at least one target point and the camera optical center on the Z-axis of the world coordinate system is obtained. The camera optical center is the camera optical center corresponding to the image acquisition device. The at least one target point is located on the surface of the calibration pattern on one side of the image acquisition device. Obtain the focal length of the image acquisition device when acquiring the two-dimensional image; Determining the extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate includes: Based on the third coordinates of the at least one target point in the image coordinate system, the first offset, and the focal length, determine the second offset (ΔX, ΔY) between the at least one target point and the camera optical center in the X and Y axes of the world coordinate system when acquiring the two-dimensional image; Based on the first offset, the second offset, and the first coordinate, determine the fourth coordinate of the camera optical center in the world coordinate system when acquiring the two-dimensional image; The translation vector is determined based on the second coordinate and the fourth coordinate.

7. The method according to claim 6, characterized in that, The step of determining the second offset of the at least one target point from the camera optical center in the X and Y axes of the world coordinate system when acquiring the two-dimensional image, based on the third coordinate of the at least one target point in the image coordinate system, the first offset, and the focal length, includes: Based on the third coordinates, determine the third offset (Δx, Δy) between the at least one target point and the principal point of the two-dimensional image on each coordinate axis of the image coordinate system; The second offset is determined based on the first offset, the third offset, and the focal length.

8. The method according to claim 7, characterized in that, Determining the second offset based on the first offset, the third offset, and the focal length includes: Where, ΔX m ΔY m ΔZ m These are the offsets of the camera optical center relative to the m-th target point along the X, Y, and Z axes of the world coordinate system when acquiring a 2D image including the m-th target point, respectively. m The focal length Δx of the image acquisition device when acquiring a two-dimensional image including the m-th target point is given by [reference needed]. m Δy m These are the offsets of the principal point of the two-dimensional image including the m-th target point and the m-th target point on the x-axis and y-axis of the image coordinate system, respectively, where the m-th target point is one of the at least one target point.

9. The method according to any one of claims 1 to 8, characterized in that, Before acquiring the first coordinates of at least one target point in the world coordinate system, the second coordinates of the printing nozzle in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and the third coordinates of the at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, the method further includes: The print head is controlled to print the calibration pattern onto the heated bed of the 3D printer; Control the image acquisition device to acquire the two-dimensional image or the three-dimensional point cloud.

10. A calibration device for an image acquisition equipment used in a 3D printer, characterized in that, The image acquisition device is fixed to the print head of the 3D printer, and the device includes: The acquisition unit is used to acquire the first coordinates of at least one target point in the world coordinate system, the second coordinates of the printing nozzle in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and the third coordinates of the at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in the calibration graphic printed by the printing nozzle. The processing unit is configured to determine the extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate, so as to calibrate the image acquisition device, wherein the extrinsic parameters include the translation vector between the image acquisition device and the print head.

11. A 3D printer, characterized in that, include: heated bed; A printhead for printing calibration patterns on the heated bed; An image acquisition device is used to acquire a two-dimensional image or a three-dimensional point cloud including target points of the calibration graphic, the image acquisition device being mounted on the print head; and... The main control chip is used to control the print head and the image acquisition device, and is also used to acquire the image acquired by the image acquisition device and process the two-dimensional image or three-dimensional point cloud according to the method of any one of claims 1 to 9 to obtain the external parameters between the image acquisition device and the print head.

12. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 9.

Images