Image collection device calibration method and apparatus applied to 3D printer

Abstract

Provided in the present application are an image collection device calibration method and apparatus applied to a 3D printer. An image collection device is fixed on a nozzle of a 3D printer. The method comprises: acquiring first coordinates of at least one target point in a world coordinate system, second coordinates of a printing nozzle in the world coordinate system when an image collection device collects a two-dimensional image or a three-dimensional point cloud which comprises the at least one target point, and third coordinates of the at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the third-dimensional point cloud, wherein the at least one target point is a point in a calibration pattern printed by the printing nozzle; and determining extrinsic parameters of the image collection device according to the first coordinates, the second coordinates and the third coordinates, so as to calibrate the image collection device, wherein the extrinsic parameters comprise a translation vector between the image collection device and the printing nozzle. By means of the image collection device calibration method and apparatus provided in the embodiments of the present application, an image collection device can be accurately calibrated.

Description

Image acquisition device calibration method and device for 3D printer

[0001] This application claims priority to the Chinese patent application with application number 202310688341.1 filed with the State Intellectual Property Office of China on June 9, 2023, and priority to the Chinese patent application entitled “A method and apparatus for calibrating image acquisition equipment for 3D printers”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of 3D printing technology, and more specifically, to a method and apparatus for calibrating an image acquisition device applied to a 3D printer. Background Art

[0003] Hot bed leveling, flow calibration, and first-layer detection are essential processes before three-dimensional (3D) printing, directly determining print quality and success rate. Currently, most 3D printers perform these tasks ineffectively. For example, hot bed leveling, which relies on pressure sensors and manual controls, can only measure information at a limited number of locations on the hot bed, resulting in poor leveling and a time-consuming process.

[0004] By installing an image acquisition device such as a 3D laser profiler on the print head and accurately calibrating the image acquisition device, the hot bed can be leveled, the flow rate can be calibrated, the first layer detection can be performed with the help of the calibrated image acquisition device. For example, by using the image acquisition device to obtain the point cloud data of the hot bed to level the hot bed, the effect of hot bed leveling can be improved, the time of hot bed leveling can be reduced, the flow calibration accuracy of the printing nozzle and the accuracy of the first layer detection can be improved, thereby improving the quality and success rate of 3D printing.

[0005] Therefore, how to accurately calibrate the image acquisition device of a 3D printer is one of the problems that need to be solved urgently.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a method and apparatus for calibrating an image acquisition device for a 3D printer, which can accurately calibrate the image acquisition device, thereby improving the quality of 3D printing.

[0008] In a first aspect, a method for calibrating an image acquisition device applied to a 3D printer is provided, wherein the image acquisition device is fixed to a print head of the 3D printer, the method comprising: obtaining a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of the at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in a calibration figure printed by the print head; determining an extrinsic parameter of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate to calibrate the image acquisition device, wherein the extrinsic parameter comprises a translation vector between the image acquisition device and the print head.

[0009] In an embodiment of the present application, the external parameters of the image acquisition device can be accurately calibrated by using the first coordinate of the target point in the calibration figure printed by the print head in the world coordinate system, the second coordinate of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud of the target point, and the image coordinate system of the two-dimensional image of the target point or the third coordinate of the camera coordinate system corresponding to the three-dimensional point cloud of the target point. Therefore, the effects of hot bed leveling, flow calibration, and first-layer detection can be improved with the help of the calibrated image acquisition device, thereby improving the quality of 3D printing.

[0010] On the other hand, by accurately calibrating the external parameters of the image acquisition device, the installation accuracy requirements of the image acquisition device can be reduced and the installation efficiency of the image acquisition device can be improved.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the external parameters of the image acquisition device are determined based on the first coordinate, the second coordinate, and the third coordinate, including: determining the fourth coordinate of at least one target point in the coordinate system with the print head as the origin based on the first coordinate and the second coordinate when acquiring the three-dimensional point cloud; and determining the external parameters based on the fourth coordinate and the third coordinate of at least one target point in the camera coordinate system.

[0012] In combination with the first aspect, in some implementations of the first aspect, determining the external parameters of the image acquisition device according to the first coordinate, the second coordinate, and the third coordinate includes: T=(X W ,Y W ,Z W )-(X C ,Y C ,Z C )-(P - X W ,P - Y W ,P - Z W ), where T is the translation vector, (X W ,YW ,Z W ) is the first coordinate, (X C ,Y C ,Z C ) is the third coordinate of at least one target point in the camera coordinate system, (P - X W ,P - Y W ,P - Z W ) is the second coordinate of the printing head when collecting three-dimensional point clouds, and at least one target point includes a target point.

[0013] In an embodiment of the present application, the external parameters of the image acquisition device can be accurately determined based on the coordinates of the target point in the world coordinate system, the coordinates of the printing head in the world coordinate system when collecting the three-dimensional point cloud including the target point, and the coordinates of the target point in the camera coordinate system, that is, using the coordinates of the target point in the world coordinate system and the relevant information of the collected three-dimensional point cloud of the target point.

[0014] In combination with the first aspect, in some implementations of the first aspect, the external parameter further includes a rotation matrix between the camera coordinate system and the world coordinate system.

[0015] In an embodiment of the present application, the image acquisition device can be calibrated based on multiple degrees of freedom of the translation vector and the rotation matrix, thereby improving the accuracy of the calibration of the image acquisition device.

[0016] In combination with the first aspect, in certain implementations of the first aspect, determining the external parameters of the image acquisition device based on the first coordinate, the second coordinate, and the fourth coordinate includes: determining the rotation matrix and the translation vector using the following formula: (X Wi ,Y Wi ,Z Wi )=R c (X Ci ,Y Ci ,Z Ci )+T c +(P - X Wi ,P - Y Wi ,P - Z Wi ), where (X Wi ,Y Wi ,Z Wi ) is the first coordinate of the i-th target point, (X Ci ,Y Ci ,Z Ci ) is the third coordinate of the i-th target point in the camera coordinate system, (P - X Wi ,P - YWi ,P - Z Wi ) is the second coordinate of the print head when collecting the three-dimensional point cloud including the i-th target point, R c is the rotation matrix, T c is the translation vector.

[0017] In combination with the first aspect, in certain implementations of the first aspect, before determining the external parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate, the method also includes: obtaining a first offset ΔZ between at least one target point and the optical center of the camera in the Z axis of the world coordinate system when acquiring a two-dimensional image, the camera optical center being the camera optical center corresponding to the image acquisition device, and at least one target point being located on the surface of the calibration figure on one side of the image acquisition device; obtaining the focal length of the image acquisition device when acquiring the two-dimensional image; determining the external parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate, including: determining a second offset (ΔX, ΔY) between at least one target point and the optical center of the camera in the X axis and Y axis of the world coordinate system when acquiring the two-dimensional image based on the third coordinate of the at least one target point in the image coordinate system, the first offset, and the focal length; determining a fourth coordinate of the optical center of the camera in the world coordinate system when acquiring the two-dimensional image based on the first offset, the second offset, and the first coordinate; and determining a translation vector based on the second coordinate and the fourth coordinate.

[0018] In an embodiment of the present application, the offset between the target point and the camera's optical center in the world coordinate system can be determined based on the coordinates of the target point in the world coordinate system, the offset between the image acquisition device and the target point in the world coordinate system Z axis when acquiring a two-dimensional image, and the focal length of the image acquisition device when acquiring a two-dimensional image. Thus, the coordinates of the camera's optical center in the world coordinate system can be determined based on this offset and the coordinates of the target point in the world coordinate system. Furthermore, the translation vector between the image acquisition device and the print head can be accurately determined based on the coordinates of the camera's optical center and the print head's coordinates in the world coordinate system. In other words, the solution provided by the present application can accurately calibrate the image acquisition device using the coordinates of the target point in the world coordinate system and relevant information about the two-dimensional image of the target point.

[0019] In combination with the first aspect, in certain implementations of the first aspect, a second offset between at least one target point and the optical center of the camera in the X-axis and Y-axis of the world coordinate system when capturing a two-dimensional image is determined based on the third coordinate of the at least one target point in the image coordinate system, the first offset, and the focal length, including: determining a third offset (Δx, Δy) between the at least one target point and the principal point of the two-dimensional image in each coordinate axis of the image coordinate system based on the third coordinate; and determining a second offset based on the first offset, the third offset, and the focal length.

[0020] In combination with the first aspect, in certain implementations of the first aspect, determining the second offset according to the third offset, the first offset, and the focal length includes: Where ΔX m , ΔY m , ΔZ m are the offsets of the camera optical center and the mth target point in the world coordinate system on the X-axis, Y-axis, and Z-axis when acquiring a two-dimensional image including the mth target point, respectively. m is the focal length of the image acquisition device when acquiring a two-dimensional image including the mth target point, Δx m , Δy m are respectively the offsets between the principal point of the two-dimensional image including the m-th target point and the m-th target point in the image coordinate system on the x-axis and the y-axis, where the m-th target point is one of the at least one target point.

[0021] In an embodiment of the present application, the offset between the target point and the optical center of the camera in the X-axis and Y-axis of the world coordinate system can be accurately determined based on the offset between the target point and the principal point of the two-dimensional image in the image coordinate system using similar triangles.

[0022] In combination with the first aspect, in certain implementations of the first aspect, before obtaining the first coordinate of at least one target point in the world coordinate system, the second coordinate of the printing head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and the third coordinate of the at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, the method also includes: controlling the printing head to print a calibration graphic on a hot bed, the 3D printer including a hot bed; controlling the image acquisition device to acquire the two-dimensional image or the three-dimensional point cloud.

[0023] In a second aspect, a calibration device for an image acquisition device applied to a 3D printer is provided, wherein the image acquisition device is fixed to the print head of the 3D printer, and the device includes: an acquisition unit for acquiring a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of the at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in a calibration figure printed by the print head; a processing unit for determining the external parameters of the image acquisition device based on the first coordinate, the second coordinate and the third coordinate to calibrate the image acquisition device, wherein the external parameters include a translation vector between the image acquisition device and the print head.

[0024] In a third aspect, a 3D printer is provided, comprising: a heated bed; a print head for printing a calibration figure on the heated bed; an image acquisition device for acquiring a two-dimensional image or a three-dimensional point cloud including target points of the calibration figure, the image acquisition device being mounted on the print head; a main control chip for controlling the print head and the image acquisition device, and further for acquiring the two-dimensional image or the three-dimensional point cloud acquired by the image acquisition device and processing the two-dimensional image or the three-dimensional point cloud according to the method in the first aspect or any one of the implementations of the first aspect to obtain external parameters between the image acquisition device and the print head.

[0025] In a fourth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method according to the first aspect or any one of the implementations of the first aspect.

[0026] The beneficial effects of the second to fourth aspects mentioned above can be referred to the relevant description in the first aspect, and this application will not elaborate on them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a schematic structural diagram of a 3D printer provided in an embodiment of the present application.

[0028] FIG2 is a schematic diagram of the working principle of the image acquisition device provided in an embodiment of the present application.

[0029] FIG3 is a schematic diagram of an image acquisition device provided in an embodiment of the present application being fixed on a printing nozzle.

[0030] FIG4 is a flow chart of a calibration method for an image acquisition device provided in an embodiment of the present application.

[0031] FIG5 is a schematic diagram of a calibration graph provided in an embodiment of the present application.

[0032] FIG6 is a schematic diagram of a two-dimensional image captured by the image capture device provided in an embodiment of the present application.

[0033] FIG7 is a flow chart of a calibration method for an image acquisition device provided in an embodiment of the present application.

[0034] FIG8 is a schematic diagram of a principle for determining a second offset provided in an embodiment of the present application.

[0035] FIG9 is a schematic block diagram of a calibration apparatus for an image acquisition device according to an embodiment of the present application.

[0036] FIG10 is a schematic block diagram of a 3D printer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solution in this application will be described below with reference to the accompanying drawings.

[0038] The terms "including" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatus.

[0039] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0040] The terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0041] FIG1 is a schematic structural diagram of a 3D printer applicable to an embodiment of the present application.

[0042] As shown in FIG1 , a fused deposition modeling (FDM) 3D printer 100 includes a linear coil 101, a feeding device 102, an extruder 103, a print head 104, and a hot bed 105. The feeding device 102 can be connected to the linear coil 101. During the printing process, the feeding device 102 can obtain a filamentary material from the linear coil 101, and after the filamentary material is melted by the extruder 103, it is ejected by the print head 104 and hardened and accumulated on the hot bed 105. The materials used in 3D printing are generally thermoplastic materials. For example, the printing materials of a 3D printer include high molecular polymers, low melting point metals, and other materials that can be made into a fluid paste (such as paste-like ceramics, high melting point metal powder mixtures, cement, etc.).

[0043] Before using a 3D printer for printing, it is generally necessary to perform hot bed leveling, flow calibration, and first-layer testing. The effectiveness of these procedures directly determines the print quality and success rate.

[0044] Currently, the hot bed leveling, flow calibration, and first-layer detection performance of most 3D printers are generally mediocre. For example, a pressure sensor is used in combination with manual or automatic compensation methods to level the hot bed. However, this method can only measure information at a limited number of locations on the hot bed, resulting in mediocre leveling results and a relatively time-consuming and labor-intensive process. Furthermore, this method may not be able to detect localized unevenness on the hot bed, making it impossible to effectively compensate for it in subsequent prints, thus affecting print quality. Alternatively, the flow rate of the print nozzle can be controlled by comprehensively considering the time-varying patterns of the print nozzle's pipeline flow rate and the characteristics of the printed material. This flow calibration method requires a large amount of data statistics on the pipeline flow rate and the characteristics of the printed material in advance, which is time-consuming and complex.

[0045] To address the above problems, the present application fixes an image acquisition device, such as a 3D laser profiler, on the print head of a 3D printer, accurately calibrates the image acquisition device, and then uses the calibrated image acquisition device to perform hot bed leveling, flow calibration, first-layer detection, etc., which can improve the effects of hot bed leveling, flow calibration, and first-layer detection.

[0046] Specifically, using an image acquisition device to obtain point cloud data of the hot bed and perform three-dimensional reconstruction to level the hot bed can improve the leveling effect of the hot bed and reduce the leveling time of the hot bed; the three-dimensional reconstructed point cloud can also be used to detect whether the hot bed has unevenness, so that effective compensation can be made in the subsequent printing process; using an image acquisition device to obtain point cloud data of the ejection flow of the print head and perform three-dimensional reconstruction, it is possible to quickly and accurately determine whether the flow of the print head meets the requirements based on the three-dimensional reconstructed point cloud data, so as to calibrate the flow rate.

[0047] The image acquisition device provided in the embodiments of the present application may include a light source and a camera. The image acquisition device may be a depth camera, a laser profiler, or a single-line laser radar. For the sake of simplicity, FIG2 uses a laser profiler as an example to illustrate the working principle of the image acquisition device.

[0048] Laser profiler 200 includes a laser emitter 210 (light source) and a camera 220. The optical axis of laser emitter 210 and the optical axis of camera 220 can be at a certain angle or parallel. Based on the principle of laser triangulation, laser emitter 210 projects a line laser onto the object being measured. The centerline of the line laser in the image captured by camera 220 is extracted. Then, using the light plane equation and the calibrated internal and external parameters of camera 210, a point cloud of the object in the world coordinate system is calculated.

[0049] Specifically, the laser emitter 210 emits a line laser toward the object to be measured. A line laser forms a light knife plane, and a light knife plane corresponds to a light knife plane equation, which can be obtained through calibration. The camera 220 generates a line image on the imaging plane based on the collected light beam reflected by the object to be measured. Any point on the line laser in the line image is extracted and a ray is formed from the optical center of the camera 220 and any point on the line laser. The three-dimensional coordinates of the arbitrary point in the camera coordinate system are determined by the intersection of the ray and the light knife plane. This allows the point cloud of the object to be measured in the camera coordinate system to be obtained, and then the point cloud information of the object to be measured in the world coordinate system is obtained based on the external parameters of the camera 220.

[0050] In some embodiments of the present application, to obtain accurate point cloud information, a centerline extraction algorithm can be used to extract the centerline of the line laser in the line image. Simultaneously, a ray is formed from the optical center of camera 220 and any center point of the line laser. By determining the intersection of this ray and the light knife plane, the three-dimensional coordinates of the corresponding center point can be obtained, thereby obtaining point cloud data of the measured object in the camera coordinate system. Compared to directly calculating the point cloud of the measured object from any point on the line laser, calculating the centerline of the line laser to obtain the corresponding center point has sub-pixel coordinates, thereby improving the accuracy of the point cloud data.

[0051] In some embodiments of the present application, the accuracy of the point cloud data may also be improved by filtering the point cloud data of the measured object in the camera coordinate system.

[0052] In some embodiments of the present application, the camera 220 of the laser profiler 200 can also be used alone to capture a two-dimensional image of the object being measured, thereby obtaining information such as line width, graphics, and texture of the object being measured.

[0053] FIG3 shows a top view of the image acquisition device fixed to the print head. As shown in FIG3 , the image acquisition device 310 is fixed to the print head 104 so that the image acquisition device 310 can move with the print head 104. The image acquisition device 310 can be installed next to the print head 104, such as by mechanical connection, or inside the print head 104. This application does not limit the installation position of the image acquisition device 310. When the two are installed, the relative position of the image acquisition device 310 and the print head 104 is fixed, that is, the relative deviation between the two in the movement direction of the three drive axes X, Y, and Z of the 3D printer is fixed.

[0054] Camera lens distortion can be corrected using camera intrinsic parameters calibrated by the image acquisition device. Specifically, a calibration plate 320 can be provided on the side of the heated bed, relatively parallel to the heated bed 105. The image acquisition device 310 captures a calibration image including the calibration plate 320, and then calibrates the camera according to a preset calibration algorithm to obtain the camera intrinsic parameters, thereby correcting the camera lens distortion. Preset calibration algorithms may include the Zhang Zhengyou calibration algorithm, among others.

[0055] The point cloud data obtained by the image acquisition device 310 is expressed in the camera coordinate system of the image acquisition device 310. The origin of the camera coordinate system is the optical center of the camera of the image acquisition device 310. Assume that a point cloud is expressed as (X C , Y C , Z C ), since the camera coordinate system with the optical center of the camera of the image acquisition device 310 as the origin moves relative to the world coordinate system during the process of moving the image acquisition device 310 to acquire multiple frames of point clouds, the multiple frames of point clouds acquired solely by the image acquisition device 310 cannot convert the point clouds in the camera coordinate system into the world coordinate system constructed with a corner of the hot bed as the origin to achieve point cloud splicing. However, the print head needs to be controlled accordingly to perform printing on the hot bed, that is, the coordinates of the print head in the world coordinate system are known, so each frame of point cloud can be mapped to the coordinate system of the print head first, and then the point cloud in the coordinate system of the print head can be converted to the world coordinate system to achieve point cloud splicing. It should be noted that the origin of the world coordinate system can be defined as any corner point on the hot bed 105, and the X-axis and Y-axis of the world coordinate system are parallel to the movement directions of the X and Y drive shafts of the 3D printer, respectively.

[0056] However, when the image acquisition device 310 is mounted on the print head 104, a certain installation tolerance is introduced, and the installation tolerance on each 3D printer is different. In order to ensure the normal use of the image acquisition device 310, the external parameters of the image acquisition device 310 need to be calibrated before use.

[0057] During the 3D printer's scanning process, the print head 104 can move along a preset printing trajectory. Since the image acquisition device 310 is fixed to the print head 104, the print head 104 can move with it. Therefore, by using the real-time coordinates of the print head 104 in the world coordinate system and the relative positional relationship between the image acquisition device 310 and the print head 104, the point cloud in the camera coordinate system of the image acquisition device 310 can be mapped to the world coordinate system.

[0058] Formula (1) can be used to map a point cloud of the object under test in the camera coordinate system corresponding to the image acquisition device 310 to the world coordinate system. (XW ,Y W ,Z W )=R(X C ,Y C ,Z C )+T+(P - X W ,P - Y W ,P - Z W ) Formula (1)

[0059] Among them, (X W ,Y W ,Z W ) is the coordinate of a point of the object being measured in the world coordinate system, R is the rotation matrix between the camera coordinate system and the world coordinate system, (X C ,Y C ,Z C ) is the coordinate of a point on the object being measured in the camera coordinate system, T is the translation vector between the image acquisition device 310 and the print head 104, (P - X W ,P - Y W ,P - Z W ) is the coordinate of the print head 104 in the world coordinate system when the image acquisition device captures an image of a certain point on the measured object. In some embodiments of the present application, the origin of the world coordinate system can be defined as a corner point of the heated bed, with its X-axis and Y-axis parallel to the movement directions of the 3D printer's X and Y drive shafts, respectively.

[0060] R can also be viewed as the rotation matrix between the coordinate system with the print head 104 as the origin and the camera coordinate system, and T can be viewed as the translation vector between the coordinate system with the print head 104 as the origin and the camera coordinate system. That is, R can be the rotation matrix (rotation amount) between the image acquisition device 310 and the print head 104, and T can be the translation vector (translation amount) between the image acquisition device 310 and the print head 104.

[0061] It should be understood that in the embodiment of the present application, the rotation amount and translation amount between the image acquisition device 310 and the print head 104 refer to the rotation amount and translation amount between the camera of the image acquisition device 310 and the print head 104.

[0062] Typically, the rotation matrix R is relatively small, and the offset introduced by the rotation matrix R is much smaller than the offset introduced by the translation vector T. In this case, R can be ignored. That is, R is a 3*3 identity matrix with the main diagonal values ​​being 1 and the other values ​​being 0. For example, if the structural installation tolerances ensure that the angle between the image acquisition device 310 and the print head 104 on each axis of the world coordinate system is less than 5°, the rotation matrix R can be considered negligible.

[0063] When the rotation matrix R can be ignored, formula (1) can be simplified to formula (2). (X W ,Y W ,Z W )=(X C ,Y C ,Z C )+T+(P - X W ,P - Y W ,P - Z W ) Formula (2)

[0064] Therefore, in the embodiment of the present application, at least the translation vector T between the image acquisition device 310 and the print head 104 is obtained to calibrate the image acquisition device 310, so that the point cloud in the camera coordinate system corresponding to the image acquisition device 310 can be converted into the world coordinate system, thereby realizing the splicing of multiple frames of point clouds acquired during the movement of the image acquisition device 310.

[0065] FIG4 is a flow chart of a calibration method for an image acquisition device according to an embodiment of the present application. The method of FIG4 can be applied to the image acquisition device shown in FIG3.

[0066] 410. Obtain a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in the world coordinate system when the image acquisition device captures a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of the at least one target point in the image coordinate system corresponding to the two-dimensional image or the camera coordinate system corresponding to the three-dimensional point cloud. The at least one target point is a point in a calibration pattern printed by the print head.

[0067] In some embodiments of the present application, before obtaining the above-mentioned first coordinate, second coordinate and third coordinate, the print head is controlled to print a calibration pattern on the hot bed according to a preset trajectory; and the image acquisition device is controlled to acquire an image including at least one target point.

[0068] In some embodiments of the present application, due to limitations in the camera's capture angle of view, the image capture device may be unable to capture a clear 2D image / 3D point cloud, or may be unable to capture a 2D image / 3D point cloud that includes a target point. Therefore, before controlling the image capture device to capture a 2D image or 3D point cloud that includes a target point, the print head may be moved to drive the image capture device to move, so that the image capture device can clearly capture a 2D image / 3D point cloud that includes the target point.

[0069] It should be understood that in the embodiments of the present application, if the camera of the image acquisition device can clearly capture a 2D image including the target point, it can be considered that the image acquisition device is capable of clearly capturing a 3D point cloud including the target point. In other words, the coordinates of the print head in the world coordinate system are consistent when capturing the 2D image and the 3D point cloud.

[0070] Calibration patterns can take on a variety of styles. As shown in Figure 5, when viewed from a top view, they can include single-point patterns, circles, rings, squares, triangles, and more. The design of calibration patterns primarily considers the following aspects: fast printing, high printing accuracy, and ease of extracting target point coordinates. The target point can be the geometric center of the calibration pattern, the center of its top surface, or a corner point of a triangular calibration pattern.

[0071] In some embodiments of the present application, a calibration figure may include one target point, or may include multiple target points. For example, in the above-mentioned triangular calibration figure, the three corner points on its upper surface may serve as three target points respectively.

[0072] In some embodiments of the present application, a frame of a two-dimensional image captured by an image acquisition device may include one target point, or may include multiple target points. For example, with the center of a circular calibration image as a target point, a frame of a two-dimensional image may include multiple target points of circular calibration patterns.

[0073] In the embodiment of the present application, since it is necessary to print on a hot bed by controlling the print head, the coordinates of the print head in the world coordinate system can be obtained in real time, so that the coordinates of the print head in the world coordinate system can be obtained when the image acquisition device captures a two-dimensional image or a three-dimensional point cloud.

[0074] Because the print head prints the calibration pattern according to a preset print trajectory, the coordinates of the print head in the world coordinate system and the coordinates of the target point of the calibration pattern in the world coordinate system can be obtained in real time. In one embodiment, the position of the edge contour of the calibration pattern in the world coordinate system can be obtained based on the motion trajectory of the print head in the world coordinate system when printing the calibration pattern, and the coordinates of the geometric center of the calibration pattern in the world coordinate system can be calculated based on the edge contour. In another embodiment, the position of the edge contour of the calibration pattern in the world coordinate system can also be obtained based on the motion trajectory of the print head in the world coordinate system when printing the calibration pattern, and the center coordinates of the calibration pattern on the plane formed by the X and Y axes of the world coordinate system can be calculated based on the edge contour. In addition, when the print head prints the calibration pattern, the bottom of the print head fits perfectly with the calibration pattern. Therefore, the height of the calibration pattern can be obtained based on the height of the print head on the Z axis of the world coordinate system. In this way, the coordinates of the center point of the upper surface of the calibration pattern in the world coordinate system can be obtained.

[0075] In an embodiment of the present application, after a calibration pattern is printed, an image including the target points of the calibration pattern can be captured, and then the next calibration pattern can be printed and an image including the target points of the next calibration pattern can be captured, and so on. Alternatively, after all calibration patterns are printed, images including the target points of these calibration patterns can be captured. The image can be a two-dimensional image of the object being measured, or it can be a three-dimensional point cloud of the object being measured.

[0076] Figure 6 shows a two-dimensional image of the target point of the calibration figure collected by the image acquisition device. In some embodiments of the present application, the two-dimensional image includes a complete calibration figure, and the calibration figure in the two-dimensional image can be identified by an image recognition algorithm, and the position of the target point of the calibration figure in the image coordinate system can be further calculated. For example, the calibration figure in the two-dimensional image can be identified by an edge detection algorithm or a deep learning algorithm, and the center coordinates (Px, Py) of the calibration figure can be further calculated based on the contour of the calibration figure. In other embodiments of the present application, the two-dimensional image of the target point collected above includes a local image of the calibration figure, and the coordinates of the target point in the image coordinate system can be calculated by identifying the local features of the calibration figure in the two-dimensional image.

[0077] In some embodiments of the present application, an image acquisition device acquires a three-dimensional point cloud including a target point, thereby obtaining the coordinates of the target point in the camera coordinate system corresponding to the three-dimensional point cloud. For example, on the one hand, the three-dimensional point cloud acquired by the image acquisition device may include point cloud information of a complete calibration figure, and based on the acquired three-dimensional point cloud of the calibration figure, the position of the geometric center of the calibration figure is calculated, thereby determining the coordinates of the geometric center in the camera coordinate system; on the other hand, the three-dimensional point cloud acquired by the image acquisition device may include a point cloud of a portion of the calibration image, so that the position of the geometric center of the figure is calculated using the local point cloud.

[0078] 420. Determine extrinsic parameters of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate to calibrate the image acquisition device; wherein the extrinsic parameters of the image acquisition device include a translation vector between the image acquisition device and the print head.

[0079] In some embodiments of the present application, the fourth coordinate of at least one target point in a coordinate system with the print head as the origin can be determined based on the first coordinate and the second coordinate when collecting the three-dimensional point cloud; the external parameters of the image acquisition device can be determined based on the fourth coordinate and the third coordinate of at least one target point in the camera coordinate system.

[0080] For example, the coordinates of the target point in the coordinate system with the print head as the origin can be obtained by taking the difference between the coordinates of the target point in the world coordinate system and the coordinates of the print head in the world coordinate system. The directions of the coordinate axes of the coordinate system with the print head as the origin and the world coordinate system are consistent. The extrinsic parameters of the image acquisition device are then determined based on the coordinates of the target point in the coordinate system with the print head as the origin (which can be considered a movable world coordinate system) and the coordinates of the target point in the camera coordinate system.

[0081] In some embodiments of the present application, the external parameters of the image acquisition device can be determined according to formula (3). W ,Y W ,Z W )-(X C ,Y C ,Z C )-(P - X W ,P - Y W ,P - Z W ) Formula (3)

[0082] Where T is the translation vector, (X W ,Y W ,Z W ) is the first coordinate, (X C ,Y C ,Z C) is the third coordinate of at least one target point in the camera coordinate system, (P - X W ,P - Y W ,P - Z W ) is the second coordinate of the printing head when collecting three-dimensional point clouds, and at least one target point includes a target point.

[0083] That is to say, the translation vector T between the image acquisition device and the print head can be determined by the coordinates of a target point in the world coordinate system, the coordinates of the print head in the world coordinate system when collecting a three-dimensional point cloud including the target point, and the coordinates of the target point in the camera coordinate system corresponding to the three-dimensional point cloud.

[0084] Optionally, using the above formula (3), multiple translation vectors can be calculated based on the coordinates of each target point in the world coordinate system, the coordinates of the print head in the world coordinate system when collecting the three-dimensional point cloud of each target point, and the coordinates of each target point in the camera coordinate system corresponding to its three-dimensional point cloud. The translation vector T between the image acquisition device and the print head is then determined based on the multiple translation vectors. For example, the average of the multiple translation vectors is used as the translation vector T between the image acquisition device and the print head.

[0085] In some embodiments of the present application, the external parameters of the image acquisition device may also include a rotation matrix R between the camera coordinate system and the world coordinate system.

[0086] Formula (4) can be used to first determine the initial value of the rotation matrix and the initial value of the translation vector. (X Wi ,Y Wi ,Z Wi )=R c (X Ci ,Y Ci ,Z Ci )+T c +(P - X Wi ,P - Y Wi ,P - Z Wi ) Formula (4)

[0087] Among them, (X Wi ,Y Wi ,Z Wi ) is the first coordinate of the i-th target point, (X Ci ,Y Ci ,Z Ci ) is the third coordinate of the i-th target point in the camera coordinate system, (P - X Wi ,P- Y Wi ,P - Z Wi ) is the second coordinate of the print head when collecting the three-dimensional point cloud including the i-th target point, R c is the rotation matrix, T c is the translation vector. Since the rotation matrix and translation vector are parameters of six degrees of freedom, the rotation matrix and translation vector can be obtained through the first coordinate, second coordinate, and fourth coordinate corresponding to the six target points.

[0088] Furthermore, the rotation matrix and translation vector obtained based on the six target points are regarded as initial values, and the initial values ​​of the translation vector and the rotation matrix are updated using the first coordinates of one or more target points, the second coordinates of the print head when collecting the three-dimensional point cloud of one or more target points, and the third coordinates of one or more target points in the camera coordinate system and the loss function to obtain the optimal translation vector and rotation matrix. It should be noted that the target points used in the process of obtaining the initial values ​​of the translation vector and the rotation matrix and the target points used in the process of updating the initial values ​​of the translation matrix and the rotation matrix can be the same group of target points or different groups of target points, and this application does not impose any restrictions on this.

[0089] Exemplarily, the predicted coordinates of the one or more target points in the world coordinate system are calculated based on the second coordinate of the print head when collecting a three-dimensional point cloud including one or more target points, the third coordinates of the one or more target points in the camera coordinate system corresponding to their three-dimensional point cloud, the initial value of the translation vector and the initial value of the rotation matrix; then, based on the measured coordinates of the one or more target points in the world coordinate system (i.e., the first coordinates) and the above-mentioned predicted coordinates, a loss function such as the least squares method is used to update the initial value of the translation vector and the initial value of the rotation matrix.

[0090] For example, the initial value of the translation vector and the initial value of the rotation matrix can be updated using formula (5) to obtain the translation vector T and the rotation matrix R.

[0091] in, is the loss function, which represents the variable combination in the function when the square of the norm of * (loss value) is minimized, (X Cj ,Y Cj ,Z Cj ) is the coordinate of the jth target point in the camera coordinate system, (P - X wj ,P - Y Wj ,P - Z Wj ) is the coordinate of the print head in the world coordinate system when collecting the three-dimensional point cloud including the j-th target point, (X Wj ,Ywj ,Z Wj ) is the coordinate of the j-th target point in the world coordinate system, and n is the number of target points used.

[0092] By using the above formula (5), the initial value of the rotation matrix and the initial value of the translation vector are iteratively updated until the loss value of the loss function is minimized. Then the T c and R c As the external parameters of the image acquisition device, namely the translation vector T and the rotation matrix R.

[0093] In an embodiment of the present application, the external parameters of the image acquisition device can also be determined based on the first coordinate of the target point in the world coordinate system, the second coordinate of the printing head in the world coordinate system when acquiring the two-dimensional image of the target point, and the third coordinate of the target point in the image coordinate system corresponding to the two-dimensional image. The method of determining the external parameters of the image acquisition device is introduced in detail below in conjunction with Figure 7.

[0094] FIG7 is a flow chart of a method for determining external parameters of an image acquisition device provided in an embodiment of the present application, which is applied to a 3D printer. The device includes a heated bed, a print head, and an image acquisition device. The image acquisition device is mounted on the print head and can move synchronously with the print head. The method is specifically as follows:

[0095] 710, controlling the print head to print the calibration pattern on the hot bed.

[0096] 720, controlling the movement of the print head to drive the image acquisition device to move.

[0097] Due to limitations in the camera's field of view, the image capture device may not be able to clearly capture a two-dimensional image that includes the target point. Therefore, before controlling the image capture device to capture a two-dimensional image that includes the target point, the print head can be controlled to move, thereby driving the image capture device until the image capture device can capture a clear image that includes the target point. It should be noted that if the image capture device can clearly capture a two-dimensional image that includes the target point, controlling the print head's movement is not necessary.

[0098] 730 , controlling an image acquisition device to acquire a two-dimensional image including at least one target point.

[0099] 740. Obtain a first offset ΔZ between at least one target point and the camera optical center of the image acquisition device on the Z axis of the world coordinate system when acquiring a two-dimensional image; the at least one target point is located on the surface of the calibration pattern on one side of the image acquisition device.

[0100] Specifically, the image acquisition device can measure the offset ΔZ between the optical center of the camera of the image acquisition device and the upper surface of the calibration pattern on the Z axis of the world coordinate system. For example, the calibration pattern on the hot bed can be scanned by a line laser and a camera of a laser profiler to obtain a point cloud of the calibration pattern in the camera coordinate system, and then the first offset ΔZ can be obtained based on the coordinates of the point cloud in the camera coordinate system.

[0101] 750, obtaining the focal length corresponding to the image acquisition device when acquiring a two-dimensional image.

[0102] 760. Obtain a first coordinate of at least one target point in a world coordinate system, a second coordinate of a print head in the world coordinate system when an image acquisition device acquires a two-dimensional image including the at least one target point, and a third coordinate of the at least one target point in an image coordinate system corresponding to the two-dimensional image; wherein the at least one target point is a point in a calibration pattern printed by the print head.

[0103] It should be noted that the description of step 760 can refer to the relevant content in step 410, and this application will not go into details here.

[0104] 770. Determine, based on the third coordinate, the first offset, and the focal length, a second offset (ΔX, ΔY) between at least one target point and the optical center of the camera on the X-axis and the Y-axis of the world coordinate system when capturing the two-dimensional image.

[0105] In some embodiments of the present application, a third offset (Δx, Δy) between at least one target point and the principal point of the two-dimensional image on each coordinate axis of the image coordinate system can be determined based on the third coordinate; and a second offset (ΔX, ΔY) can be determined based on the third offset, the first offset, and the focal length of the image acquisition device.

[0106] As shown in Figure 6, the coordinates (Px, Py) of the target point of the calibration figure in the image coordinate system of the two-dimensional image can be calculated based on the position of the contour of the calibration figure, and the offset (Δx, Δy) between the coordinates of the target point (Px, Py) and the principal point (Cx, Cy) of the two-dimensional image on each coordinate axis in the image coordinate system can be further calculated.

[0107] The above offset (Δx, Δy) mapped to the hot bed plane represents the offset (ΔX, ΔY) between the target point of the calibration figure and the optical center of the camera in the X-axis and Y-axis of the world coordinate system. As shown in Figure 8, according to the similar triangle, we can get f is the focal length of the image acquisition device when acquiring a 2D image that includes the target point. Given ΔZ, f, and (Δx, Δy), we can determine (ΔX, ΔY) based on ΔZ, f, and (Δx, Δy).

[0108] That is, the offset (ΔX, ΔY) between at least one target point and the optical center on the X-axis and Y-axis of the world coordinate system can be determined using formula (6) and formula (7).

[0109] Where ΔX m , ΔY m , ΔZ m are the offsets of the camera optical center and the mth target point in the world coordinate system on the X-axis, Y-axis, and Z-axis when acquiring a two-dimensional image including the mth target point, respectively. m is the focal length of the image acquisition device when acquiring a two-dimensional image including the mth target point, Δx m , Δy m are respectively the offsets between the principal point of the two-dimensional image including the m-th target point and the m-th target point in the image coordinate system on the x-axis and the y-axis, where the m-th target point is one of the at least one target point.

[0110] At 780, a fourth coordinate of the camera optical center in the world coordinate system when acquiring the two-dimensional image is determined based on the first offset, the second offset, and the first coordinate. Specifically, the coordinate of the camera optical center in the world coordinate system can be determined based on the coordinates of the target point in the world coordinate system and the offsets between the target point and the camera optical center along each coordinate axis of the world coordinate system.

[0111] 790. Determine a translation vector between the image acquisition device and the print head based on the second coordinate and the fourth coordinate.

[0112] The translation vector between the image acquisition device and the print head is determined according to the coordinates of the print head in the world coordinate system and the coordinates of the image acquisition device in the world coordinate system when acquiring a two-dimensional image including the target point.

[0113] In some embodiments of the present application, the at least one target point may be a single target point, that is, a translation vector may be determined based on the coordinates and other parameters corresponding to the single target point and used as the translation vector between the image acquisition device and the print head.

[0114] In some embodiments of the present application, the at least one target point may be multiple target points. That is, multiple translation vectors may be determined based on the coordinates and other parameters corresponding to the multiple target points. The translation vector T between the image acquisition device and the print head may then be determined based on these multiple translation vectors. For example, the average of these multiple translation vectors may be used as the translation vector T between the image acquisition device and the print head.

[0115] In the embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution, and the above-mentioned processes do not have to be executed. The execution order of each process and whether it is executed or not should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0116] The above describes the calibration method of the image acquisition device provided in the embodiment of the present application in combination with Figures 4 to 7. The following describes the device embodiment of the embodiment of the present application in combination with Figures 9 and 10.

[0117] FIG9 shows a schematic block diagram of a calibration device 2000 for an image acquisition device provided in an embodiment of the present application. The image acquisition device is fixed to a print head of a 3D printer. The calibration device 2000 includes an acquisition unit 2010 and a processing unit 2020. The acquisition unit 2010 is configured to acquire a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in a world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in a calibration figure printed by the print head. The processing unit 2020 is configured to determine an extrinsic parameter of the image acquisition device based on the first coordinate, the second coordinate, and the third coordinate to calibrate the image acquisition device, wherein the extrinsic parameter includes a translation vector between the image acquisition device and the print head.

[0118] In some embodiments of the present application, the processing unit 2020 is used to: determine the fourth coordinate of at least one target point in a coordinate system with the print head as the origin based on the first coordinate and the second coordinate when collecting the three-dimensional point cloud; determine the external parameter based on the fourth coordinate and the third coordinate of at least one target point in the camera coordinate system.

[0119] In some embodiments of the present application, the processing unit 2020 is configured to calculate the translation vector according to the following formula: W ,Y W ,Z W )-(X C ,Y C ,Z C )-(P - X W ,P - Y W ,P - Z W ), where T is the translation vector, (X W ,Y W ,Z W ) is the first coordinate, (X C ,Y C ,ZC ) is the third coordinate of at least one target point in the camera coordinate system, (P - X W ,P - Y W ,P - Z W ) is the second coordinate of the print head when collecting 3D point cloud.

[0120] In some embodiments of the present application, the external parameters also include a rotation matrix between the camera coordinate system and the world coordinate system.

[0121] In some embodiments of the present application, the processing unit 2020 is configured to determine the initial value of the rotation matrix and the initial value of the translation vector using the following formula: Wi ,Y Wi ,Z Wi )=R c (X Ci ,Y Ci ,Z Ci )+T c +(P - X Wi ,P - Y Wi ,P - Z Wi ), where (X Wi ,Y Wi ,Z Wi ) is the first coordinate of the i-th target point in the target point, (X Ci ,Y Ci ,Z Ci ) is the third coordinate of the i-th target point in the camera coordinate system, (P - X Wi ,P - Y Wi ,P - Z Wi ) is the second coordinate of the print head when collecting the three-dimensional point cloud including the i-th target point, R c is the initial value of the rotation matrix, T c is the initial value of the translation vector; using the first coordinate of the target point, the second coordinate of the print head when collecting the three-dimensional point cloud including the target point, the third coordinate of the target point in the camera coordinate system and the loss function, the initial value of the translation vector and the initial value of the rotation matrix are updated to obtain the translation vector and the rotation matrix.

[0122] In some embodiments of the present application, the acquisition unit 2010 is used to obtain a first offset ΔZ between at least one target point and the optical center of the camera in the Z axis of the world coordinate system when capturing a two-dimensional image, where the optical center of the camera is the optical center of the camera corresponding to the image capture device, and at least one target point is located on the surface of the calibration figure on one side of the image capture device; obtain the focal length of the image capture device when capturing the two-dimensional image; the processing unit 2020 is used to: determine a second offset (ΔX, ΔY) between at least one target point and the optical center of the camera in the X axis and Y axis of the world coordinate system when capturing the two-dimensional image based on the third coordinate of the at least one target point in the image coordinate system, the first offset and the focal length; determine a fourth coordinate of the optical center of the camera in the world coordinate system when capturing the two-dimensional image based on the first offset, the second offset and the first coordinate; and determine a translation vector based on the second coordinate and the fourth coordinate.

[0123] In some embodiments of the present application, the processing unit 2020 is used to: determine a third offset (Δx, Δy) between at least one target point and the principal point of the two-dimensional image on each coordinate axis of the image coordinate system based on the third coordinate; and determine a second offset based on the first offset, the third offset and the focal length.

[0124] In some embodiments of the present application, the processing unit 2020 is configured to calculate the second offset according to the following formula: Where ΔX m , ΔY m , ΔZ m are the offsets of the camera optical center and the mth target point in the world coordinate system on the X-axis, Y-axis, and Z-axis when acquiring a two-dimensional image including the mth target point, respectively. m is the focal length of the image acquisition device when acquiring a two-dimensional image including the mth target point, Δx m , Δy m are respectively the offsets between the principal point of the two-dimensional image including the m-th target point and the m-th target point in the image coordinate system on the x-axis and the y-axis, where the m-th target point is one of the at least one target point.

[0125] In some embodiments of the present application, the processing unit 2020 is further used to: control the print head to print the calibration pattern on the hot bed; and control the image acquisition device to acquire a two-dimensional image or a three-dimensional point cloud.

[0126] In some embodiments of the present application, the processing unit 2020 is further configured to control the movement of the print head to drive the image acquisition device to move, thereby acquiring a two-dimensional image or a three-dimensional point cloud.

[0127] Figure 10 shows a schematic block diagram of a 3D printer 5000 provided in an embodiment of the present application. The 3D printer 5000 includes a heated bed 5100, a print head 5200, an image acquisition device 5300, and a main control chip 5400. The print head 5200 is configured to print a calibration pattern on the heated bed 5100; the image acquisition device 5300 is configured to acquire a two-dimensional image or a three-dimensional point cloud including target points of the calibration pattern, and the image acquisition device 5300 is mounted on the print head; and the main control chip 5400 is configured to control the print head and the image acquisition device. The main control chip 5400 is further configured to acquire the two-dimensional image or three-dimensional point cloud captured by the image acquisition device and process the two-dimensional image or three-dimensional point cloud according to the calibration method provided in one or more embodiments of the present application to obtain external parameters of the image acquisition device 5300 and the print head 5200.

[0128] In one embodiment, the main control chip 5400 includes a memory and a processor, wherein the memory is used to store computer programs; the processor is used to call and run the computer programs stored in the memory to execute the various steps of the calibration method of the embodiment of the present application.

[0129] An embodiment of the present application further provides a computer-readable medium storing a program code for execution by a device, wherein the program code includes instructions for executing the image acquisition device calibration method in an embodiment of the present application.

[0130] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface and executes the image acquisition device calibration method in the embodiment of the present application.

[0131] Optionally, as an implementation method, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the image acquisition device calibration method in the embodiment of the present application.

[0132] It should be noted that although the above-mentioned device or system only shows some components such as a processor and a memory, in the specific implementation process, those skilled in the art should understand that the above-mentioned device or system may also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the above-mentioned device or system may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the above-mentioned device or system may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include all the devices shown in Figures 9 to 10.

[0133] It should be understood that the processor in any of the above embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0134] In addition, the memory in any of the above embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM). By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0135] It should be understood that the formulas in the embodiments of the present application are merely examples and do not limit the scope of the embodiments of the present application. The formulas can be modified, and these modifications should also fall within the scope of protection of the present application.

[0136] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0137] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of the present application are not limited to this.

[0138] Unless otherwise indicated, all technical and scientific terms used in the embodiments of this application have the same meaning as those commonly understood by those skilled in the art in the art of this application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application. The term "and / or" used in this application includes any and all combinations of one or more related listed items.

[0139] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for calibrating an image acquisition device for a 3D printer. It is characterized in that The image acquisition device is fixed on the print head of the 3D printer, and the method comprises: Acquire a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of the at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in a calibration figure printed by the print head; According to the first coordinate, the second coordinate and the third coordinate, an external parameter of the image acquisition device is determined to calibrate the image acquisition device, wherein the external parameter includes a translation vector between the image acquisition device and the print head.

2. The method according to claim 1, It is characterized in that Determining the external parameters of the image acquisition device according to the first coordinate, the second coordinate, and the third coordinate includes: Determine, according to the first coordinate and the second coordinate when the three-dimensional point cloud is collected, a fourth coordinate of the at least one target point in a coordinate system with the print head as the origin; The extrinsic parameter is determined according to the fourth coordinate and the third coordinate of the at least one target point in the camera coordinate system.

3. The method according to claim 1 or 2, It is characterized in that Determining the external parameters of the image acquisition device according to the first coordinate, the second coordinate and the third coordinate includes: T = (X W ,Y W ,Z W )-(X C ,Y C ,Z C )-(P - X W ,P - Y W ,P - Z W ), Where T is the translation vector, (X W ,Y W ,Z W ) is the first coordinate, (X C ,Y C ,Z C ) is the third coordinate of the at least one target point in the camera coordinate system, (P - X W ,P - Y W ,P - Z W ) is the second coordinate of the printing head when collecting the three-dimensional point cloud, and the at least one target point includes a target point.

4. The method according to claim 1 or 2, It is characterized in that The external parameters also include a rotation matrix between the camera coordinate system and the world coordinate system.

5. The method according to claim 4, It is characterized in that Determining the external parameters of the image acquisition device according to the first coordinate, the second coordinate and the third coordinate includes determining the rotation matrix and the translation vector using the following formula: Wi ,Y Wi ,Z Wi )=R c (X Ci ,Y Ci ,Z Ci )+T c +(P - X Wi ,P - Y Wi ,P - Z Wi ), Among them, (X Wi ,Y Wi ,Z Wi ) is the first coordinate of the i-th target point, (X Ci ,Y Ci ,Z Ci ) is the third coordinate of the i-th target point in the camera coordinate system, (P - X Wi ,P - Y Wi ,P - Z Wi ) is the second coordinate of the printing head when collecting the three-dimensional point cloud including the i-th target point, R c is the rotation matrix, T c is the translation vector.

6. The method according to claim 1, It is characterized in that Before determining the external parameters of the image acquisition device according to the first coordinate, the second coordinate and the third coordinate, the method further includes: Acquire a first offset ΔZ between the at least one target point and the optical center of the camera on the Z axis of the world coordinate system when acquiring the two-dimensional image, wherein the optical center of the camera is the optical center of the camera corresponding to the image acquisition device, and the at least one target point is located on the surface of the calibration figure on one side of the image acquisition device; Acquire the focal length of the image acquisition device when acquiring the two-dimensional image; Determining the external parameters of the image acquisition device according to the first coordinate, the second coordinate, and the third coordinate includes: Determine, according to the third coordinate of the at least one target point in the image coordinate system, the first offset and the focal length, a second offset (ΔX, ΔY) between the at least one target point and the optical center of the camera in the X-axis and Y-axis of the world coordinate system when acquiring the two-dimensional image; Determine, according to the first offset, the second offset and the first coordinate, a fourth coordinate of the optical center of the camera in the world coordinate system when acquiring the two-dimensional image; The translation vector is determined according to the second coordinate and the fourth coordinate.

7. The method according to claim 6, It is characterized in that The determining, according to the third coordinate of the at least one target point in the image coordinate system, the first offset and the focal length, a second offset between the at least one target point and the optical center of the camera in the X-axis and the Y-axis of the world coordinate system when acquiring the two-dimensional image comprises: Determine, according to the third coordinate, a third offset (Δx, Δy) between the at least one target point and the principal point of the two-dimensional image on each coordinate axis of the image coordinate system; The second offset is determined according to the first offset, the third offset, and the focal length.

8. The method according to claim 7, It is characterized in that The determining the second offset according to the first offset, the third offset and the focal length includes: Where ΔX m , ΔY m , ΔZ m are respectively the offsets of the optical center of the camera and the m-th target point on the X-axis, Y-axis and Z-axis of the world coordinate system when acquiring a two-dimensional image including the m-th target point, m is the focal length of the image acquisition device when acquiring a two-dimensional image including the mth target point, Δx m , Δy m They are respectively the offsets of the principal point of the two-dimensional image including the m-th target point and the m-th target point on the x-axis and y-axis of the image coordinate system, and the m-th target point is one of the at least one target point.

9. The method according to any one of claims 1 to 8, It is characterized in that Before obtaining a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of the at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the three-dimensional point cloud, the method further includes: Control the print head to print the calibration pattern on the hot bed of the 3D printer; Control the image acquisition device to acquire the two-dimensional image or the three-dimensional point cloud.

10. An image acquisition device calibration device for a 3D printer, It is characterized in that The image acquisition device is fixed on the print head of the 3D printer, and the device includes: an acquisition unit, configured to acquire a first coordinate of at least one target point in a world coordinate system, a second coordinate of the print head in the world coordinate system when the image acquisition device acquires a two-dimensional image or a three-dimensional point cloud including the at least one target point, and a third coordinate of the at least one target point in an image coordinate system corresponding to the two-dimensional image or a camera coordinate system corresponding to the three-dimensional point cloud, wherein the at least one target point is a point in a calibration figure printed by the print head; a processing unit, configured to determine an external parameter of the image acquisition device according to the first coordinate, the second coordinate, and the third coordinate, so as to calibrate the image acquisition device, wherein the external parameter includes The translation vector between the print head and the print head.

11. A 3D printer, It is characterized in that include: Hot bed; A print head, used for printing a calibration pattern on the hot bed; An image acquisition device, used for acquiring a two-dimensional image or a three-dimensional point cloud including target points of the calibration figure, wherein the image acquisition device is mounted on the print nozzle; and A main control chip is used to control the print head and the image acquisition device, and is also used to obtain the image acquired by the image acquisition device and process the two-dimensional image or three-dimensional point cloud according to the method described in any one of claims 1 to 9 to obtain external parameters between the image acquisition device and the print head.

12. A computer-readable storage medium, It is characterized in that Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 9.

Images