Cladding distance regulation and control method and system based on camera image estimation

Abstract

The invention relates to a cladding distance regulation and control method and system based on camera image estimation. The cladding distance regulation and control method comprises the following steps: S1, determining parameters of a paraxial camera; s2, the paraxial camera is installed on the side of the cladding head; s3, determining expected position information of an expected position of a molten pool center in the image based on the cladding head parameters; s4, based on the molten pool image collected by the paraxial camera in real time, the actual position of the center of the molten pool in the molten pool image is determined; s5, based on the expected position information and the actual position information of the molten pool center in the image, the cladding distance error of the expected position and the actual position is determined; and S6, on the basis of the cladding distance error, the rate variable quantity and the current feeding rate value are determined, and therefore the consistency of the actual cladding distance and the expected cladding distance value is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D printing, and in particular to a cladding distance control method based on camera image estimation and a system thereof. Background Art

[0002] 3D printing, also known as additive manufacturing (AM), is a technology that manufactures physical parts by adding materials layer by layer based on three-dimensional CAD data.

[0003] Compared with traditional manufacturing processes such as casting, hot extrusion, and hot rolling, additive manufacturing technology has its unique advantages. Additive manufacturing technology can flexibly and quickly build customized objects. It is customized, low-cost, and has short cycles. Therefore, it is often used in prototype design and small-batch customized manufacturing.

[0004] Among them, direct energy deposition is an additive manufacturing technology / technology that uses metal as the deposition material. This technology heats the substrate by focusing the energy source to form a molten pool in the deposition area, and directly feeds the metal material in the form of powder or filament into the high-temperature molten pool, so that the material melts and deposits at the molten pool position, accumulating layer by layer to finally form a printed finished part. Compared with other metal additive manufacturing technologies, this technology is more suitable for forming unsupported structures using multi-axis printing methods, or for surface strengthening and defect repair of complex curved surfaces.

[0005] The regulation of process parameters in direct energy deposition technology is crucial, and it directly affects the geometric accuracy and mechanical properties of the workpiece after manufacturing. Among them, the feed rate of the cladding head during deposition will have a great influence on the forming accuracy of the workpiece. By measuring the cladding distance during the additive process and appropriately adjusting the feed rate according to the cladding distance so that the cladding distance is always kept near the expected value, the geometric accuracy of the final workpiece can be guaranteed.

[0006] In the prior art, most of the deposited layer height is directly measured by profilometers, line scanning sensors and other equipment, and the feed rate is adjusted according to the difference between the deposited layer height and the expected value, so as to achieve precision control. However, since the deposited layer height is integrated to obtain the overall morphology of the workpiece, this method of adjusting the feed rate based on the deposited layer height cannot directly control the morphology accuracy.

[0007] In addition, there is also a solution that uses a rangefinder camera for precision control, in which the rangefinder camera is installed nearly perpendicular to the printing axis of the cladding head. The rangefinder camera estimates the height of the deposited layer by the size of the molten pool area in the collected molten pool image, and adjusts the feed rate according to the difference between the height of the deposited layer and the expected value. However, since the camera used in the existing solution is often installed so that the camera optical axis is nearly perpendicular to the printing axis of the cladding head, and the camera optical center position is near the cladding distance below the end face of the cladding head to ensure that the molten pool can fall within the camera's field of view. As a result, cameras installed in this way are prone to interference and collision with workpieces when used in multi-axis additive technology or when facing the printing, repair, and strengthening needs of workpieces with complex curved surfaces. Summary of the invention

[0008] Based on this, it is necessary to provide a cladding distance control method and system based on camera image estimation.

[0009] An embodiment of the present invention provides a cladding distance control method based on camera image estimation, comprising the following steps:

[0010] S1, calibrating the rangefinder camera and determining the rangefinder camera parameters, wherein the rangefinder camera parameters include the focal length and the coordinates of the optical center; calibrating the cladding head and determining the cladding head parameters, wherein the cladding head parameters are the cladding head powder coke;

[0011] S2, installing the rangefinder camera to the side of the cladding head, so that the rangefinder camera can collect the image of the molten pool during cladding, the optical axis of the rangefinder camera and the printing axis of the cladding head have a first angle, the value range of the first angle is (10-85) degrees, and the rangefinder camera and the cladding head are both installed on the displacement device;

[0012] S3, based on the cladding head parameters, determining the expected cladding distance, and based on the expected cladding distance, determining the expected position c of the molten pool center in the image e Expected location information;

[0013] S4, based on the molten pool image collected in real time by the rangefinder camera, inputting the molten pool detection algorithm, the molten pool detection algorithm outputs the molten pool contour, and based on the molten pool contour, determining the actual position c of the molten pool center in the molten pool image i The actual location information;

[0014] S5, based on the expected position information and actual position information of the molten pool center in the image obtained at the same time, call the position matching algorithm to determine the expected position c e and the actual position c i The cladding distance error Δx;

[0015] S6, based on cladding distance error Δx, determine the rate change and the current feed rate value, send instructions to the displacement device according to the rate change and the current feed rate value, and the displacement device adjusts the movement rate and moving direction in real time according to the instructions to ensure the consistency between the actual cladding distance and the expected cladding distance value.

[0016] Preferably, the expected position information includes the horizontal coordinate and the vertical coordinate of the expected molten pool center in the image coordinate system; the actual position information includes the horizontal coordinate and the vertical coordinate of the actual molten pool center in the image coordinate system.

[0017] Preferably, step S4 specifically includes the following steps:

[0018] S401, binarizing the acquired molten pool image to obtain a binary molten pool image;

[0019] S402, extracting a melt pool contour of the melt pool in the binary melt pool image by an operator, wherein the pixels in the binary melt pool image are expressed in image coordinates, and the pixels in the melt pool contour constitute a first sequence set of melt pool pixel coordinates;

[0020] S403, respectively accumulating the image coordinates in the first sequence set of the molten pool pixel point coordinates and taking the average value to obtain the average value coordinates, where the average value coordinates are the actual position of the molten pool center in the image, that is, the actual position information.

[0021] Preferably, in step S5, determining the cladding distance error specifically includes the following steps:

[0022] S501, obtaining a projection line equation of the printing axis of the cladding head through a calibration experiment, and then matching the actual position information with the projection line equation, thereby matching the projection line information of the printing axis of the cladding head, the expected position information, and the actual position information in a coordinate system of an image;

[0023] S502, calculating the expected position c of the distance from the center of the molten pool on the projection line in the image e The coordinates of the nearest point c m0 , that is, the coordinates of the first pixel point c m0 ; Calculate the actual position c of the distance from the center of the molten pool in the image on the projection line i The coordinates of the nearest point c m1 , that is, the coordinates of the second pixel point c m1 ;

[0024] S503, calculate the first pixel coordinate c m0 and the second pixel coordinate c m1 The coordinate distance Δz between the desired position and the actual position is determined based on the coordinate distance Δz.

[0025] Preferably, the pixel coordinates c of the projection line m It is expressed as follows:

[0026]

[0027] Among them, α t is the direction vector expressed in pixel coordinates, c n is a pixel expressed in pixel coordinates, bn is a constant, α t 、c n With b n All of them are obtained through fitting during the calibration experiment.

[0028] Preferably, the cladding distance error Δx is expressed by the following formula:

[0029]

[0030] Among them, α t is the direction vector expressed in pixel coordinates, b n is a constant, above α t With b n They are all obtained by fitting during the calibration process, Δz is the coordinate of the first pixel point c m0 and the second pixel coordinate c m1 The coordinate distance between .

[0031] Preferably, the position information of the molten pool contour is a set of pixel point position information of the molten pool contour in the image, that is, a pixel set.

[0032] Preferably, the pixel point position information in the pixel set includes pixel point position information at the boundary of the molten pool contour.

[0033] The present invention also provides a cladding distance control system based on camera image estimation, the system comprising:

[0034] Image acquisition unit: used for acquiring cladding images with molten pool;

[0035] Image processing unit: used for preprocessing the collected cladding image;

[0036] Judgment and decision-making unit: used to determine whether it is necessary to send instructions to the displacement unit according to the cladding distance error, so as to adjust the existing movement speed and movement direction of the robot or machine tool;

[0037] Displacement unit: used to control the real-time movement rate and direction of the cladding head, so that the actual cladding distance remains consistent with the expected cladding distance value.

[0038] Preferably, the image acquisition unit comprises a paraxial camera, the cladding head and the paraxial camera are both mounted on the displacement unit, and the displacement unit comprises a robot or a machine tool.

[0039] The present invention collects images through a rangefinder camera to directly estimate the cladding distance error corresponding to the expected molten pool center and the actual molten pool center during the additive process, ensuring a fully closed-loop control of the morphology accuracy of the formed workpiece and ensuring the consistency between the workpiece design drawing and the 3D printed product. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and other purposes, features and advantages of the present invention will become more clear by more specific description of the preferred embodiments of the present invention shown in the accompanying drawings. The same reference numerals indicate the same parts in all the accompanying drawings, and the accompanying drawings are not deliberately scaled to the actual size, and the focus is on illustrating the main purpose of the present invention.

[0041] Figure 1 Schematic diagram of the positional relationship between the rangefinder camera and the cladding head in a preferred embodiment of the present invention;

[0042] Figure 2 The molten pool image after binarization is collected by the rangefinder camera of the preferred embodiment of the present invention;

[0043] Figure 3 for Figure 2 A molten pool image in which the molten pool outline is identified;

[0044] Figure 4 A molten pool image of a preferred embodiment of the present invention is obtained by fitting the projection line of the printing axis in the image;

[0045] Figure 5 The present invention is a flowchart of a cladding distance control method according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0047] like Figure 1-Figure 5 As shown, an embodiment of the present invention provides a cladding distance control method based on camera image estimation, comprising the following steps:

[0048] S1. Calibrate the rangefinder camera and determine the parameters of the rangefinder camera, wherein the parameters of the rangefinder camera include the focal length and the coordinates of the optical center; calibrate the cladding head and determine the parameters of the cladding head, wherein the cladding head parameters are the cladding head powder coke; specifically, determine the expected cladding distance according to the pre-printing process experiment, theoretical calculation or the cladding head parameters given by the manufacturer, wherein the cladding head powder coke refers to the distance between the actual intersection point of the powder sprayed by the cladding head and the bottom plane of the cladding head.

[0049] S2, installing the rangefinder camera to the side of the cladding head so that the rangefinder camera can capture the image of the molten pool during cladding, the optical axis of the rangefinder camera and the printing axis of the cladding head have a first angle, the value range of the first angle is (10-85) degrees, and the rangefinder camera and the cladding head are both installed on the displacement device; in this embodiment, the camera is installed as follows Figure 1 As shown, the camera forms a rigid connection structure with the cladding head through tools such as a clamp or a fixed bracket. The direction of the camera optical axis needs to ensure that the molten pool can fall within the camera field of view. During cladding, the distance between the molten pool and the optical center of the camera needs to ensure that the molten pool can be stably imaged in the image, and the area occupied by the molten pool in the image is greater than 16 pixels and less than 96% of the total number of pixels in the image.

[0050] S3, based on the cladding head parameters, determine the expected cladding distance, and based on the expected cladding distance, determine the expected position c of the molten pool center in the image. e The expected position information includes the horizontal coordinate and the vertical coordinate of the center of the expected molten pool in the image coordinate system;

[0051] S4, based on the molten pool image collected in real time by the rangefinder camera, inputting the molten pool detection algorithm, the molten pool detection algorithm outputs the molten pool contour, and based on the molten pool contour, determining the actual position c of the molten pool center in the molten pool image i The actual position information includes the abscissa and ordinate of the actual molten pool center in the image coordinate system. The cladding head and the substrate are placed at the desired cladding distance, and then the rangefinder camera captures the image, and the algorithm in step S4 is used to calculate the actual position of the molten pool center in the image, which is c e ;

[0052] In a preferred embodiment, step S4 specifically includes the following steps:

[0053] S401, binarizing the acquired molten pool image to obtain a binary molten pool image;

[0054] S402, extracting the molten pool contour of the molten pool from the binary molten pool image by using an operator, wherein the pixel points in the binary molten pool image are expressed in image coordinates (x i ,y i) is expressed as follows: the pixel points within the molten pool contour constitute a first sequence set Q1 of molten pool pixel point coordinates;

[0055] S403, the image coordinates (x i ,y i ) are accumulated and averaged to obtain the average coordinate (c x ,c y ):

[0056]

[0057] The above average coordinates are the actual position c of the center of the melt pool in the image i , that is, the actual location information, that is, the actual location coordinates (c x ,c y ).

[0058] In a preferred embodiment, the position information of the molten pool contour is a set of pixel position information of a circumscribed rectangular frame of the molten pool contour in the image, that is, a pixel set.

[0059] In a preferred embodiment, the pixel point position information in the pixel set includes the pixel point position information within the circumscribed rectangular frame of the molten pool contour.

[0060] S5, based on the expected position information and actual position information of the molten pool center in the image obtained at the same time, call the position matching algorithm to determine the expected position c e and the actual position c i The cladding distance error Δx;

[0061] In a preferred embodiment, in step S5, determining the cladding distance error Δx specifically includes the following steps:

[0062] S501, obtaining a projection line equation of the printing axis of the cladding head through a calibration experiment, and then matching the actual position information with the projection line equation, thereby matching the projection line information of the printing axis of the cladding head, the expected position information, and the actual position information in a coordinate system of an image;

[0063] Through experimental fitting Figure 4 The projection line of the cladding head printing axis in the image, the pixel coordinates of the projection line c m It is expressed as follows:

[0064]

[0065] In the above formula, α t is the direction vector expressed in pixel coordinates, c n is a pixel expressed in pixel coordinates, bn is a constant, α t 、c n With b n All of them are obtained through fitting during the calibration experiment.

[0066] S502, calculating the expected position c of the distance from the center of the molten pool on the projection line in the image e The coordinates of the nearest point c m0 , that is, the coordinates of the first pixel point c m0 ; Calculate the actual position c of the distance from the center of the molten pool in the image on the projection line i The coordinates of the nearest point c m1 , that is, the coordinates of the second pixel point c m1 ;

[0067] S503, calculate the first pixel coordinate c m0 and the second pixel coordinate c m1 The coordinate distance Δz between the desired position and the actual position is determined based on the coordinate distance Δz. The cladding distance Δx is expressed by the following formula:

[0068]

[0069] In the above formula, α t is the direction vector expressed in pixel coordinates, b n is a constant, above α t With b n They are all obtained by fitting during the calibration process, Δz is the coordinate of the first pixel point c m0 and the second pixel coordinate c m1 The coordinate distance between .

[0070] S6. Based on the cladding distance error Δx, the PID controller performs calculations to determine the rate change and the current feed rate value, and sends instructions to the displacement device according to the rate change and the current feed rate value. The displacement device adjusts the movement rate and movement direction in real time according to the instructions to ensure the consistency between the actual cladding distance and the expected cladding distance value.

[0071] The present invention uses a rangefinder camera to collect images to directly estimate the cladding distance error corresponding to the expected molten pool center and the actual molten pool center during the additive process, ensuring a fully closed-loop control of the morphology accuracy of the formed workpiece. In addition, the cladding distance measurement algorithm described in the present invention allows a certain angle between the camera optical axis and the cladding head printing axis, that is, there is a first angle between the camera optical axis of the rangefinder camera and the cladding head printing axis, and the value range of the first angle is (10-85) degrees, so as to avoid collision interference problems that are prone to occur in multi-axis additive processes.

[0072] The present invention uses a rangefinder camera to measure the cladding distance from the side, while most existing solutions require devices such as line scanning sensors or laser interferometers that can measure distances non-contactly. Other solutions require the purchase of expensive distance measuring equipment, and the information obtained by the measuring equipment is limited. The camera used in this solution is relatively low in cost, and the image information contains a large amount of information. In addition to measuring the cladding distance, it can also be used to predict and estimate many indicators related to process parameters.

[0073] The algorithm of the present invention allows the rangefinder camera to Figure 1 The method shown is to install the camera at an angle, rather than perpendicular to the printing axis. Compared with the method of installing the camera perpendicular to the printing axis, this installation method can reduce the interference of the rangefinder camera on the printing process and avoid interference between the camera and the workpiece.

[0074] In the present invention, the cladding distance error estimation algorithm is completed by using the image coordinates of the center point of the molten pool and the projection line of the printing axis on the camera image plane. Through the error between the actual measured image coordinates of the center point of the molten pool and the expected image coordinates, the method projects the error onto the projection line and calculates the cladding distance error based on the projection line equation. Compared with the existing method of directly estimating the true coordinates of the center point of the molten pool and then calculating the cladding distance error, the cladding distance calculated by the method proposed in the present invention is more accurate.

[0075] The present invention also provides a cladding distance control system based on camera image estimation, the system comprising:

[0076] Image acquisition unit: used for acquiring cladding images with molten pool;

[0077] Image processing unit: used for preprocessing the collected cladding image;

[0078] Judgment and decision-making unit: used to determine whether it is necessary to send instructions to the displacement unit according to the cladding distance error Δx, so as to adjust the existing movement rate and movement direction of the robot or machine tool;

[0079] Displacement unit: used to control the real-time movement rate and direction of the cladding head, so that the actual cladding distance remains consistent with the expected cladding distance value.

[0080] In a preferred embodiment, the image acquisition unit comprises a paraxial camera, the cladding head and the paraxial camera are both mounted on the displacement unit, and the displacement unit comprises a robot or a machine tool.

[0081] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A cladding distance control method based on camera image estimation, characterized in that: The steps include: S1, calibrating the rangefinder camera and determining the rangefinder camera parameters, wherein the rangefinder camera parameters include the focal length and the coordinates of the optical center; calibrating the cladding head and determining the cladding head parameters, wherein the cladding head parameters are the cladding head powder coke; S2, installing the rangefinder camera to the side of the cladding head, so that the rangefinder camera can collect the image of the molten pool during cladding, the optical axis of the rangefinder camera and the printing axis of the cladding head have a first angle, the value range of the first angle is (10-85) degrees, and the rangefinder camera and the cladding head are both installed on the displacement device; S3, based on the cladding head parameters, determining the expected cladding distance, and based on the expected cladding distance, determining the expected position c of the molten pool center in the image e Expected location information; S4, based on the molten pool image collected in real time by the rangefinder camera, inputting the molten pool detection algorithm, the molten pool detection algorithm outputs the molten pool contour, and based on the molten pool contour, determining the actual position c of the molten pool center in the molten pool image i The actual location information; S5, based on the expected position information and the actual position information of the molten pool center in the image obtained at the same time, calling the position matching algorithm to determine the expected position c e and the actual position c i Cladding distance error Δx ; S6, based on cladding distance error Δx , determine the rate change and the current feed rate value, send instructions to the displacement device according to the rate change and the current feed rate value, and the displacement device adjusts the movement rate and moving direction in real time according to the instructions to ensure the consistency between the actual cladding distance and the expected cladding distance value.

2. The control method according to claim 1, characterized in that: The expected position information includes the horizontal coordinate and the vertical coordinate of the expected molten pool center in the image coordinate system; the actual position information includes the horizontal coordinate and the vertical coordinate of the actual molten pool center in the image coordinate system.

3. The control method according to claim 1, characterized in that: Step S4 specifically includes the following steps: S401, binarizing the acquired molten pool image to obtain a binary molten pool image; S402, extracting a melt pool contour of the melt pool in the binary melt pool image by an operator, wherein the pixels in the binary melt pool image are expressed in image coordinates, and the pixels in the melt pool contour constitute a first sequence set of melt pool pixel coordinates; S403, respectively accumulating the image coordinates in the first sequence set of the molten pool pixel coordinates and taking the average value to obtain the average value coordinates, where the average value coordinates are the actual position of the molten pool center in the image.

4. The control method according to claim 1, characterized in that: In step S5, determining the cladding distance error specifically includes the following steps: S501, obtaining a projection line equation of the printing axis of the cladding head through a calibration experiment, and then matching the actual position information with the projection line equation, thereby matching the projection line information of the printing axis of the cladding head, the expected position information, and the actual position information in a coordinate system of an image; S502, calculating the expected position c of the distance from the center of the molten pool on the projection line in the image e The coordinates of the nearest point c m0 , that is, the coordinates of the first pixel point c m0 ; Calculate the actual position c of the distance from the center of the molten pool in the image on the projection line i The coordinates of the nearest point c m1 , that is, the coordinates of the second pixel point c m1 ; S503: Calculate the coordinates of the first pixel point c m0 and the second pixel coordinate c m1 The coordinate distance Δz between the desired position and the actual position is determined based on the coordinate distance Δz.

5. The control method according to claim 1, characterized in that: The pixel coordinates c of the projection line m It is expressed as the following formula: Among them, α t is the direction vector expressed in pixel coordinates, c n is a pixel expressed in pixel coordinates, bn is a constant, α t 、c n With b n All of them are obtained through fitting during the calibration experiment.

6. The control method according to claim 1, characterized in that: The cladding distance error Δx is expressed by the following formula: Among them, α t is the direction vector expressed in pixel coordinates, b n is a constant, above α t With b n They are all obtained by fitting during the calibration process, Δz is the coordinate of the first pixel point c m0 and the second pixel coordinate c m1 The coordinate distance between .

7. The control method according to claim 1, characterized in that: The position information of the molten pool contour is a set of pixel position information of the molten pool contour in the image, that is, a pixel set.

8. The control method according to claim 7, characterized in that: The pixel point position information in the pixel set includes the pixel point position information at the boundary of the molten pool contour.

9. A cladding distance control system based on camera image estimation, characterized in that: The system comprises: Image acquisition unit: used for acquiring cladding images with molten pool; Image processing unit: used for preprocessing the collected cladding image; Judgment and decision-making unit: used to determine whether it is necessary to send instructions to the displacement unit according to the cladding distance error, so as to adjust the existing movement speed and movement direction of the robot or machine tool; Displacement unit: used to control the real-time movement rate and direction of the cladding head, so that the actual cladding distance remains consistent with the expected cladding distance value.

10. The system according to claim 9, characterized in that The image acquisition unit includes a paraxial camera, the cladding head and the paraxial camera are both mounted on the displacement unit, and the displacement unit includes a robot or a machine tool.

Images