Workpiece cutting track determination method, device and computer equipment

Abstract

The application relates to a workpiece cutting track determination method, device and computer equipment. The method comprises the following steps: acquiring a nesting drawing of a workpiece to be cut, and determining a plurality of initial positioning points corresponding to the nesting drawing; collecting image data of the workpiece to be cut through a structured light sensor; the image data comprises an ordered point cloud; each initial positioning point is converted into the ordered point cloud to obtain a plurality of candidate positioning points, and a mechanical arm end pose corresponding to each candidate positioning point is determined; a line laser sensor is triggered based on the mechanical arm end pose to determine a target positioning point associated with the line laser sensor; and a target track is obtained by performing track planning on the mechanical arm end according to the target positioning point. The method can accurately obtain a high-precision groove cutting track.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic control, in particular to a workpiece cutting trajectory determination method and device and computer equipment. BACKGROUND

[0002] When using a 3D vision guided robot system to cut a groove, a manual teaching method is usually adopted, that is, the workpiece to be cut is first fixed, then the robot moving trajectory is planned by manual, and finally saved and reused. This method is simple in logic and easy to master, but requires high fixing accuracy of the workpiece, and the manual trajectory planning is tedious and time-consuming, which requires workers to have high proficiency. In addition, once the relative position of the workpiece to the robot base or the model of the workpiece changes, the trajectory needs to be re-planned, that is, the degree of automation is low. Therefore, how to accurately obtain a high-precision groove cutting trajectory is a problem to be solved at present. SUMMARY

[0003] Based on this, the present application aims to provide a method, device and computer equipment for accurately generating a workpiece cutting trajectory to solve the technical problems mentioned in the background.

[0004] In a first aspect, the present application provides a workpiece cutting trajectory determination method, comprising:

[0005] obtaining a nesting drawing of a workpiece to be cut, and determining a plurality of initial positioning points corresponding to the nesting drawing;

[0006] collecting image data of the workpiece to be cut through a structured light sensor; the image data includes ordered point clouds;

[0007] converting each of the initial positioning points into the ordered point clouds to obtain a plurality of candidate positioning points, and determining a respective corresponding robot end pose of each of the candidate positioning points;

[0008] triggering a line laser sensor based on the robot end pose to determine a target positioning point associated with the line laser sensor;

[0009] performing trajectory planning on the robot end based on the target positioning point to obtain a target trajectory.

[0010] In one embodiment, determining a plurality of initial positioning points corresponding to the nesting drawing comprises: extracting geometric information of a line segment or an arc segment in the nesting drawing, and converting the geometric information into a training drawing; the training drawing represents a binary drawing with a resolution of a preset value; extracting a plurality of initial positioning points from the training drawing; the initial positioning points represent points composed of non-end points in the line segment or the arc segment.

[0011] In one embodiment, the image data further comprises a grayscale image; after the image data of the workpiece to be cut is collected by the structured light sensor, the grayscale image is further subjected to a de-distortion process to obtain a de-distorted grayscale image; a homogeneous transformation matrix is obtained, and the ordered point cloud is corrected by the homogeneous transformation matrix to obtain a corrected ordered point cloud; the de-distorted grayscale image is resampled according to the corrected ordered point cloud to obtain a resampled grayscale image; and the resampled grayscale image is matched with a training image associated with the nesting drawing to obtain a coordinate conversion matrix.

[0012] In one embodiment, resampling the de-distorted grayscale image according to the corrected ordered point cloud to obtain a resampled grayscale image comprises: obtaining a blank image with a preset resolution; the blank image comprises a plurality of pixel points; each pixel point is converted onto the corrected ordered point cloud to obtain a plurality of first point clouds; each first point cloud is converted onto the ordered point cloud before correction based on the homogeneous transformation matrix to obtain a plurality of second point clouds; and each second point cloud is mapped onto the de-distorted grayscale image based on a preset imaging principle to obtain a resampled grayscale image.

[0013] In one embodiment, converting each initial positioning point into the ordered point cloud to obtain a plurality of candidate positioning points comprises: converting each initial positioning point into a de-distorted grayscale image based on a coordinate conversion matrix to obtain a plurality of grayscale image positioning points; and converting each grayscale image positioning point into the ordered point cloud based on a preset conversion algorithm to obtain a plurality of candidate positioning points.

[0014] In one embodiment, determining the respective mechanical arm end pose corresponding to each candidate positioning point comprises: obtaining a first hand-eye calibration matrix and a TCP calibration matrix; the first hand-eye calibration matrix is used to represent the relationship between the structured light sensor and the mechanical arm; and the TCP calibration matrix is used to represent the relationship between the line laser sensor and the mechanical arm; each candidate positioning point is converted into the coordinate system of the mechanical arm according to the first hand-eye calibration matrix to obtain a plurality of first positioning points; the scanning pose of the line laser sensor when scanning at each first positioning point is determined, and each scanning pose is converted into a mechanical arm end pose by the TCP calibration matrix.

[0015] In one embodiment, triggering the line laser sensor based on the end-of-arm pose to determine the target positioning points associated with the line laser sensor comprises: triggering the line laser sensor based on the end-of-arm pose to collect point cloud data of the workpiece to be cut; preprocessing the point cloud data collected by the line laser sensor to obtain a plurality of second positioning points; obtaining a second hand-eye calibration matrix; the second hand-eye calibration matrix is used to represent the relationship between the line laser sensor and the robot; and converting each of the second positioning points to the coordinate system of the robot according to the second hand-eye calibration matrix to obtain a plurality of target positioning points.

[0016] In one embodiment, trajectory planning for the end of the robot arm according to the target positioning points to obtain a target trajectory comprises: determining a parametric equation corresponding to a plurality of the target positioning points, and determining a plurality of target corner points in a training graph associated with the nesting graph according to the parametric equation; obtaining an ordered edge according to a plurality of the target corner points and the parametric equation, and performing trajectory planning on the ordered edge to obtain a target trajectory of the end of the robot arm.

[0017] In one embodiment, trajectory planning for the ordered edge to obtain a target trajectory of the end of the robot arm comprises: obtaining cutting information in the nesting graph and a TCP calibration matrix; the TCP calibration matrix is also used to represent the relationship between the cutting gun and the robot; fitting and sampling the ordered edge to obtain edge trajectory point coordinates, and determining a cutting gun pose of the cutting gun according to the edge trajectory point coordinates and the cutting information; and converting the cutting gun pose through the TCP calibration matrix to obtain a target trajectory of the end of the robot arm.

[0018] In a second aspect, the present application also provides a workpiece cutting trajectory determination device. The device comprises:

[0019] An image data acquisition module is configured to obtain a nesting graph of a workpiece to be cut, and determine a plurality of initial positioning points corresponding to the nesting graph; and collect image data of the workpiece to be cut through a structured light sensor; the image data comprises ordered point cloud.

[0020] An end-of-arm pose determination module is configured to convert each of the initial positioning points to the ordered point cloud to obtain a plurality of candidate positioning points, and determine a respective end-of-arm pose of each of the candidate positioning points.

[0021] A target positioning point determination module is configured to trigger a line laser sensor based on the end-of-arm pose to determine target positioning points associated with the line laser sensor; and perform trajectory planning for the end of the robot arm according to the target positioning points to obtain a target trajectory.

[0022] In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

[0023] Obtaining a nesting drawing of the workpiece to be cut, and determining a plurality of initial positioning points corresponding to the nesting drawing;

[0024] Collecting image data of the workpiece to be cut through a structured light sensor. The image data comprises an ordered point cloud;

[0025] Converting each of the initial positioning points into the ordered point cloud to obtain a plurality of candidate positioning points, and determining a respective end-of-arm pose of each of the candidate positioning points;

[0026] Triggering a line laser sensor based on the end-of-arm pose to determine a target positioning point associated with the line laser sensor;

[0027] Performing trajectory planning on the end of the arm according to the target positioning point to obtain a target trajectory.

[0028] The workpiece cutting trajectory determination method, device, and computer device described above. By obtaining a nesting drawing of the workpiece to be cut, and determining a plurality of initial positioning points corresponding to the nesting drawing, the structured light sensor can be used to collect image data of the workpiece to be cut, and each of the initial positioning points can be converted into an ordered point cloud in the image data to obtain a plurality of candidate positioning points. By determining a respective end-of-arm pose of each of the candidate positioning points, the line laser sensor can be triggered based on the end-of-arm pose to determine a target positioning point associated with the line laser sensor. In this way, the end of the arm can be subjected to trajectory planning according to the target positioning point to obtain a target trajectory for cutting the workpiece to be cut. Since the target trajectory can be determined directly based on the nesting drawing, the need for trajectory re-planning for the workpiece with changed positions in traditional manual teaching is avoided, thereby improving the automation degree and efficiency of cutting trajectory determination. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 An application environment diagram of the workpiece cutting trajectory determination method in one embodiment;

[0030] Figure 2 A flowchart of the workpiece cutting trajectory determination method in one embodiment;

[0031] Figure 3 A structural diagram of an abstract model of the arm in one embodiment;

[0032] Figure 4 A diagram for cutting the workpiece to be cut in one embodiment;

[0033] Figure 5 An effect diagram of point cloud data collected by a line laser sensor in an embodiment;

[0034] Figure 6 A flowchart of pre-processing of image data in an embodiment;

[0035] Figure 7 A structural diagram of determining a trajectory of a cutting gun in an embodiment;

[0036] Figure 8 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0037] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0038] The workpiece cutting trajectory determination method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . The terminal 102 communicates with the server 104 through a network. The terminal 102 can be a mechanical arm system for cutting various workpieces. The server 104 is used to control the operation of the mechanical arm system. The data storage system can store the data required to be processed by the server 104, and can be integrated on the server 104, or placed on the cloud or other network servers. When the server 104 obtains a nesting drawing of a workpiece to be cut, a plurality of initial positioning points corresponding to the nesting drawing are determined; an ordered point cloud in image data of the workpiece to be cut is collected through a structured light sensor; each initial positioning point is converted into the ordered point cloud to obtain a plurality of candidate positioning points, and a respective mechanical arm end pose corresponding to each candidate positioning point is determined. The server 104 is also used to trigger a line laser sensor based on the mechanical arm end pose, to determine a target positioning point associated with the line laser sensor, and to perform trajectory planning on the mechanical arm end according to the target positioning point, to obtain a target trajectory, so that the terminal 102 cuts the workpiece to be cut according to the target trajectory. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0039] In an embodiment, as shown in Figure 2 , a workpiece cutting trajectory determination method is provided, which is applied to a server in Figure 1 , and includes the following steps:

[0040] Step 202, obtaining a nesting drawing of a workpiece to be cut, and determining a plurality of initial positioning points corresponding to the nesting drawing.

[0041] Specifically, asFigure 3 As shown, Figure 3 A structural diagram of a mechanical arm system is provided, which includes a mechanical arm 1, a line laser sensor 2, a cutting gun 3, a structured light sensor 4, and a structured light sensor platform 5. When it is required to control the mechanical arm system to perform a running operation of workpiece cutting, the server first performs a preprocessing operation, which at least includes calibrating the mechanical arm system and parsing a nesting drawing of the workpiece to be cut. Therefore, the server obtains the nesting drawing of the workpiece to be cut collected in advance from a data storage system, and parses the nesting drawing to obtain geometric information and cutting information in the nesting drawing. Then, the server determines a plurality of initial positioning points of the workpiece to be cut according to the geometric information.

[0042] The geometric information of the workpiece to be cut is used to determine a training drawing and a positioning point. A schematic diagram of cutting the workpiece to be cut is shown in Figure 4 The single-line segment or arc segment cutting information at least includes a steel plate thickness D, a remaining thickness d, a cutting angle θ, and whether a line segment or an arc segment is to be cut or not.

[0043] In one embodiment, determining a plurality of initial positioning points corresponding to the nesting drawing includes: extracting geometric information of a line segment or an arc segment in the nesting drawing, and converting the geometric information into a training drawing; and extracting a plurality of initial positioning points from the training drawing.

[0044] The training drawing represents a binary drawing with a preset resolution, for example, 1 mm; and the initial positioning point represents a point composed of a non-end point in the line segment or the arc segment.

[0045] Specifically, the server converts the geometric information of the line segment or the arc segment in the nesting drawing into a binary drawing with a preset resolution, which is referred to as a training drawing G train , and obtains a positioning point coordinate in the binary drawing, that is, an initial positioning point. The training drawing also includes a corner point, which has the same purpose as the positioning point, that is, to accurately determine the line segment or the arc segment. However, the corner point is difficult to accurately determine in actual production, and therefore a method of obtaining an intersection point by fitting a line segment or an arc segment is used subsequently.

[0046] In step 204, image data of the workpiece to be cut is collected by the structured light sensor.

[0047] The image data includes a grayscale image and an ordered point cloud.

[0048] Specifically, after completing the preprocessing operation, the server can control the structured light sensor in the robot arm system to collect data of the workpiece to be cut in response to a data collection trigger operation of the user, and obtain image data of the workpiece to be cut. Then, the coarse positioning operation of the robot arm system can be realized through the image data, that is, the coarse positioning operation is realized based on the structured light sensor. The coarse positioning operation at least includes filtering processing, distortion removal processing, correction processing, resampling processing and matching processing of the image data.

[0049] In one embodiment, the filtering processing of the image data includes: after the robot arm system determines that the placement position of the workpiece to be cut has a dynamic range, the server can obtain an ROI region (Region of Interest) from the dynamic range, and then filter out part of the noise and non-target region through the ROI region. Therefore, the server first obtains a mask image according to the set ROI region, and then performs a logical AND operation on the mask image and the gray image, so that the values in the ROI region of the gray image remain unchanged, and the gray values outside the ROI region are 0, thereby achieving the filtering effect. Since the ordered point cloud and the gray image correspond to each other, according to the correspondence between the two, the noise points and non-target points in the ordered point cloud can be determined and set as invalid points (0, 0, 0).

[0050] In one embodiment, the coarse positioning based on the structured light sensor can be replaced by mechanical positioning. The placement of the workpiece has a certain range, and the coarse positioning of the workpiece can be realized by scanning the corner points and the two edges adjacent thereto.

[0051] Step 206: converting each initial positioning point into an ordered point cloud to obtain a plurality of candidate positioning points, and determining a respective corresponding robot arm end pose of each candidate positioning point.

[0052] Specifically, the server obtains a coordinate conversion matrix, and converts each initial positioning point into an ordered point cloud to obtain a plurality of candidate positioning points based on the coordinate conversion matrix and a preset conversion algorithm; the coordinate conversion matrix is used to realize the conversion between the training image and the resampled gray image. Then, the server obtains a hand-eye calibration matrix and a TCP calibration matrix, and determines a respective corresponding robot arm end pose of each candidate positioning point according to the hand-eye calibration matrix and the TCP calibration matrix.

[0053] In one embodiment, converting each initial positioning point into an ordered point cloud to obtain a plurality of candidate positioning points includes: converting each initial positioning point into a de-distorted gray image based on a coordinate conversion matrix to obtain a plurality of gray image positioning points; and converting each gray image positioning point into an ordered point cloud based on a preset conversion algorithm to obtain a plurality of candidate positioning points.

[0054] The preset conversion algorithm includes nearest neighbor interpolation, bilinear convolution interpolation, etc.

[0055] Specifically, the server converts each initial positioning point to the coordinate system of the robot arm based on the coordinate conversion matrix T g , to obtain a plurality of gray image positioning points P l2g . Then, the server converts each gray image positioning point to the uncorrected ordered point cloud, i.e., in the coordinate system of the structured light sensor, by using a preset conversion algorithm, to obtain a plurality of candidate positioning points P cs .

[0056] In one embodiment, determining the respective end-of-arm pose of each candidate positioning point comprises: obtaining a first hand-eye calibration matrix and a TCP calibration matrix; converting each candidate positioning point to the coordinate system of the robot arm based on the first hand-eye calibration matrix to obtain a plurality of first positioning points; and determining the scanning pose of the line laser sensor when scanning at each first positioning point, and converting each scanning pose to an end-of-arm pose by using the TCP calibration matrix.

[0057] The hand-eye calibration matrix includes the first hand-eye calibration matrix; the first hand-eye calibration matrix is used to represent the relationship between the structured light sensor and the robot arm; and the TCP calibration matrix is used to represent the relationship between the line laser sensor and the robot arm. It is easy to understand that the server has completed the determination of the hand-eye calibration matrix and the TCP calibration matrix in the process of performing the preprocessing operation.

[0058] Specifically, the server converts each candidate positioning point P cs to the coordinate system of the robot arm based on the first hand-eye calibration matrix to obtain a plurality of first positioning points P rs . Then, the server determines the pose when scanning at the first positioning point P rs , denoted as a scanning pose V rs , according to the designed scanning requirement of the line laser sensor. The server can convert the scanning pose V rs of the line laser sensor when scanning to an end-of-arm pose V rn according to the TCP calibration matrix.

[0059] Step 208: Triggering the line laser sensor based on the end-of-arm pose to determine the target positioning point associated with the line laser sensor.

[0060] Specifically, the server triggers the line laser sensor based on the end-of-arm pose after completing the coarse positioning operation of the robot arm system, and then implements the fine positioning operation of the robot arm system, i.e., implements the fine positioning operation based on the line laser sensor. The line laser sensor repeatedly acquires a single frame of data based on a preset time mode and the end-of-arm pose, to obtain a plurality of single frames of data, and then determines the target positioning point associated with the line laser sensor based on the plurality of single frames of data. Figure 5An effect picture of the point cloud data is shown. The server determines a plurality of target positioning points corresponding to the point cloud data according to the hand-eye calibration matrix.

[0061] In one embodiment, the linear laser sensor is triggered based on the end pose of the robot arm to determine the target positioning points associated with the linear laser sensor, including: triggering the linear laser sensor based on the end pose of the robot arm to collect point cloud data of the workpiece to be cut; preprocessing the point cloud data collected by the linear laser sensor to obtain a plurality of second positioning points; obtaining a second hand-eye calibration matrix; and converting each second positioning point to the coordinate system of the robot arm according to the second hand-eye calibration matrix to obtain a plurality of target positioning points.

[0062] The hand-eye calibration matrix includes the second hand-eye calibration matrix; and the second hand-eye calibration matrix is used to represent the relationship between the linear laser sensor and the robot arm.

[0063] Specifically, when the server collects the point cloud data of the workpiece to be cut, the point cloud data is preprocessed, that is, a plurality of second positioning points are obtained by denoising the point cloud data, fitting a plane, intersecting a line, and taking the middle point of the intersection line. The server converts each second positioning point to the coordinate system of the robot arm according to the second hand-eye calibration matrix to obtain a plurality of target positioning points.

[0064] Step 210, trajectory planning is performed on the end of the robot arm according to the target positioning points to obtain a target trajectory.

[0065] Specifically, after the server completes the fine positioning operation of the robot arm system, the trajectory planning of the end of the robot arm can be realized. The server determines a plurality of target corner points in the training diagram associated with the nesting diagram according to the target positioning points, and then constructs an ordered edge of the workpiece to be cut according to the plurality of target corner points. The server determines the edge trajectory point coordinates in the ordered edge, and determines the cutting gun pose of the cutting gun according to the edge trajectory point coordinates. The server converts the cutting gun pose through the TCP calibration matrix to obtain the target trajectory of the end of the robot arm. The TCP calibration matrix is also used to represent the relationship between the cutting gun and the robot arm.

[0066] In the above workpiece cutting trajectory determination method, the nesting drawing of the workpiece to be cut is obtained, and a plurality of initial positioning points corresponding to the nesting drawing are determined, so that the image data of the workpiece to be cut is collected by the structured light sensor, and each initial positioning point can be converted to an ordered point cloud in the image data to obtain a plurality of candidate positioning points. By determining the respective mechanical arm end pose corresponding to each candidate positioning point, the line laser sensor can be further triggered based on the mechanical arm end pose to determine the target positioning point associated with the line laser sensor. In this way, the target trajectory for cutting the workpiece to be cut can be obtained by trajectory planning of the mechanical arm end according to the target positioning point. Since the target trajectory can be determined directly based on the nesting drawing, the need for trajectory re-planning of the workpiece with changed position in traditional manual teaching is avoided, thereby improving the automation and efficiency of cutting trajectory determination.

[0067] In addition, since the rough positioning is first performed based on the structured light sensor and then the precise positioning is performed based on the line laser sensor, the problem of poor edge imaging quality caused by a single structured light sensor is avoided, and the error caused by the difference between the actual workpiece and the nesting drawing is more effectively avoided, thereby ensuring the accurate generation of the high-precision groove cutting trajectory.

[0068] In one embodiment, as shown in Figure 6 , after the image data of the workpiece to be cut is collected by the structured light sensor, it further includes:

[0069] Step 602, the gray image is deformed to obtain a deformed gray image.

[0070] Wherein, since the 2D gray image will have distortion when acquired, the radial distortion model and the tangential distortion model in the distortion model are as follows:

[0071] ;

[0072] Wherein, u, v are pixel coordinates in the deformed gray image; u', v' are pixel coordinates in the image before deforming; k1, k2, k3 are radial distortion parameters; r1, r2 are tangential distortion parameters. That is, the server can convert the gray image into the deformed gray image through the distortion model.

[0073] Step 604, obtain the homogeneous transformation matrix, and correct the ordered point cloud by the homogeneous transformation matrix to obtain the corrected ordered point cloud.

[0074] Specifically, when the server obtains the homogeneous transformation matrix T w , the ordered point cloud P before correction can be converted into the ordered point cloud P trans after correction in the manner of P .

[0075] In one embodiment, the homogeneous transformation matrix is ​​determined by the server performing Ransac plane fitting (Random Sample Consensus) on the ordered point cloud to obtain the plane equation of the ordered point cloud. The normal vector N corresponding to the plane equation w (a w b w c w The server is based on the origin (0, 0, 0) and the axis vector Z. a Construct an arbitrary line (0, 0, 1) that intersects the plane containing the equation of the plane, and obtain the intersection point P. lp (x0, y0, z0). The server is based on the normal vector N. w (a w b w c w ) and the intersection point P lp (x0, y0, z0), and construct the homogeneous transformation matrix T according to the Rodriguez rotation formula. w This ensures that the workpiece to be cut is horizontal and located in the z0 plane. The server, based on... Determine the axis vector Z a (0, 0, 1) and normal vector N w (a w b w c w The cosine of the included angle C angle The server will use the axis vector Z. a (0, 0, 1) and normal N w (a w b w c w Performing the cross product, we obtain the rotation axis vector as follows: The cross product matrix is ​​obtained as follows:

[0076] ;

[0077] Next, the server uses the Rodriguez rotation formula, combined with the cosine of the included angle and the cross product matrix, to obtain the final rotation matrix as follows: Therefore, based on the rotation matrix and the intersection coordinates, the server obtains the homogeneous transformation matrix as follows:

[0078] ;

[0079] Step 606: Resample the distortion-free grayscale image based on the corrected ordered point cloud to obtain a resampled grayscale image.

[0080] Specifically, the server pre-creates a blank image G of the same size as the grayscale image img , and sets the resolution to 1 mm. Furthermore, according to the pinhole imaging principle, the following resampling model can be obtained:

[0081] ;

[0082] wherein dx, dy and f are intrinsic parameters, and the two "1"s in the denominator represent a resolution of 1 mm. Based on the resampling model, the z corresponding to the image plane with a resolution of 1 mm can be obtained, and then according to the coordinates (x, y) on the image plane in the resampling model, the real coordinates (X, Y, Z) of the target point in the resampled grayscale image can be obtained.

[0083] In one embodiment, resampling the grayscale image after distortion correction based on the corrected ordered point cloud comprises: obtaining a blank image with a resolution of a preset value; converting each pixel point to the corrected ordered point cloud to obtain a plurality of first point clouds; converting each first point cloud to the ordered point cloud before correction based on the homogeneous transformation matrix to obtain a plurality of second point clouds; and mapping each second point cloud to the grayscale image after distortion correction based on a preset imaging principle to obtain the resampled grayscale image.

[0084] wherein the blank image comprises a plurality of pixel points; and the preset imaging principle comprises a pinhole imaging principle.

[0085] Specifically, for each pixel point p ixel on the blank image G img , the server converts it to the corrected ordered point cloud based on the above resampling model, denoted as the first point cloud P ci . The server then converts the first point cloud P ci to the ordered point cloud before correction based on the above homogeneous transformation matrix to obtain the second point cloud P i . The server then maps each second point cloud to the grayscale image after distortion correction based on the pinhole imaging principle. In summary, the correspondence between the blank image G img and the grayscale image after distortion correction is established, i.e., the resampled grayscale image G cali after correction and with a resolution of 1 mm is obtained.

[0086] Step 608: matching the resampled grayscale image with the training image associated with the nesting map to obtain a coordinate conversion matrix.

[0087] Specifically, after obtaining the resampled grayscale image G cali , the server can match it with the training image G trainThe matching is performed, such as the matching method of line-mod (line-Multimodal templates). The server first performs scaling and rotation training on the training graph G train to obtain sufficient feature information, and performs resampling on the gray image G cali to obtain the training graph G train and the coordinate conversion matrix T cali of the resampled gray image G g .

[0088] In the above embodiment, by performing the distortion filtering processing, correction processing, resampling processing and matching processing on the image data in the coarse positioning operation based on the structured light sensor, each initial positioning point can be accurately converted into an ordered point cloud according to the coordinate conversion matrix, and the respective corresponding end pose of the robot arm of each candidate positioning point is determined, thereby providing a basis for the subsequent precise positioning operation of the robot arm system and providing a basis for accurately obtaining the target trajectory.

[0089] In one embodiment, the target trajectory is obtained by performing trajectory planning on the robot arm end according to the target positioning point, including: determining the parameter equation corresponding to the plurality of target positioning points, and determining a plurality of target corner points in the training graph associated with the nesting drawing according to the parameter equation; obtaining an ordered edge according to the plurality of target corner points and the parameter equation, and performing trajectory planning on the ordered edge to obtain the target trajectory of the robot arm end.

[0090] Specifically, the server obtains the parameter equation corresponding to the target positioning point according to different forms of line segments or arc segments in the nesting drawing, and obtains the target corner point in the line segment or arc segment according to the intersection of the parameter equation. Based on the plurality of target corner points determined, the line segment or arc segment is re-joined and a new parameter equation is calculated, and then when sampling is performed according to the set order and density, the ordered edge corresponding to the new parameter equation is obtained. Wherein, the direction of each line segment and arc segment has been set in advance, that is, a serial number is set for each corner point to determine the order according to the serial number. For a line segment, the starting point and the ending point are known, a straight line is fitted and sampling is performed according to the set density. For an arc segment, the starting point, the intermediate point and the ending point are known, a circle is fitted and sampling is performed according to the set density.

[0091] Further, the target trajectory of the robot arm end is obtained by performing trajectory planning on the ordered edge, including: obtaining the cutting information in the nesting drawing and the TCP calibration matrix; fitting and sampling the ordered edge to obtain edge trajectory point coordinates, and determining the cutting gun pose of the cutting gun according to the edge trajectory point coordinates and the cutting information; and converting the cutting gun pose through the TCP calibration matrix to obtain the target trajectory of the robot arm end.

[0092] Specifically, the server can use NURBS (Non-Uniform Rational B-Splines) to fit and sample the ordered edges, obtaining the coordinates P of the edge trajectory points. edge The server then combines the cutting information of the workpiece to be cut, the workpiece plane normal, the perpendicular direction of the line segment or the radial direction of the arc, and determines the cutting gun pose corresponding to the coordinates of the edge trajectory points, i.e., the trajectory of the cutting gun. Next, the server converts the cutting gun pose into the pose of the robotic arm end effector, i.e., the target trajectory, based on the TCP calibration matrix.

[0093] In the above embodiments, compared to methods using only structured light sensors, there may be difficulties in accurately locating edges. By using a line laser sensor to actually acquire point cloud data of the workpiece to be cut to determine the ordered edges, and then determining the target trajectory of the robotic arm's end effector based on the ordered edges and cutting information in the nesting diagram, acquisition errors can be effectively reduced, and the accuracy of edge acquisition can be significantly improved. Especially for complex scenarios such as workpiece defects or slag buildup, the precision of bevel cutting can be greatly improved.

[0094] In one embodiment, such as Figure 7 As shown, Figure 7 A schematic diagram illustrating the structure for determining the trajectory of the cutting gun. The process of determining the trajectory of the cutting gun includes: knowing the coordinates P of the edge trajectory point. edge The centroid P of the upper surface of the workpiece center The normal vector V of the upper surface of the workpiece normal The normal vector of the line V line Workpiece cutting information: steel plate thickness D, remaining thickness d, and cutting angle θ. (via V) normal With V line Cross product yields V vertical Vector, and using P edge With P center The vector formed is redirected to point inward toward the workpiece. P edge Along V normal The movement in the opposite direction (D - d) occurs in V. normal With V vertical Within the plane formed, V normal Rotate counterclockwise by an angle θ to obtain the cutting gun pose V. cutter V cutter With V vertical The intersection point is the location of the cutting gun head.

[0095] It should be understood that although each step in the flowchart involved in the above embodiments is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0096] Based on the same inventive concept, the embodiments of the present application also provide a workpiece cutting trajectory determination device for implementing the workpiece cutting trajectory determination method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more workpiece cutting trajectory determination device embodiments provided below can refer to the limitations of the workpiece cutting trajectory determination method in the above text, which will not be repeated here.

[0097] In one embodiment, a workpiece cutting trajectory determination device is provided, comprising: an image data acquisition module, an end pose determination module, and a target positioning point determination module, wherein:

[0098] The image data acquisition module is configured to acquire a nesting drawing of a workpiece to be cut and determine a plurality of initial positioning points corresponding to the nesting drawing; and collect image data of the workpiece to be cut through a structured light sensor. The image data includes an ordered point cloud.

[0099] The end pose determination module is configured to convert each initial positioning point into the ordered point cloud to obtain a plurality of candidate positioning points, and determine a respective mechanical arm end pose corresponding to each candidate positioning point.

[0100] The target positioning point determination module is configured to trigger a line laser sensor based on the mechanical arm end pose to determine a target positioning point associated with the line laser sensor; and perform trajectory planning on the mechanical arm end based on the target positioning point to obtain a target trajectory.

[0101] Each module in the above workpiece cutting trajectory determination can be realized by software, hardware, and combinations thereof, in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0102] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (J / O), and a communication interface. The processor, memory, and J / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the J / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores joint characteristics. The J / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a workpiece cutting trajectory determination method.

[0103] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0104] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0105] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0106] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. The non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0108] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0109] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for determining a workpiece cutting trajectory, characterized in that, The method includes: Obtain the nesting diagram of the workpiece to be cut, and determine multiple initial positioning points corresponding to the nesting diagram; Image data of the workpiece to be cut is acquired using a structured light sensor; the image data includes an ordered point cloud. Each initial positioning point is converted into the ordered point cloud to obtain multiple candidate positioning points, and the end-effector pose corresponding to each candidate positioning point is determined. The line laser sensor is triggered based on the end-effector pose of the robotic arm to determine the target positioning point associated with the line laser sensor. The trajectory of the robotic arm end effector is planned based on the target positioning point to obtain the target trajectory. The determination of the multiple initial positioning points corresponding to the nesting diagram includes: Extract the geometric information of line segments or arc segments from the nesting diagram, and convert the geometric information into a training image; the training image represents a binary image with a preset resolution; Multiple initial positioning points are extracted from the training image; the initial positioning points represent the points formed by the non-endpoints of the line segment or the arc segment; Determining the end-effector pose corresponding to each of the candidate positioning points includes: Obtain the first hand-eye calibration matrix and the TCP calibration matrix; the first hand-eye calibration matrix is ​​used to characterize the relationship between the structured light sensor and the robotic arm; the TCP calibration matrix is ​​used to characterize the relationship between the line laser sensor and the robotic arm. Based on the first hand-eye calibration matrix, each candidate positioning point is transformed into the coordinate system of the robotic arm to obtain multiple first positioning points; The scanning pose of the line laser sensor at each first positioning point is determined, and each scanning pose is converted into the end effector pose of the robotic arm through the TCP calibration matrix. The step of triggering the line laser sensor based on the end-effector pose to determine the target positioning point associated with the line laser sensor includes: The line laser sensor is triggered based on the end-effector pose of the robotic arm to collect point cloud data of the workpiece to be cut. The point cloud data collected by the line laser sensor is preprocessed to obtain multiple second positioning points; Obtain the second hand-eye calibration matrix; the second hand-eye calibration matrix is ​​used to characterize the relationship between the line laser sensor and the robotic arm; Based on the second hand-eye calibration matrix, each of the second positioning points is transformed into the coordinate system of the robotic arm to obtain multiple target positioning points; The step of planning the trajectory of the robotic arm end effector based on the target positioning point to obtain the target trajectory includes: Determine the parametric equations corresponding to multiple target positioning points, and based on the parametric equations, determine multiple target corner points in the training diagram associated with the nesting diagram; Based on the multiple target corner points and the parametric equations, an ordered edge is obtained, and trajectory planning is performed on the ordered edge to obtain the target trajectory of the robotic arm end effector. The process of trajectory planning on the ordered edges to obtain the target trajectory of the robotic arm's end effector includes: Obtain the cutting information and TCP calibration matrix from the nesting diagram; the TCP calibration matrix is ​​also used to characterize the relationship between the cutting gun and the robotic arm; The ordered edges are fitted and sampled to obtain the coordinates of the edge trajectory points, and the cutting gun pose of the cutting gun is determined based on the coordinates of the edge trajectory points and the cutting information. The pose of the cutting gun is converted using the TCP calibration matrix to obtain the target trajectory at the end of the robotic arm.

2. The method according to claim 1, characterized in that, The image data also includes grayscale images; after acquiring the image data of the workpiece to be cut using a structured light sensor, it further includes: The grayscale image is subjected to distortion correction processing to obtain a distortion-corrected grayscale image; Obtain the homogeneous transformation matrix, and then use the homogeneous transformation matrix to perform correction processing on the ordered point cloud to obtain the corrected ordered point cloud; The distorted grayscale image is resampled based on the corrected ordered point cloud to obtain a resampled grayscale image. The resampled grayscale image is matched with the training image associated with the nesting image to obtain a coordinate transformation matrix.

3. The method according to claim 2, characterized in that, The step of resampling the distortion-free grayscale image based on the corrected ordered point cloud to obtain a resampled grayscale image includes: Obtain a blank image with a preset resolution; the blank image includes multiple pixels. Each pixel is converted onto the corrected ordered point cloud to obtain multiple first point clouds; Based on the homogeneous transformation matrix, each of the first point clouds is transformed onto the ordered point cloud before correction to obtain multiple second point clouds; Based on a preset imaging principle, each second point cloud is mapped onto the distortion-free grayscale image to obtain a resampled grayscale image.

4. The method according to any one of claims 1 to 3, characterized in that, The step of converting each initial localization point into the ordered point cloud to obtain multiple candidate localization points includes: Based on the coordinate transformation matrix, each initial positioning point is transformed into the distortion-free grayscale image to obtain multiple grayscale positioning points; Based on a preset conversion algorithm, each grayscale image location point is converted into the ordered point cloud to obtain multiple candidate location points.

5. A workpiece cutting trajectory determination device, characterized in that, The apparatus for implementing the method according to any one of claims 1 to 4 comprises: An image data acquisition module is used to acquire the nesting diagram of the workpiece to be cut and determine multiple initial positioning points corresponding to the nesting diagram; and to acquire image data of the workpiece to be cut through a structured light sensor; the image data includes an ordered point cloud. The end-effector pose determination module is used to convert each of the initial positioning points into the ordered point cloud to obtain multiple candidate positioning points, and to determine the end-effector pose of each candidate positioning point. The target positioning point determination module is used to trigger the line laser sensor based on the pose of the robotic arm end effector to determine the target positioning point associated with the line laser sensor; and to perform trajectory planning on the robotic arm end effector based on the target positioning point to obtain the target trajectory.

Images