Hot press workpiece edge detection and correction method based on image data analysis

Abstract

The invention discloses a hot press workpiece edge detection alignment method based on image data analysis, and relates to the technical field of image processing, and the method comprises the steps: collecting edge image data and point cloud data of a hot press workpiece, and carrying out the preprocessing; calculating the preprocessed edge image data gradient by using a Sobel operator to obtain an edge image point set of the hot press workpiece; analyzing geometric characteristics of the edge image point set through Hough transform and spline fitting to obtain a feature point set; analyzing the preprocessed point cloud data through normal vector change analysis and curvature calculation to obtain an edge point cloud of the hot press workpiece; the edge point cloud of the hot press workpiece is processed, and a datum line is obtained; and respectively utilizing the feature point set and the reference line to calculate the offset of the hot press workpiece, obtaining an image mode offset and a point cloud mode offset, and fusing the image mode offset and the point cloud mode offset to obtain a fused offset. According to the method, the image data and the point cloud data are combined, so that the edge detection and alignment precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting and correcting the edges of hot press workpieces based on image data analysis. Background Art

[0002] Traditional methods for detecting and correcting the edges of hot press workpieces mainly rely on image processing techniques. Usually, edge detection algorithms such as Canny edge detection are used, but there are some deficiencies in actual applications. First of all, when dealing with complex shapes and irregular edges, traditional methods are easily interfered by noise, resulting in inaccurate edge extraction. Even if denoising techniques such as Gaussian filtering are adopted, it is still difficult to effectively distinguish the details of the edges, especially in the case of blurred or occluded edges.

[0003] Traditional methods usually fail to make full use of the geometric information in the point cloud data, ignoring the normal vector and curvature change characteristics of the edge point cloud, resulting in insufficient accuracy of edge alignment. How to effectively combine image data and point cloud data to improve the accuracy of edge detection and alignment has become an important problem that traditional methods have not solved. Summary of the Invention

[0004] In view of the above existing problems, the present invention proposes...

[0005] Therefore, the present invention provides a method for detecting and aligning the edges of hot press workpieces based on image data analysis to solve the problem that traditional methods do not effectively combine image data and point cloud data and improve the accuracy of edge detection and alignment.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for detecting and correcting the edges of hot press workpieces based on image data analysis, which includes collecting edge image data and point cloud data of the hot press workpieces and performing preprocessing;

[0008] Calculating the gradient of the preprocessed edge image data by using the Sobel operator to obtain the edge image point set of the hot press workpieces;

[0009] Analyzing the geometric characteristics of the edge image point set through Hough transform and spline fitting to obtain a feature point set;

[0010] Analyzing the preprocessed point cloud data through normal vector change analysis and curvature calculation to obtain the edge point cloud of the hot press workpieces;

[0011] Adopting the least squares fitting method to process the edge point cloud of the hot press workpieces to obtain a reference line;

[0012] Calculate the offset of the hot press workpiece using the feature point set and the reference line respectively, obtain the image pattern offset and the point cloud pattern offset and fuse them to obtain the fused offset;

[0013] Use the fused offset and the robotic arm to align the edges of the hot press workpiece.

[0014] As a preferred solution of the method for edge detection and alignment of hot press workpieces based on image data analysis according to the present invention, wherein: the preprocessing includes denoising the edge image data through a Gaussian filter, graying and enhancing the contrast of the edge image data, and denoising, filtering, downsampling and aligning the point cloud data through statistics.

[0015] As a preferred solution of the method for edge detection and alignment of hot press workpieces based on image data analysis according to the present invention, wherein: calculate the gradient of the preprocessed edge image data using the Sobel operator to obtain the edge image point set of the hot press workpiece. The specific steps are as follows.

[0016] Initialize the convolution kernel of the Sobel operator in the horizontal direction and the convolution kernel in the vertical direction to obtain the Sobel operator in the horizontal direction and the Sobel operator in the vertical direction;

[0017] Apply the Sobel operator in the horizontal direction and the vertical direction respectively to calculate the gradient values of each pixel point of the preprocessed edge image data in the horizontal direction and the vertical direction to obtain the gradient value of each pixel point;

[0018] Calculate the gradient magnitude value of each pixel point through modulo operation, which is expressed as

[0019]

[0020] where M(x,y) is the gradient magnitude value of each pixel point (x,y), x is the coordinate of the pixel point in the horizontal direction, y is the coordinate of the pixel point in the vertical direction, Gx is the gradient in the horizontal direction of the pixel point (x,y), and Gy is the gradient in the vertical direction of the pixel point (x,y);

[0021] Through non-maximum suppression, compare the gradient magnitude values of the current pixel point with the two pixel points along the gradient direction in its neighborhood to obtain the refined edge image;

[0022] Perform threshold processing on the refined edge image through the Otsu threshold method to obtain a preliminary edge point set;

[0023] Retain all strong edge points in the preliminary edge point set and connect weak edge points through the neighborhood to obtain the edge image point set.

[0024] As a preferred solution of the method for detecting and aligning the edges of workpieces of a hot press based on image data analysis according to the present invention, the following steps are included: Analyze the geometric characteristics of the edge image point set through Hough transform and spline fitting to obtain a set of feature points. The specific steps are as follows:

[0025] Apply the Hough transform to the edge image point set to obtain the detected lines and their parameters as line features.

[0026] Use cubic splines and the least squares method to perform spline fitting on the line features to obtain a spline curve.

[0027] Extract key feature points from the spline curve by analyzing the derivative and curvature of the spline curve, denoted as

[0028]

[0029] where C is the curvature value of the spline curve, S ″ (z) is the second derivative of the spline curve, S'(z) is the derivative of the spline curve, and z is the abscissa of a certain point on the spline curve.

[0030] Combine the key feature points into a point set to obtain a set of feature points.

[0031] As a preferred solution of the method for detecting and aligning the edges of workpieces of a hot press based on image data analysis according to the present invention, the following steps are included: Analyze the preprocessed point cloud data through normal vector change analysis and curvature calculation to obtain the edge point cloud of the hot press workpiece. The specific steps are as follows:

[0032] Calculate the normal vector of each point in the local neighborhood of the point cloud data through the covariance matrix to obtain the normal vector of each point, denoted as

[0033]

[0034] where V is the covariance matrix, k is the number of points in the local neighborhood, j is the index of the points in the local neighborhood, P j is the coordinate of the j-th point in the local neighborhood, is the average coordinate of all points in the local neighborhood, and T is the matrix transpose symbol;

[0035] Calculate the eigenvector corresponding to the minimum eigenvalue of the covariance matrix through eigenvalue decomposition to obtain the normal vector;

[0036] Use the change of the normal vector to calculate the curvature of each point to obtain the curvature of each point;

[0037] Set a curvature threshold to identify and mark the points in the high-curvature region, and combine the obtained edge points to obtain the edge point cloud of the hot press workpiece.

[0038] As a preferred solution of the method for detecting and aligning the edges of hot press workpieces based on image data analysis according to the present invention, the following steps are included: Using the least squares fitting method to process the edge point cloud of the hot press workpiece to obtain a reference line. The specific steps are as follows,

[0039] Using a cubic spline curve as the fitting model for the edge point cloud of the hot press workpiece, and the sum of squared errors as the objective function, which is expressed as,

[0040]

[0041] where E is the sum of squared errors, n represents the number of data points in the edge point cloud, y i is the y-coordinate of the i-th data point in the edge point cloud, S(x i ) is the predicted value of the cubic spline curve at the i-th point in the point cloud data, i is the index variable, and x i is the x-coordinate of the i-th data point in the edge point cloud;

[0042] Initializing the control point positions of the spline curve using the edge point cloud;

[0043] Using the least squares method to minimize the objective function and solve for the coefficients of the spline curve to obtain the optimal control points of the spline curve and the spline curve parameters;

[0044] Obtaining the reference line by obtaining the horizontal range of the workpiece edge from the edge point cloud and combining the spline curve parameters.

[0045] As a preferred solution of the method for detecting and aligning the edges of hot press workpieces based on image data analysis according to the present invention, the following steps are included: Respectively using the feature point set and the reference line to calculate the offset of the hot press workpiece, obtaining the image mode offset and the point cloud mode offset and fusing them to obtain the fused offset. The specific steps are as follows,

[0046] Using the SIFT feature point matching algorithm to calculate the offset of each feature point in the feature point set using the Euclidean distance, obtaining the offset of each feature point, and selecting the workpiece with the largest total number of matches as the reference workpiece;

[0047] Calculating the average of the offsets of each feature point to obtain the image mode offset of the hot press workpiece;

[0048] Taking the reference line of the reference workpiece as the reference baseline, and comparing the reference lines of other workpieces with the reference baseline to obtain the point cloud mode offset;

[0049] Performing a weighted average of the image mode offset and the point cloud mode offset to obtain the fused offset.

[0050] As a preferred solution of the method for edge detection and alignment of hot press workpieces based on image data analysis according to the present invention, the following steps are included: Using the fusion offset and the robotic arm to perform edge alignment on the hot press workpieces, and the specific steps are as follows.

[0051] According to the fusion offset, calculate the translation adjustment amount and rotation adjustment amount required for the robotic arm.

[0052] Generate a control command for the robotic arm based on the calculated adjustment amounts and send it to the robotic arm.

[0053] After the robotic arm completes the adjustment, re-collect the image data and point cloud data of the workpiece to verify the edge alignment situation of the workpiece.

[0054] In a second aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and the following is included: When the computer program is executed by the processor, it implements any step of the method for edge detection and alignment of hot press workpieces based on image data analysis as described in the first aspect of the present invention.

[0055] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and the following is included: When the computer program is executed by the processor, it implements any step of the method for edge detection and alignment of hot press workpieces based on image data analysis as described in the first aspect of the present invention.

[0056] The beneficial effects of the present invention are as follows: The method for edge detection and alignment of hot press workpieces based on image data analysis can effectively improve the accuracy of edge detection and alignment of hot press workpieces. By combining image data and point cloud data, it not only overcomes the deficiencies of traditional methods in detecting complex workpiece shapes and fuzzy edges, but also makes full use of the normal vector and curvature information in the point cloud data to improve the accuracy of edge extraction. Advanced algorithms such as the Sobel operator, Hough transform, and spline fitting are used for edge feature extraction and analysis, enabling more accurate capture of the geometric features of the workpiece edge. By fusing the offset of the image pattern and the point cloud pattern, it solves the common matching error problem in the edge alignment process of traditional methods and realizes a more stable and efficient alignment process. This method can significantly improve the alignment accuracy of hot press workpieces and reduce the processing errors caused by inaccurate edge detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for description in the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1It is a flowchart of the method for detecting and correcting the edge of the workpiece of the hot press based on image data analysis in Embodiment 1.

[0059] Figure 2 It is a flowchart of obtaining the reference line in Embodiment 1. Specific embodiments

[0060] To make the above objects, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings of the specification.

[0061] Many specific details are set forth in the following description in order to fully understand the present invention, but the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0062] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure or characteristic that can be included in at least one implementation manner of the present invention. The "in one embodiment" appearing in different places in this specification does not all refer to the same embodiment, nor is it an embodiment that is separate or selectively mutually exclusive with other embodiments.

[0063] Embodiment 1, referring to Figure 1 and Figure 2 , is the first embodiment of the present invention. This embodiment provides a method for detecting and correcting the edge of the workpiece of the hot press based on image data analysis, including the following steps:

[0064] S1. Collect the edge image data and point cloud data of the hot press workpiece, and perform preprocessing, including the following steps.

[0065] Use a high-definition industrial camera to take pictures of the hot press workpiece. During image acquisition, the workpiece should be kept stationary to avoid motion blur, and at the same time, ensure uniform illumination to avoid the influence of shadows. Use high-resolution images to capture the edge area of the workpiece to ensure that the images have sufficient details, especially in the complex areas of the workpiece contour.

[0066] Use a three-dimensional laser scanner or a structured light scanning device to collect the three-dimensional point cloud data of the hot press workpiece. The point cloud data should cover the entire surface of the workpiece and contain sufficiently dense points to ensure that the shape and edge features of the workpiece can be accurately described. Sample the point cloud data multiple times to ensure the integrity of the point cloud data at different angles, thereby avoiding data loss or noise influence.

[0067] Denoise the image using a Gaussian filter to reduce random noise in the image, especially uneven illumination or device noise that may occur during shooting. The standard deviation and window size of the Gaussian filter should be adjusted according to the noise characteristics of the image. Grayscale the image to convert the color image into a grayscale image for subsequent edge detection. Enhance the contrast of the grayscale image using histogram equalization or adaptive histogram equalization methods to make the contrast between the edge part and the background in the image more obvious, thereby improving the accuracy of subsequent edge detection.

[0068] Denoise the point cloud data using statistical filtering, mean filtering or median filtering to remove abnormal points caused by measurement errors or environmental factors. Downsample the point cloud using the voxel grid method. By setting the volume size, the density of the point cloud can be reduced while retaining the necessary shape information. The point cloud data comes from scans at different angles. Use the ICP point cloud registration algorithm to align multiple point cloud data to the same coordinate system to ensure the overall consistency of the point cloud data.

[0069] S2. Calculate the gradient of the preprocessed edge image data using the Sobel operator to obtain the edge image point set of the hot press workpiece, including the following steps.

[0070] Initialize the convolution kernels in the horizontal and vertical directions of the Sobel operator. The Sobel convolution kernels used are

[0071]

[0072] Apply the Sobel operators in the horizontal and vertical directions respectively to calculate the gradient values of each pixel point in the preprocessed edge image data in the horizontal and vertical directions to obtain the gradient value of each pixel point.

[0073] It should be noted that the structure and weight distribution of the Sobel operator convolution kernels Gx and Gy are based on the principle of discrete difference calculation and are used to detect changes in image grayscale values. The horizontal direction convolution kernel Gx is designed to calculate edge changes in the vertical direction, and the vertical direction convolution kernel Gy is used to calculate edge changes in the horizontal direction.

[0074] Calculate the gradient magnitude value of each pixel point through modulo operation, expressed as

[0075]

[0076] Among them, M(x, y) is the gradient magnitude value of each pixel point (x, y). The larger the gradient magnitude value, the more drastic the change in the grayscale value at that pixel point, that is, the more likely that pixel point is located at the edge of the image. x is the coordinate of the pixel point in the horizontal direction used to locate each pixel point in the image to ensure that the gradient magnitude value can correspond to the specific position. y is the coordinate of the pixel point in the vertical direction used to locate each pixel point in the image to ensure that the gradient magnitude value can correspond to the specific position. Gx is the gradient in the horizontal direction of the pixel point (x, y), which is used to extract the horizontal edge information in the image. Gy is the gradient in the vertical direction of the pixel point (x, y), which is used to eliminate the negative influence of the direction (because the gradient value may be positive or negative, and both become positive after squaring), and at the same time emphasize the contribution of larger gradient values to the result.

[0077] Through non-maximum suppression, the gradient magnitude values of the current pixel point and two pixel points along the gradient direction in its neighborhood are compared to obtain a refined edge image. It should be noted that non-maximum suppression is used to refine the gradient magnitude map, remove non-edge points, and make the edge clearer and single-pixelized. Its core idea is to retain the local maximum points of the gradient magnitude (i.e., edge points) and suppress non-local maximum points (i.e., noise points or weak edge points).

[0078] Through the Otsu threshold method (Tianjin threshold method), the refined edge image is thresholded to obtain a preliminary set of edge points. It should be noted that the Otsu threshold method is used to binarize the refined gradient magnitude map, classifying pixel points into two categories: "edge points" and "non-edge points". It automatically calculates the global optimal threshold, avoiding the subjectivity of manually selecting the threshold.

[0079] All strong edge points in the preliminary set of edge points are retained, and weak edge points are connected through the neighborhood to obtain a set of edge image points. Specifically, the eight-neighborhood connection rule is used to determine whether a weak edge point is connected to a strong edge point. If there is at least one strong edge point in the eight-neighborhood of the weak edge point, then the weak edge point is retained; otherwise, the point is removed. It should be noted that strong edge points refer to pixel points whose gradient magnitude values exceed the high threshold (the global threshold calculated by the Otsu method). These points are usually located in the main area of the edge and have a higher credibility. Weak edge points are pixel points whose gradient magnitude values are lower than the high threshold but higher than the low threshold. These points may be the details of the edge or noise points. The processing rule is that strong edge points are directly retained; weak edge points need to be associated with strong edge points through neighborhood connection to be retained, otherwise they are regarded as noise points and removed.

[0080] S3. Analyze the geometric characteristics of the set of edge image points through the Hough transform and spline fitting to obtain a set of feature points, including the following steps,

[0081] Apply the Hough transform to the edge image point set to obtain the detected lines and their parameters as line features. It should be noted that the Hough transform is used to detect lines in the image and analyze the geometric characteristics of the edge point set. The Hough transform converts the points in the image from the Cartesian coordinate system to the Hough space (polar coordinate system), transforming the problem of detecting lines into the problem of curve intersections in the parameter space.

[0082] Use cubic splines and the least squares method to perform spline fitting on the line features to obtain a spline curve. It should be noted that a set of homogeneous polynomial functions are used to smoothly fit the edge point set to generate a continuous and differentiable spline curve, which is suitable for describing complex curve features. The role of the least squares method is to minimize the fitting error and ensure that the fitting curve is as close as possible to the original edge point set.

[0083] The cubic spline is expressed as

[0084] S(z) = az3 + bz2 + cz + d;

[0085] where S(z) is the predicted value of the cubic spline curve in the point cloud data, a is the coefficient of the cubic term, b is the coefficient of the quadratic term, c is the coefficient of the linear term, d is the offset, and z is the abscissa of a certain point on the spline curve.

[0086] By analyzing the derivative and curvature of the spline curve, extract key feature points from the spline curve, expressed as

[0087]

[0088] where C is the curvature value of the spline curve, used to represent the degree of bending of the spline curve at a certain point, S ″ (z) is the second derivative of the spline curve, representing the slope or rate of change of the spline curve at a certain point, that is, the tangent slope of the curve at the independent variable z, S'(z) is the derivative of the spline curve, representing the rate of change of the curvature of the spline curve at a certain point, that is, the degree of bending of the curve, x is the abscissa of a certain point on the spline curve, representing the independent variable of the spline curve.

[0089] Combine the key feature points into a point set to obtain a feature point set. It should be noted that the feature point set is a high-level abstraction of the geometric characteristics of the edge image, including contour information, curvature information, inflection point (or turning point) information, and global geometric distribution.

[0090] S4. Analyze the preprocessed point cloud data through normal vector change analysis and curvature calculation to obtain the edge point cloud of the hot press workpiece, including the following steps

[0091] Calculate the normal vector of each point in the local neighborhood of the point cloud data through the covariance matrix to obtain the normal vector of each point, expressed as

[0092]

[0093] Among them, V is the covariance matrix, which is used to represent the covariance matrix of the point cloud distribution within the local neighborhood of the current point. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix is the normal vector direction of this point. k is the number of points within the local neighborhood, which is used to represent the number of points participating in the calculation within the local neighborhood of the current point. j is the index of the points within the local neighborhood, which is used to calculate the covariance matrix of the current point point by point. P j is the coordinate of the j-th point within the local neighborhood, and its function is to participate in the calculation of the covariance matrix, reflecting the position of the point in three-dimensional space. is the average coordinate of all points within the local neighborhood, which is used to normalize the coordinates of the local points into a coordinate system relative to the centroid, facilitating the analysis of the distribution characteristics of the points. T is the matrix transpose symbol.

[0094] The eigenvector corresponding to the minimum eigenvalue of the covariance matrix is calculated through eigenvalue decomposition to obtain the normal vector. It should be noted that by decomposing the covariance matrix, calculating its eigenvalues and eigenvectors, the normal vector can be obtained. The principle is that the eigenvalues and eigenvectors of the covariance matrix reflect the local geometric characteristics of the point cloud. Further, the eigenvector corresponding to the maximum eigenvalue represents the main direction of the point cloud distribution; the eigenvector corresponding to the minimum eigenvalue represents the normal vector direction (i.e., the change of the point cloud in this direction is the smallest).

[0095] The curvature of each point is calculated using the change of the normal vector to obtain the curvature of each point. It should be noted that the curvature reflects the degree of bending of the point cloud in the local area and is calculated through the rate of change of the normal vector or the eigenvalues of the covariance matrix.

[0096] A curvature threshold is set to identify and mark the points in the high-curvature region, and the edge points are obtained and combined to obtain the edge point cloud of the hot press workpiece. Specifically, according to the curvature distribution of the point cloud data, a curvature threshold is set. For each point, if the curvature value is greater than the curvature threshold, it is marked as a high-curvature point; otherwise, it is considered a low-curvature point. It should be noted that the edge point cloud is a key part with geometric features in the point cloud data, representing the edge or feature region of the target object.

[0097] S5. The least squares fitting method is used to process the edge point cloud of the hot press workpiece to obtain the reference line, including the following steps.

[0098] A cubic spline curve is used as the fitting model for the edge point cloud of the hot press workpiece, and the sum of squared errors is used as the objective function, which is expressed as

[0099]

[0100] Among them, E is the sum of squared errors, representing the total error between the fitted curve and the actual data points. n is the number of data points in the edge point cloud, y i is the y-coordinate of the i-th data point in the edge point cloud, S(x i ) is the predicted value of the cubic spline curve at the i-th point in the point cloud data. i is the index variable of the data point, x i is the x-coordinate of the i-th data point in the edge point cloud.

[0101] Initialize the control point positions of the spline curve using the edge point cloud. Specifically, use the x-coordinates of the edge point cloud as the knots of the spline curve. Further, select some key points in the edge point cloud as the knots (control points) of the spline curve. These knots are usually data points evenly distributed in the horizontal direction or points with large curvature changes.

[0102] Minimize the objective function using the least squares method and solve for the coefficients of the spline curve to obtain the optimal spline curve control points and spline curve parameters. Specifically, first, design a matrix where each row corresponds to a single data point and contains the values of the basis functions. Second, for the cubic spline curve, the basis functions include a constant term, a linear term, a quadratic term, and a cubic term. Then, the observation vector contains the actual y-coordinates of all data points. Next, find the coefficients of the spline curve by solving a system of linear equations. The solution of this system of equations minimizes the sum of squared errors. Finally, through the least squares method, find the coefficient values that minimize the sum of squared errors. It should be noted that the least squares method is a commonly used data fitting method for finding a curve or a straight line that minimizes the sum of squared errors between the curve or the straight line and the given data points. This method is widely used in fields such as regression analysis and curve fitting.

[0103] Obtain the reference line by obtaining the horizontal range of the workpiece edge from the edge point cloud and combining the spline curve parameters. Specifically, find the minimum and maximum values of the x-coordinates from the edge point cloud. Within this range, use the fitted cubic spline curve to generate a series of y-coordinates corresponding to the x-coordinates, and the generated points form a smooth reference line.

[0104] S6. Calculate the offsets of the hot press workpiece using the feature point set and the reference line respectively, obtain the image mode offset and the point cloud mode offset and fuse them to obtain the fused offset, including the following steps

[0105] Using the SIFT feature point matching algorithm (Scale-Invariant Feature Transform), calculate the offset of each feature point in the feature point set using the Euclidean distance to obtain the offset of each feature point, and select the workpiece with the largest total number of matches as the reference workpiece. It should be noted that the main feature of the SIFT feature point matching algorithm is that it can detect key points in the image under different scales, rotations, and lighting conditions and generate descriptors for these key points. The Euclidean distance is used to calculate the distance between two vectors and is a commonly used metric in SIFT feature point matching for comparing the similarity between descriptors.

[0106] Perform an average calculation on the offset of each feature point to obtain the image pattern offset of the hot press workpiece, denoted as

[0107]

[0108] where ΔF represents the offset of each feature point and is used to measure the displacement of two corresponding feature points in the image, x F is the x-coordinate of a certain feature point in the feature point set, x F' is the x-coordinate of the corresponding feature point in the feature point set, y F is the y-coordinate of a certain feature point in the feature point set, y F' is the y-coordinate of the corresponding feature point in the feature point set.

[0109] Take the reference baseline of the reference workpiece as the reference baseline and compare the baselines of other workpieces with the reference baseline to obtain the point cloud pattern offset. It should be noted that the reference baseline of the reference workpiece (usually the standard geometric shape or ideal shape of the target workpiece) is used to measure the geometric offset of other workpieces. In industrial applications, the actual shape and size of workpieces may deviate due to factors such as errors and deformations during the production process. Therefore, the reference baseline of the reference workpiece provides an ideal standard.

[0110] Perform a weighted average on the image pattern offset and the point cloud pattern offset to obtain the fusion offset. It should be noted that the purpose of performing a weighted average on the image pattern offset and the point cloud pattern offset is to utilize the advantages of two different data sources to improve the calculation accuracy. The image offset is based on two-dimensional image data and can provide displacement information of the workpiece surface contour; while the point cloud offset is based on three-dimensional scan data and can provide more comprehensive spatial position change information. By performing a weighted average, the advantages of these two offsets can be combined to avoid the errors and limitations that may be brought by a single data source.

[0111] S7. Use the fusion offset and the robotic arm to perform edge correction on the hot press workpiece, including the following steps

[0112] Based on the fusion offset, calculate the translation adjustment amount and rotation adjustment amount that the robotic arm needs to perform, expressed as,

[0113]

[0114] where, ΔT is the translation adjustment amount, ΔX is the translation adjustment amount of the workpiece on the horizontal axis, ΔY is the translation adjustment amount of the workpiece on the depth axis, ΔZ is the translation adjustment amount of the workpiece on the vertical axis, ΔR is the rotation adjustment amount, and Δθ x is the rotation angle adjustment amount of the workpiece around the horizontal axis, and Δθ y is the rotation angle adjustment amount of the workpiece around the depth axis, and Δθ z is the rotation angle adjustment amount of the workpiece around the vertical axis.

[0115] Specifically, based on the calculated fusion offset (i.e., the weighted average of the image mode offset and the point cloud mode offset), determine the adjustment amount of the robotic arm. This adjustment amount includes the changes in the position and orientation of the workpiece, specifically including translation (displacement in the X, Y, and Z directions) and rotation (angular rotation around the X, Y, and Z axes).

[0116] Based on the calculated adjustment amount, generate the control instructions for the robotic arm and send them to the robotic arm. It should be noted that after calculating the required translation and rotation adjustment amounts, the next step is to convert these adjustment amounts into the control instructions for the robotic arm. These instructions will tell the robotic arm how to operate its various joints, how to move in various directions, or how to adjust its orientation (rotation). Specifically, the control instructions include translation instructions, which specifically describe how much distance the end effector (such as a gripper) of the robotic arm needs to move along the X, Y, and Z axes respectively. The rotation instructions will specify the rotation angle and the rotation axis. For example, if it is calculated that the workpiece needs to be rotated by a certain angle to align with the reference line, the instructions will tell the robotic arm to rotate by the specified angle.

[0117] After the robotic arm completes the adjustment, re - collect the image data and point cloud data of the workpiece to verify the edge alignment of the workpiece. It should be noted that the new image data and point cloud data will be compared with the previous standard (reference workpiece or ideal state) to ensure the effectiveness of the edge correction process. If it is found that the offset still exists, further adjustments may be required until the edge alignment accuracy of the workpiece meets the requirements.

[0118] This embodiment also provides a computer device, applicable to the case of the edge detection and alignment method of the hot press workpiece based on image data analysis, including: a memory and a processor; the memory is used to store computer - executable instructions, and the processor is used to execute the computer - executable instructions to implement the edge detection and alignment method of the hot press workpiece based on image data analysis as proposed in the above embodiment.

[0119] The computer device may be a terminal, which includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be achieved through WIFI, carrier networks, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0120] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for edge detection and alignment of hot press workpieces based on image data analysis as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM for short), Electrically Erasable Programmable Read-Only Memory (EEPROM for short), Erasable Programmable Read-Only Memory (EPROM for short), Programmable Read-Only Memory (PROM for short), Read-Only Memory (ROM for short), magnetic memory, flash memory, magnetic disks, or optical discs.

[0121] In summary, by combining image data and point cloud data, the present invention not only overcomes the deficiencies of traditional methods in detecting complex workpiece shapes and fuzzy edges, but also makes full use of the normal vector and curvature information in the point cloud data to improve the accuracy of edge extraction. Advanced algorithms such as Sobel operator, Hough transform, and spline fitting are used for edge feature extraction and analysis, so that the geometric features of the workpiece edge can be captured more accurately. By fusing the image pattern and the offset of the point cloud pattern, the common matching error problem in the edge alignment process of traditional methods is solved, and a more stable and efficient alignment process is achieved. This method can significantly improve the alignment accuracy of hot press workpieces and reduce processing errors caused by inaccurate edge detection.

[0122] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for detecting and aligning the edge of a workpiece of a hot press based on image data analysis, characterized in that: include, Collect edge image data and point cloud data of hot press workpieces and perform preprocessing; The Sobel operator is used to calculate the gradient of the preprocessed edge image data to obtain the edge image point set of the hot press workpiece; The geometric characteristics of the edge image point set are analyzed through Hough transform and spline fitting to obtain the feature point set; The pre-processed point cloud data is analyzed by normal vector change analysis and curvature calculation to obtain the edge point cloud of the hot press workpiece; The edge point cloud of the hot press workpiece is processed by the least square fitting method to obtain the baseline; The offset of the hot press workpiece is calculated by using the feature point set and the baseline respectively, and the image mode offset and the point cloud mode offset are obtained and fused to obtain the fused offset; Edge alignment of workpieces in hot presses using fusion offsets and robotic arms.

2. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 1, characterized in that: The preprocessing includes performing image denoising on edge image data through a Gaussian filter, graying and contrast enhancement on edge image data, and performing denoising, filtering, downsampling and alignment on point cloud data through statistics.

3. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 2, characterized in that: The Sobel operator is used to calculate the gradient of the preprocessed edge image data to obtain the edge image point set of the hot press workpiece. The specific steps are as follows: Initialize the convolution kernel in the horizontal direction and the convolution kernel in the vertical direction of the Sobel operator to obtain the Sobel operator in the horizontal direction and the Sobel operator in the vertical direction; Apply Sobel operators in the horizontal direction and the vertical direction respectively to calculate the gradient value of each pixel point in the horizontal direction and the vertical direction of the preprocessed edge image data to obtain the gradient value of each pixel point; The gradient value of each pixel is calculated by modulo operation to obtain the gradient amplitude value of each pixel, which is expressed as: Where M(x,y) is the gradient amplitude value of each pixel point (x,y), x is the horizontal coordinate of the pixel point, y is the vertical coordinate of the pixel point, Gx is the horizontal gradient of the pixel point (x,y), and Gy is the vertical gradient of the pixel point (x,y); By using non-maximum suppression, the gradient amplitude values ​​of the current pixel and two pixels in its neighborhood along the gradient direction are compared to obtain a refined edge image. The refined edge image is threshold processed by Otsu threshold method to obtain a preliminary edge point set; All strong edge points are retained for the preliminary edge point set, and weak edge points are connected through neighborhoods to obtain the edge image point set.

4. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 3, characterized in that: The geometric characteristics of the edge image point set are analyzed through Hough transform and spline fitting to obtain the feature point set. The specific steps are as follows: Apply Hough transform to the edge image point set to obtain the detected straight line and its parameters as the straight line feature; Using cubic spline and least square method, spline fitting is performed on the straight line features to obtain the spline curve; By analyzing the derivative and curvature of the spline curve, key feature points are extracted from the spline curve, which is expressed as, Among them, C is the curvature value of the spline curve, S ″ (z) is the second-order derivative of the spline curve, S'(z) is the derivative of the spline curve, and z is the horizontal coordinate of a point on the spline curve; The key feature points are combined into a point set to obtain a feature point set.

5. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 4, characterized in that: The pre-processed point cloud data is analyzed through normal vector change analysis and curvature calculation to obtain the edge point cloud of the hot press workpiece. The specific steps are as follows: Through the covariance matrix, the normal vector of each point in the local neighborhood of the point cloud data is calculated to obtain the normal vector of each point, which is expressed as: Where V is the covariance matrix, k is the number of points in the local neighborhood, j is the index of the point in the local neighborhood, and P j is the coordinate of the jth point in the local neighborhood, is the average coordinate of all points in the local neighborhood, and T is the matrix transpose symbol; The eigenvalue decomposition is used to calculate the eigenvector corresponding to the minimum eigenvalue of the covariance matrix, and the normal vector is obtained; The curvature of each point is calculated using the change of the normal vector to obtain the curvature of each point; The curvature threshold is set, the points marking the high curvature area are identified, the edge points are obtained and combined, and the edge point cloud of the hot press workpiece is obtained.

6. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 5, characterized in that: The least squares fitting method is used to process the edge point cloud of the hot press workpiece to obtain the baseline. The specific steps are as follows: The cubic spline curve is used as the fitting model of the edge point cloud of the hot press workpiece, and the error square sum is used as the objective function. The objective function is expressed as: Among them, E is the sum of squared errors, n is the number of data points in the edge point cloud, and y i is the y coordinate of the i-th data point in the edge point cloud, S(x i ) is the predicted value of the cubic spline curve at the i-th point in the point cloud data, i is the index variable, x i is the x-coordinate of the i-th data point in the edge point cloud; Use edge point cloud to initialize the control point position of spline curve; Use the least square method to minimize the objective function and solve the coefficients of the spline curve to obtain the optimal spline curve control points and spline curve parameters; The reference line is obtained by obtaining the horizontal range of the workpiece edge from the edge point cloud and combining it with the spline curve parameters.

7. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 6, characterized in that: The offset of the hot press workpiece is calculated using the feature point set and the baseline respectively, and the image mode offset and the point cloud mode offset are obtained and fused to obtain the fused offset. The specific steps are as follows: Using SIFT feature point matching algorithm, the Euclidean distance is used to calculate the offset of each feature point in the feature point set, the offset of each feature point is obtained, and the workpiece with the largest total matching number is selected as the reference workpiece; The offset of each feature point is averaged to obtain the image pattern offset of the hot press workpiece; The baseline of the reference workpiece is used as the reference baseline, and the baselines of other workpieces are compared with the reference baseline to obtain the point cloud mode offset; The image mode offset and the point cloud mode offset are weighted averaged to obtain the fusion offset.

8. The method for detecting and aligning the edge of a workpiece of a hot press machine based on image data analysis according to claim 7, characterized in that: Using the fusion offset and the robotic arm, the edge alignment of the hot press workpiece is performed. The specific steps are as follows: According to the fusion offset, the translation adjustment and rotation adjustment required by the robot arm are calculated; According to the calculated adjustment amount, a control instruction of the robot arm is generated and sent to the robot arm; After the robot arm completes the adjustment, re-collect the image data and point cloud data of the workpiece to verify the edge alignment of the workpiece.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the hot press workpiece edge detection and alignment method based on image data analysis described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hot press workpiece edge detection and alignment method based on image data analysis described in any one of claims 1 to 8 are implemented.

Images