Machine vision-based longitudinal skin stretcher jaw shaping control method

Abstract

The application provides a kind of longitudinal skin stretcher jaw shape control method based on machine vision, which includes: S1, install image acquisition system;S2, calibrate camera by Zhang Zhengyou calibration method, establish the relationship between target pixel point and corresponding target point position in space;S3, turn on the light source, obtain the camera shooting image, and pretreat the image;S4, edge extraction is carried out on the pretreated gray scale image;S5, the edge gray scale image curve is fitted by least square method, the stroke of stretch machine jaw shape control hydraulic cylinder is adjusted, and longitudinal skin stretcher jaw shape control is realized.The application can realize accurate detection of skin blank and jaw state, make the jaw shape and end face shape consistent, and complete the fast and accurate feeding of skin blank into the jaw.The application applies machine vision recognition technology to aviation special equipment, improves the skin stretching precision and the efficiency of blank feeding into the jaw, and helps the transformation and upgrading of equipment automation and intelligentization.

Description

Technical Field

[0001] This invention relates to the field of skin stretching and forming technology, and in particular to a machine vision-based method for jaw adjustment control of a longitudinal skin stretching machine. Background Technology

[0002] When stretching sheet metal using a longitudinal sheet metal stretching machine, the sheet metal blank is first clamped in the machine's jaws. The jaws are then adjusted to resemble the shape of the die end face. Subsequently, the jaws stretch the sheet metal while the worktable rises to bring the stretching die into contact with the sheet, ensuring the sheet metal blank fits snugly against the die, thus obtaining the desired sheet metal shape. This technology is primarily used in aerospace and high-speed rail industries. However, due to the irregular shape of the sheet metal blank, especially the significant deformation after heat treatment, current jaw adjustment relies mainly on experience and experimentation. The jaws are adjusted visually based on the shape of the blank end face before the blank is fed into the jaws. This often results in difficulty feeding the blank due to inadequate jaw shape adjustment. When the sheet metal material is fed into the jaws, if there is no corresponding digital die for the sheet metal to be produced, the jaw shape needs to be visually adjusted based on the die end face shape before clamping. There are no specified parameter details, such as the interrelationship of the jaws, the change in the jaw angle during the sheet metal stretching process, and the control of the stretching amount. The stretching of the blank shape during this shaping process is uncontrollable, increasing workload and labor intensity, resulting in low forming quality and inevitably leading to many unpredictable defects.

[0003] With the rapid development of computer and semiconductor technologies, machine vision theory and applications have been further advanced. Currently, machine vision recognition technology is in a new stage of continuous breakthroughs and maturation. This invention represents the successful application of machine vision recognition technology in aviation-specific equipment, contributing to the automation and intelligent transformation and upgrading of such equipment. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a machine vision-based method for controlling the jaw shape of a longitudinal skin stretching machine. This invention enables precise detection of the skin blank and jaw state, ensuring that the jaw shape matches the end face shape, and allowing for rapid and accurate feeding of the skin blank into the jaw. This invention applies machine vision recognition technology to aerospace equipment, improving the skin stretching accuracy and blank feeding efficiency into the jaw, and contributing to the automation and intelligent transformation and upgrading of equipment.

[0005] This invention provides a machine vision-based method for jaw shaping control of a longitudinal skin stretching machine, the specific steps of which are as follows:

[0006] S1: Install the image acquisition system; mount the CCD camera, lens, and light source above the jaws of the stretching machine, and adjust the mounting angles of the CCD camera and light source;

[0007] S2: The CCD camera is calibrated using Zhang Zhengyou's calibration method to obtain the CCD camera's intrinsic parameter matrix A, extrinsic parameter matrix R, extrinsic parameter matrix t, and distortion coefficients k1 and k2. The relationship between the target pixels in the image and the corresponding target points in space is established as follows:

[0008]

[0009] Among them, Z c denoted as , where is the vertical coordinate of the target pixel in the image; u is the horizontal coordinate transformation coefficient of the target pixel in the image; v is the vertical coordinate transformation coefficient of the target pixel in the image; A is the intrinsic parameter matrix of the camera; R is the first extrinsic parameter matrix; t is the second extrinsic parameter matrix; X W Y is the x-coordinate of the corresponding target point in space; W Z represents the ordinate of the target point in space; W The vertical coordinates of the corresponding target point in space;

[0010] S3: Turn on the light source, acquire images captured by the CCD camera, and preprocess the images;

[0011] S31: First, perform grayscale transformation on the image to convert the color image into a grayscale image. The expression for converting a color image to a grayscale image is as follows:

[0012] Gray(i,j) = 0.299 * R c (i,j)+0.587*G c (i,j)+0.114*B c (i,j);

[0013] Where Gray(i,j) represents the grayscale value of the grayscale image; R c (i,j) represents the red value of the color image; G c (i,j) represents the green value of the color image; B c (i,j) represents the blue value of the color image; i represents the horizontal coordinate number of the pixel in the grayscale image; j represents the vertical coordinate number of the pixel in the grayscale image.

[0014] S32: Perform geometric transformations on the grayscale image to adjust its size and position;

[0015] S33: Histogram equalization is used to enhance grayscale images;

[0016] S34: Noise is processed in the image by using a hybrid filtering method;

[0017] S4: Extract edges from the preprocessed grayscale image;

[0018] S41: The grayscale image is segmented by binarization. The grayscale image obtained by binarization is then processed by mathematical morphology and Canny edge detection. Finally, edge processing is performed on the grayscale image to make the edges of the grayscale image clearer.

[0019] S42: Perform wavelet fusion on the edge grayscale image obtained in step S41 to extract the edge grayscale image curve;

[0020] S5: Fit the edge grayscale image curve extracted in step S4 using the least squares method; divide the edge grayscale image curve into multiple segments, obtain the curvature of each segment, and obtain the data coordinates (x, y) of each segment. m ,y m The fitted function is y = F(x); such that M = F(x) m )-y m In the process of calculating F(x), the sum of squared errors is minimized, as shown below:

[0021]

[0022] Where F(x) is the optimal function of the fitted curve; M is the fitting error; m is the data coordinate point number, m = 1, 2, 3, ..., n; n is the total number of data coordinate points; x m y is the ordinate of the m-th data point; m Here, represents the ordinate of the m-th data point; min is the function for finding the minimum value.

[0023] By adjusting the stroke of the hydraulic cylinder for shaping the jaws of the stretching machine using the curve curvature extracted by the optimal function F(x), the jaw shaping control of the longitudinal skin stretching machine can be achieved.

[0024] Preferably, in step S2, establishing the relationship between the target pixel in the image and the corresponding target point in space specifically involves:

[0025] S21: Camera intrinsic parameters describe the correspondence between points in the camera coordinate system and pixels in the pixel coordinate system, and are described by the following expression:

[0026]

[0027] Where α is the assumed scaling factor of the pixel coordinates on the horizontal axis; β is the assumed scaling factor of the pixel coordinates on the vertical axis; f is the rotation matrix of the origin; f x The first rotation matrix is ​​the origin; f y The second rotation matrix is ​​the origin; c x c is the first translation matrix to the origin; y The second translation matrix is ​​the origin; P W For real-world coordinates, the coordinates are [X...W Y W Z W ] T ;

[0028] S22: Camera extrinsic parameters describe the correspondence between points in the world coordinate system and those in the camera coordinate system, and are expressed by the following expression:

[0029]

[0030] Among them, P c For P W Coordinates of the projection point in the camera coordinate system;

[0031] S23: Use distortion coefficients to correct radial distortion generated during lens imaging and establish the relationship between target pixels in the image and the corresponding target points in space.

[0032] Preferably, the geometric transformation of the grayscale image in step S32 specifically includes image translation, image rotation, and image scaling;

[0033] The translation expression for the image is as follows:

[0034]

[0035] Where, p t p0 is the position after translation; p0 is the initial position; T is the translation matrix; x t y is the x-coordinate of the movement vector; t The vertical coordinate of the movement vector;

[0036] The rotation expression for the image is as follows:

[0037]

[0038] Where, p r The point after rotation; R z Here is the rotation matrix; γ is the rotation angle of the point about the origin;

[0039] The scaling expression for the image is as follows:

[0040]

[0041] Where, p s The position is the rotated point; S is the scaling matrix; s x s is the magnification factor along the x-axis; y This represents the magnification factor along the y-axis.

[0042] Preferably, in step S33, histogram equalization is used to enhance the grayscale image, specifically as follows:

[0043] S331: Determine the gray levels of the grayscale image. The gray level value range is 0-255, and here we take 8 gray levels, i.e., r k The values ​​are 0, 1 / 7, 2 / 7, 3 / 7, 4 / 7, 5 / 7, 6 / 7, and 1.

[0044] S332: Calculate the probability of the histogram of the original image and count the number of pixels n for each gray level in the original grayscale image. k And calculate its probability P(r) k ), to obtain the probability of the histogram of the original image;

[0045] S333: Calculate the cumulative probability s of the histogram probability. k As shown below

[0046]

[0047] Among them, s k The cumulative probability of the histogram probability; r k To normalize the gray levels; P(r) k ) represents the probability of the k-th gray level appearing; n k is the frequency of the k-th gray level; N is the total number of pixels in the original image; k is the number of gray levels, k∈[0-7];

[0048] S334: Calculate the pixel mapping relationship of the grayscale image, as shown below:

[0049] ss i =int(max(pix)-min(pix))*s k +0.5;

[0050] Among them, ss i This represents the pixel mapping relationship of a grayscale image; pix represents the grayscale level; max is the maximization function; int is the integer function.

[0051] S335: Perform grayscale image enhancement processing based on the mapping relationship of grayscale images.

[0052] Preferably, in step S34, noise processing of the image is performed using a hybrid filtering method, specifically as follows:

[0053] S341: Let x(i,j) be the pixel value at (i,j) in the grayscale image, then the median output of this pixel is m(i,j); set I(i,j) = |x(i,j) - m(i,j)|, which is the absolute difference between x(i,j) and the median output m(i,j); p1(i,j) and p2(i,j) are the two points closest to x(i,j) in the selected neighborhood, as shown below:

[0054]

[0055] Among them, d1 is the shortest distance between the points in the first neighborhood and the pixel point; d2 is the shortest distance between the points in the second neighborhood and the pixel point; x(i,j) is the pixel value at the (i,j) position of the grayscale image; p1(i,j) is the point in the first neighborhood of x(i,j); p2(i,j) is the point in the second neighborhood of x(i,j); v d (i,j) is the distance mean value; I(i,j) is the absolute difference between the pixel value at the (i,j) position of the grayscale image and the median output m(i,j); m(i,j) is the median output of the pixel at the (i,j) position of the grayscale image;

[0056] S342: Set the first filtering threshold T1 and the second filtering threshold T2 to 10, as shown below:

[0057]

[0058] Among them, T1 is the first filtering threshold; T2 is the second filtering threshold; N k is the grayscale median value within the filtering window; med is the statistical function for calculating the median of a set of data; is the absolute difference between the grayscale value of point l and the grayscale median value; l is the pixel point within the sliding pane; S 5x5 is a 5×5 sliding pane centered on x(i,j);

[0059] S343: If I(i,j)≥T1 and v d (i,j)≥T2, adopt the median filtering algorithm; if I(i,j)≥T1 and v d (i,j)<T2, adopt the mean filtering algorithm; if I(i,j)<T1, adopt the mean filtering algorithm; complete the filtering process of the grayscale image.

[0060] Preferably, in step S41, the grayscale image is segmented by binarization, and the obtained grayscale image after binarization is respectively subjected to mathematical morphology processing and canny edge detection processing, specifically:

[0061] S411: Segment the image by binarization; The segmentation based on the threshold divides all the pixels in the image into two categories according to the change of the grayscale level. A threshold is determined between the maximum threshold and the minimum threshold in the image as the boundary threshold, and it is divided into two categories, greater than the boundary threshold and less than the boundary threshold, that is, the result of image binarization is obtained; The expression of image binarization is as follows:

[0062]

[0063] Where Binary(i,j) is a binary function; Threshold is a set threshold.

[0064] S412: Mathematical morphology algorithms include four basic operations: erosion, dilation, opening, and closing. Erosion and dilation can be combined to perform opening and closing operations.

[0065] The expansion calculation process is as follows:

[0066]

[0067] Among them, A z For Z 2 The first set; B z For Z 2 The second set; It is an empty set; Let A be the first set. z by the second set B z The expansion; The expansion operator; (x,y) represents the second set B. z The position after the origin is translated;

[0068] The corrosion calculation process is as follows:

[0069]

[0070] Among them, A z ΘB z Let A be the first set. z by the second set B z Erosion; Θ is the erosion operator;

[0071] S413: Using the Canny operator, Gaussian transform is used to denoise and smooth the image, preserving appropriate edges. Non-maximum suppression is used, and high and low thresholds are set to effectively preserve edges. In a grayscale image, let it be g(x,y), and the Gaussian filtered image be h(x,y). The normal distribution equation in two-dimensional space is:

[0072]

[0073] Where h(x,y) is the Gaussian filtered image; σ is the standard deviation of the normal distribution;

[0074] The convolution operation is performed on the grayscale image as follows:

[0075] f(x,y)=g(x,y)*h(x,y);

[0076] Among them, f(x, y) is the image obtained after performing a convolution operation on the grayscale image; g(x, y) is the original grayscale image;

[0077] The maximum value after the convolution operation is used as the new grayscale value of the image center point; through differentiation, the direction β(x, y) of the grayscale image and the amplitude F(x, y) of the grayscale image are calculated as follows:

[0078]

[0079] Among them, β(x, y) is the direction of the grayscale image; F(x, y) is the amplitude of the grayscale image; G x is the gradient along the x direction; G y is the gradient along the y direction.

[0080] Preferably, in step S42, wavelet fusion is performed on the edge grayscale image to extract the edge grayscale image curve, specifically:

[0081] S421: Perform wavelet transform on the edge images obtained by the two algorithms for multi-scale analysis. The decomposition expression is as follows:

[0082]

[0083] Among them, WT(q, τ) is the wavelet transform function; q is the scale; τ is the translation amount; t w is the time variable; f w is the first sub-function of wavelet transform; y is the second sub-function of wavelet transform;

[0084] S422: Merge the transform coefficients according to the local variance criterion. Let O(x, y) and P(x, y) be the data values of the high-frequency sub-image obtained by the mathematical morphology algorithm and the canny algorithm respectively, and F(x, y) be the fusion value of the corresponding high-frequency sub-image. Divide O(x, y) and P(x, y) into several M×N sub-block images. Perform numerical distribution statistics on each sub-block image and calculate its variance. Determine the weighting coefficients k1 and k2 of each sub-block image of the O and P images; if the variance of the sub-block of the O image is greater than the variance of the sub-block of the P image, then k1≥k2, otherwise k1<k2; determine the data fusion value of each sub-block image as follows:

[0085] F H (x, y) = k1O(x, y) + k2P(x, y);

[0086] In the formula, F H(x,y) represents the corresponding high-frequency sub-image fusion value; O(x,y) represents the high-frequency sub-image data value obtained by the mathematical morphology algorithm; P(x,y) represents the high-frequency sub-image data value obtained by the Canny algorithm; k1 represents the weighting coefficient of each sub-block image of image O; k2 represents the weighting coefficient of each sub-block image of image P.

[0087] S423: Perform inverse wavelet transform on the merged coefficients to obtain the final fused image, as shown in the following expression:

[0088]

[0089] Where x(t) w The final fused image is obtained by using the wavelet transform function; W w This is the third sub-function of the wavelet transform.

[0090] Preferably, in step S5, the optimal function F(x) of the fitted curve function needs to be determined by identifying the function type. The curve is divided into multiple segments, and the bending angle Δθ of the mechanical structure can be calculated by obtaining the radius of curvature ρ of each segment, as shown below:

[0091]

[0092] In the formula, ρ is the radius of curvature; l is the arc length of the curve; (x1,x2) is the range of the curve length on the X-axis in the camera coordinate system; Δθ is the bending angle of the mechanical structure.

[0093] Preferably, step S5 involves controlling the jaws of the longitudinal skin stretching machine to adjust their shape, specifically including:

[0094] S51: The expression for calculating the upward bend angle is as follows:

[0095]

[0096] Where, Δθ 上 θ is the upward bending angle; θ0 is the horizontal bending angle; θ is the mechanical structure angle; Δθ 下 s is the downward bending angle; s0 is the initial position of the adjusting hydraulic cylinder; a is the first parameter of the mechanical structure; b is the second parameter of the mechanical structure;

[0097] The mechanical structure angle θ is calculated from the extracted mechanical structure angle bending angle Δθ using the two expressions above.

[0098] S52: By solving for the mechanical structure angle θ, the stroke s of the hydraulic cylinder is derived. H The relationship is as follows:

[0099] θ = arccos(a 2 +b 2 -s H2 ) / (2ab);

[0100] S53: The maximum and minimum stroke of each jaw are known; verify the stroke s of the hydraulic cylinder. H Is it in s min With s max between;

[0101] S54: Calculate the stroke of the hydraulic cylinder for adjusting each jaw and transmit it to the control system for control, thus realizing the adjustment of the jaws of the skin stretching machine.

[0102] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0103] This invention provides a machine vision-based method for adjusting the jaw shape of a longitudinal skin stretching machine. Compared with traditional methods, two sets of CCD cameras are installed above the middle jaw, equipped with lenses and light sources. The angles of the cameras and light sources are adjusted, and the intensity of the light source is adjusted to complete the configuration of the entire image acquisition system. By using machine vision, the curves of the blank end face and the mold end face are extracted and fitted. The jaw shape is adjusted according to the curvature of the obtained curves, thereby improving the quality and accuracy of the skin, reducing labor intensity, and increasing work efficiency. Attached Figure Description

[0104] Figure 1 This is a flowchart of the machine vision-based longitudinal skin stretching machine jaw shaping control method according to the present invention;

[0105] Figure 2 This is a schematic diagram of a vision-based longitudinal skin stretching machine jaw shaping device and control method according to an embodiment of the present invention.

[0106] Figure 3 This is a schematic diagram of CCD camera installation according to an embodiment of the present invention;

[0107] Figure 4 This is a flowchart of image preprocessing according to an embodiment of the present invention;

[0108] Figure 5 This is a schematic diagram of the entire working process of a vision-based skin stretching machine according to an embodiment of the present invention;

[0109] Figure 6 This is a schematic diagram showing the effect of preprocessing the mold end face image according to an embodiment of the present invention;

[0110] Figure 7 This is a schematic diagram showing the effect of preprocessing the image of the jaws of the longitudinal skin stretching machine according to an embodiment of the present invention;

[0111] Figure 8 This is a schematic diagram showing the effect of binarizing the mold end face image according to an embodiment of the present invention;

[0112] Figure 9 This is a schematic diagram showing the effect of binarization processing of the jaw image of the longitudinal skin stretching machine according to an embodiment of the present invention;

[0113] Figure 10 This is a schematic diagram of the result after the jaws of the longitudinal skin stretching machine are adjusted according to an embodiment of the present invention. Detailed Implementation

[0114] Exemplary embodiments, features, and aspects of the present invention will now be described in detail with reference to the accompanying drawings. Although various parameter values ​​and aspects of the embodiments are shown, the simulation process need not be performed with exactly the same parameters and aspects unless specifically indicated.

[0115] The present invention provides a machine vision-based jaw shaping control method for longitudinal skin stretching machines, such as... Figure 1 As shown, it includes:

[0116] Step S1: Install the image acquisition system; mount the CCD camera, lens, and light source above the jaws of the stretching machine, and adjust the mounting angles of the CCD camera and light source. For example... Figure 2 The diagram shown is a vision-based longitudinal skin stretching machine jaw shaping device and control method according to an embodiment of the present invention. A CCD camera installation diagram is also shown. Figure 3 As shown.

[0117] Step S2: Calibrate the CCD camera using Zhang Zhengyou's calibration method to obtain the CCD camera's intrinsic parameter matrix A, extrinsic parameter matrix R, extrinsic parameter matrix t, and distortion coefficients k1 and k2; establish the relationship between target pixels in the image and their corresponding spatial locations, specifically:

[0118] Step S21: Camera intrinsic parameters describe the correspondence between points in the camera coordinate system and pixels in the pixel coordinate system, and are described by the following expression:

[0119]

[0120] Where α is the assumed scaling factor of the pixel coordinates on the horizontal axis; β is the assumed scaling factor of the pixel coordinates on the vertical axis; f is the rotation matrix of the origin; f x The first rotation matrix is ​​the origin; f y The second rotation matrix is ​​the origin; c x c is the first translation matrix to the origin; y The second translation matrix is ​​the origin; P W For real-world coordinates, the coordinates are [X... W Y W Z W ]T .

[0121] Step S22: Camera extrinsic parameters describe the correspondence between points in the world coordinate system and those in the camera coordinate system, and are described by the following expression:

[0122]

[0123] Among them, P c For P W The coordinates of the projection point in the camera coordinate system.

[0124] Step S23: Use distortion coefficients to correct the radial distortion generated during lens imaging, and establish the relationship between the target pixel in the image and the corresponding target point in space, as shown below:

[0125]

[0126] Among them, Z c denoted as , where is the vertical coordinate of the target pixel in the image; u is the horizontal coordinate transformation coefficient of the target pixel in the image; v is the vertical coordinate transformation coefficient of the target pixel in the image; A is the intrinsic parameter matrix of the camera; R is the first extrinsic parameter matrix; t is the second extrinsic parameter matrix; X W Y is the x-coordinate of the corresponding target point in space; W Z represents the ordinate of the target point in space; W represents the vertical coordinates of the target point in space.

[0127] Step S3: Turn on the light source, acquire the image captured by the CCD camera, and preprocess the image; such as... Figure 4 The diagram shown is an image preprocessing flowchart according to an embodiment of the present invention.

[0128] Step S31: First, perform grayscale transformation on the image to convert the color image into a grayscale image. The expression for converting a color image to a grayscale image is as follows:

[0129] Gray(i,j) = 0.299 * R c (i,j)+0.587*G c (i,j)+0.114*B c (i,j);

[0130] Where Gray(i,j) represents the grayscale value of the grayscale image; R c (i,j) represents the red value of the color image; G c (i,j) represents the green value of the color image; B c (i,j) represents the blue value of the color image; i represents the horizontal coordinate number of the pixel in the grayscale image; j represents the vertical coordinate number of the pixel in the grayscale image.

[0131] Step S32: Perform geometric transformations on the grayscale image, adjusting its size and position. Specifically, geometric transformations include image translation, image rotation, and image scaling.

[0132] The translation expression for the image is as follows:

[0133]

[0134] Where, p t p0 is the position after translation; p0 is the initial position; T is the translation matrix; x t y is the x-coordinate of the movement vector; t The vertical coordinate of the movement vector.

[0135] The rotation expression for the image is as follows:

[0136]

[0137] Where, p r The point after rotation; R z is the rotation matrix; γ is the rotation angle of the point about the origin.

[0138] The image scaling expression is as follows:

[0139]

[0140] Where, p s The position is the rotated point; S is the scaling matrix; s x s is the magnification factor along the x-axis; y This represents the magnification factor along the y-axis.

[0141] Step S33: Enhance the grayscale image using histogram equalization, specifically as follows:

[0142] Step S331: Determine the gray levels of the grayscale image. The gray level value range is 0-255, and here we take 8 gray levels, i.e., r k The values ​​are 0, 1 / 7, 2 / 7, 3 / 7, 4 / 7, 5 / 7, 6 / 7, and 1.

[0143] Step S332: Calculate the probability of the histogram of the original image, and count the number of pixels n for each gray level in the original grayscale image. k And calculate its probability P(r) k ), which gives the probability of obtaining the histogram of the original image.

[0144] Step S333: Calculate the cumulative probability s of the histogram probability. k As shown below

[0145]

[0146] Among them, s k The cumulative probability of the histogram probability; r k To normalize the gray levels; P(r) k ) represents the probability of the k-th gray level appearing; n k is the frequency of the k-th gray level; N is the total number of pixels in the original image; k is the number of gray levels, k∈[0-7].

[0147] Step S334: Calculate the pixel mapping relationship of the grayscale image, as shown below:

[0148] ss i =int(max(pix)-min(pix))*s k +0.5;

[0149] Among them, ss i This represents the pixel mapping relationship of a grayscale image; pix represents the grayscale level; max is the maxima function; and int is the integer function.

[0150] Step S335: Perform grayscale image enhancement processing based on the mapping relationship of grayscale images.

[0151] Step S34: Noise processing of the image is performed using a hybrid filtering method, specifically as follows:

[0152] Step S341: Let x(i,j) be the pixel value at (i,j) in the grayscale image, then the median output of this pixel is m(i,j); set I(i,j) = |x(i,j) - m(i,j)|, which is the absolute difference between x(i,j) and the median output m(i,j); p1(i,j) and p2(i,j) are the two points closest to x(i,j) in the selected neighborhood, as shown below:

[0153]

[0154] Where d1 is the nearest distance between a point in the first neighborhood and a pixel; d2 is the nearest distance between a point in the second neighborhood and a pixel; x(i,j) is the pixel value at (i,j) in the grayscale image; p1(i,j) is the first neighboring point of x(i,j); p2(i,j) is the second neighboring point of x(i,j); v d (i,j) represents the mean distance; I(i,j) represents the absolute difference between the pixel value at (i,j) in the grayscale image and the median output m(i,j); m(i,j) represents the median output of the pixels at (i,j) in the grayscale image.

[0155] Step S342: Set the first filter threshold T1 and the second filter threshold T2 to 10, as shown below:

[0156]

[0157] Among them, T1 is the first filtering threshold; T2 is the second filtering threshold; N k is the median gray value within the filtering window; med is a statistical function for calculating the median of a set of data; is the absolute difference between the gray value of point l and the median gray value; l is a pixel point within the sliding pane; S 5x5 is a 5×5 sliding pane centered on x(i,j).

[0158] Step S343: If I(i,j) ≥ T1 and v d (i,j) ≥ T2, use the median filtering algorithm; if I(i,j) ≥ T1 and v d (i,j) < T2, use the mean filtering algorithm; if I(i,j) < T1, use the mean filtering algorithm; complete the filtering process of the grayscale image.

[0159] The image obtained after preprocessing is as Figure 6 is a schematic diagram of the effect after preprocessing the picture of the die end face in the embodiment of the present invention; Figure 7 is a schematic diagram of the effect after preprocessing the picture of the jaw of the longitudinal skin stretching machine in the embodiment of the present invention.

[0160] Step S4: Perform edge extraction on the preprocessed grayscale image.

[0161] Step S41: Segment the grayscale image through binarization, and perform mathematical morphology processing and canny edge detection processing on the binarized grayscale image respectively, specifically:

[0162] Step S411: Segment the image through binarization; Threshold-based segmentation divides all pixels in the image into two categories based on the change of gray levels. A threshold is determined as the boundary threshold between the maximum threshold and the minimum threshold in the image, and it is divided into two categories, greater than the boundary threshold and less than the boundary threshold, that is, the result of image binarization is obtained; the expression of image binarization is as follows:

[0163]

[0164] Among them, Binary(i,j) is a binary function; Threshold is the set threshold.

[0165] Step S412: The mathematical morphology algorithm includes four basic operation methods: erosion, dilation, opening operation, and closing operation. The combination of erosion and dilation can realize the operation of opening operation and closing operation.

[0166] The operation process of dilation is as follows:

[0167]

[0168] Among them, A z For Z 2 The first set; B z For Z 2 The second set; It is an empty set; Let A be the first set. z by the second set B z The expansion; The expansion operator; (x,y) represents the second set B. z The position after the origin is translated.

[0169] The erosion calculation process is as follows:

[0170]

[0171] Among them, A z ΘB z Let A be the first set. z by the second set B z Corrosion; Θ is the corrosion operator.

[0172] Step S413: Using the Canny operator, the image is denoised and smoothed using Gaussian transform, preserving appropriate edges. Non-maximum suppression is used, and high and low thresholds are set to effectively preserve edges. In a grayscale image, let it be g(x,y), and the Gaussian-filtered image be h(x,y). The normal distribution equation in two-dimensional space is:

[0173]

[0174] Where h(x,y) is the Gaussian filtered image; σ is the standard deviation of the normal distribution.

[0175] The convolution operation is performed on the grayscale image as follows:

[0176] f(x,y)=g(x,y)*h(x,y);

[0177] Where f(x,y) is the image obtained after convolving the grayscale image; g(x,y) is the original grayscale image.

[0178] After the convolution operation, the maximum value is used as the new gray value of the image center point; through difference, the direction β(x,y) and magnitude F(x,y) of the gray image are calculated, as shown below:

[0179]

[0180] where, β(x, y) is the direction of the grayscale image; F(x, y) is the amplitude of the grayscale image; G x is the gradient along the x direction; G y is the gradient along the y direction.

[0181] For the images obtained after processing the mold end face and jaw images, as Figure 8 is the schematic diagram of the effect after binarization processing of the mold end face picture in the embodiment of the present invention; Figure 9 is the schematic diagram of the effect after binarization processing of the longitudinal skin stretching machine jaw picture in the embodiment of the present invention.

[0182] Step S42: Perform wavelet fusion on the edge grayscale image obtained in this step S41, and extract the edge grayscale image curve. Specifically:

[0183] Step S421: Perform wavelet transform on the edge images obtained by the two algorithms, and perform multi-scale analysis. The decomposition expression is as follows:

[0184]

[0185] where, WT(q, τ) is the wavelet transform function; q is the scale; τ is the translation amount; t w is the time variable; f w is the first sub-function of wavelet transform; y is the second sub-function of wavelet transform.

[0186] Step S422: Merge the transform coefficients according to the local variance criterion. Let O(x, y) and P(x, y) be the high-frequency sub-image data values obtained by the mathematical morphology algorithm and the canny algorithm respectively, and F(x, y) be the corresponding high-frequency sub-image fusion value. Divide O(x, y) and P(x, y) into several M×N sub-block images. Perform numerical distribution statistics on each sub-block image and calculate its variance. Determine the weighting coefficients k1 and k2 of each sub-block image of the O and P images; if the variance of the sub-block image of the O image is greater than the variance of the sub-block image of the P image, then k1≥k2, otherwise k1<k2; determine the data fusion value of each sub-block image as follows:

[0187] F H (x, y) = k1O(x, y) + k2P(x, y);

[0188] In the formula, F H (x, y) is the corresponding high-frequency sub-image fusion value; O(x, y) is the high-frequency sub-image data value obtained by the mathematical morphology algorithm; P(x, y) is the high-frequency sub-image data value obtained by the canny algorithm; k1 is the weighting coefficient of each sub-block image of the O image; k2 is the weighting coefficient of each sub-block image of the P image.

[0189] Step S423: Perform inverse wavelet transform on the merged coefficients to obtain the final fused image, as shown in the following expression:

[0190]

[0191] Where x(t) w The final fused image is obtained by using the wavelet transform function; W w This is the third sub-function of the wavelet transform.

[0192] Step S5: Fit the edge grayscale image curve extracted in Step S4 using the least squares method; divide the edge grayscale image curve into multiple segments, obtain the curvature of each segment, and obtain the data coordinates (x, y) of each segment. m ,y m The fitted function is y = F(x); such that M = F(x) m )-y m In the process of calculating F(x), the sum of squared errors is minimized, as shown below:

[0193]

[0194] Where F(x) is the optimal function of the fitted curve; M is the fitting error; m is the data coordinate point number, m = 1, 2, 3, ..., n; n is the total number of data coordinate points; x m y is the ordinate of the m-th data point; m is the ordinate of the m-th data point; min is the function for finding the minimum value.

[0195] To find the optimal function F(x) for the fitted curve, it is necessary to determine the function type and divide the curve into multiple segments, in a preferred embodiment, 8 segments. By calculating the radius of curvature ρ of each segment, the bending angle Δθ of the mechanical structure can be obtained, as shown below:

[0196]

[0197] In the formula, ρ is the radius of curvature; l is the arc length of the curve; (x1,x2) is the range of the curve length on the X-axis in the camera coordinate system; Δθ is the bending angle of the mechanical structure.

[0198] By adjusting the stroke of the hydraulic cylinder for shaping the jaws of the stretching machine using the curve curvature extracted by the optimal function F(x), the longitudinal skin stretching machine jaw shaping control is achieved. Specifically, this includes:

[0199] Step S51: The expression for calculating the upward bend angle is as follows:

[0200]

[0201] Where, Δθ 上θ is the upward bending angle; θ0 is the horizontal bending angle; θ is the mechanical structure angle; Δθ 下 s is the downward bending angle; s0 is the initial position of the adjusting hydraulic cylinder; a is the first parameter of the mechanical structure; b is the second parameter of the mechanical structure.

[0202] The mechanical structure angle θ is calculated from the extracted mechanical structure angle bending angle Δθ using the two expressions above.

[0203] Step S52: By solving for the mechanical structure angle θ, derive the stroke s of the hydraulic cylinder. H The relationship is as follows:

[0204] θ = arccos(a 2 +b 2 -s H 2 ) / (2ab);

[0205] Step S53: Given the maximum and minimum stroke of each jaw, verify the stroke s of the hydraulic cylinder. H Is it in s min With s max between.

[0206] Step S54: Calculate the stroke of the hydraulic cylinder for adjusting each jaw and transmit it to the control system for control, so as to realize the adjustment of the jaw of the skin stretching machine.

[0207] like Figure 5 The diagram shown is a visual schematic of the entire working process of a skin stretching machine according to an embodiment of the present invention; the upward and downward bending angles of the jaws are controlled based on calculation results. The shaping process of this invention is faster, more accurate, and more convenient than traditional methods. The final shaping result for the longitudinal skin stretching machine jaws is as follows: Figure 10 As shown.

[0208] The beneficial effects of this invention are as follows: This invention provides a machine vision-based method for adjusting the jaw shape of a longitudinal skin stretching machine. Two sets of CCD cameras are respectively installed above the middle jaw, equipped with lenses and light sources. By adjusting the angles of the cameras and light sources, and adjusting the light source intensity, the entire image acquisition system is configured. Verification through embodiments proves that this method meets practical application requirements. This invention, through the use of machine vision, extracts and fits the curves of the blank end face and the mold end face. Based on the obtained curve curvature, the jaw shape is adjusted, improving the accuracy of skin stretching and increasing the efficiency of blank feeding into the jaw. This contributes to the automation and intelligent transformation and upgrading of aerospace-specific equipment.

[0209] Finally, it should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions or parameters described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision based control method for profile control of jaws of a longitudinal skin stretching machine, characterized by: It includes the following steps: S1: Install the image acquisition system; mount the CCD camera, lens, and light source above the jaws of the stretching machine, and adjust the mounting angles of the CCD camera and light source; S2: Calibrate the CCD camera to obtain the CCD camera's intrinsic parameter matrix A, extrinsic parameter matrix R, extrinsic parameter matrix t, and distortion coefficient A. The second distortion coefficient Establish the relationship between target pixels in the image and their corresponding locations in space, as shown below: ； in, The vertical coordinates of the target pixel in the image; These are the transformation coefficients for the x-coordinates of target pixels in the image; These are the transformation coefficients for the ordinates of target pixels in the image; This is the intrinsic parameter matrix of the camera; The first matrix of extrinsic parameters; The second matrix of extrinsic parameters; The x-coordinate of the target point in space; This represents the ordinate of the target point in space; The vertical coordinates of the corresponding target point in space; S3: Turn on the light source, acquire images captured by the CCD camera, and preprocess the images; S31: First, perform grayscale transformation on the image to convert the color image into a grayscale image. The expression for converting a color image to a grayscale image is as follows: ； in, The grayscale value of the grayscale image; The red value of the color image; The green value of the color image; is the blue value of the color image; i is the pixel x-coordinate number of the grayscale image; j is the pixel y-coordinate number of the grayscale image; S32: Perform geometric transformations on the grayscale image to adjust its size and position; S33: Histogram equalization is used to enhance grayscale images; S34: Noise is processed in the image by using a hybrid filtering method; S4: Extract edges from the preprocessed grayscale image; S41: The grayscale image is segmented by binarization. The grayscale image obtained by binarization is then processed by mathematical morphology and Canny edge detection. Finally, edge processing is performed on the grayscale image to make the edges of the grayscale image clearer. S42: Perform wavelet fusion on the edge grayscale image obtained in step S41 to extract the edge grayscale image curve; S5: Fit the edge grayscale image curve extracted in step S4 using the least squares method; divide the edge grayscale image curve into multiple segments, obtain the curvature of each segment, and obtain the data coordinates of each segment. The fitted function is ; make In seeking The process minimizes the sum of squared errors, as shown below: ； in, This is the optimal function for fitting the curve. This represents the fitting error; Number the data coordinate points. n represents the total number of data coordinate points; For the first The x-coordinate of each data point; For the first The ordinate of each data point; To find the minimum value of the function; Through the optimal function The extracted curve curvature is used to adjust the stroke of the hydraulic cylinder for adjusting the jaws of the stretching machine, thereby achieving longitudinal skin stretching machine jaw adjustment control.

2. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: In step S2, establishing the relationship between target pixels in the image and the corresponding target point positions in space specifically involves: S21: Camera intrinsic parameters describe the correspondence between points in the camera coordinate system and pixels in the pixel coordinate system, and are described by the following expression: ； in, This is the factor by which the assumed pixel coordinates are scaled on the horizontal axis; This is the factor by which the assumed pixel coordinates are scaled along the vertical axis; Rotation matrix with origin; The first rotation matrix is ​​the origin; The second rotation matrix is ​​the origin; The first translation matrix is ​​the one that points to the origin. The second translation matrix is ​​the one that moves the origin. The coordinates of the real-world point are [ ] T ; S22: Camera extrinsic parameters describe the correspondence between points in the world coordinate system and those in the camera coordinate system, and are expressed by the following expression: ； in, for Coordinates of the projection point in the camera coordinate system; S23: Use distortion coefficients to correct radial distortion generated during lens imaging and establish the relationship between target pixels in the image and the corresponding target points in space.

3. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: In step S32, geometric transformations are performed on the grayscale image, specifically including image translation, image rotation, and image scaling. The translation expression for the image is as follows: ； in, The position after translation; This is the initial point; It is a translation matrix; The x-coordinate of the movement vector; The vertical coordinate of the movement vector; The rotation expression for the image is as follows: ； in, The point after rotation; It is a rotation matrix; Let be the angle of rotation of the point about the origin; The scaling expression for the image is as follows: ； in, Here are the points after rotation; S is the scaling matrix. This represents the magnification factor along the x-axis; This represents the magnification factor along the y-axis.

4. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: In step S33, histogram equalization is used to enhance the grayscale image, specifically as follows: S331: Determine the gray levels of the grayscale image. The gray level value range is 0-255, and here we take 8 gray levels. The values ​​are 0, 1 / 7, 2 / 7, 3 / 7, 4 / 7, 5 / 7, 6 / 7, and 1; S332: Calculate the probability of the histogram of the original image and count the number of pixels for each gray level in the original grayscale image. And calculate its probability. The probability of obtaining the histogram of the original image; S333: Calculate the cumulative probability of the histogram probability. for: ； in, The sum of the histogram probabilities; To normalize the gray levels; The probability of the k-th gray level appearing; The frequency of the k-th gray level; is the total number of pixels in the original image; k is the number of gray levels, k∈[0-7]; S334: The pixel mapping relationship of the grayscale image is obtained as follows: ； in, This represents the pixel mapping relationship of a grayscale image; Grayscale level; To be a function that maximizes the value; This is an integer function; S335: Perform grayscale image enhancement processing based on the mapping relationship of grayscale images.

5. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: In step S34, noise processing of the image is performed using a hybrid filtering method, specifically as follows: S341: Let grayscale image If the pixel value at a given location is [value], then the median value of that pixel will be [value]. ;set up That is and median output The absolute difference between them; and To find the answer in the selected neighborhood. The two closest points are shown below: ； in, It is the nearest distance between a point in the first neighborhood and a pixel. It is the nearest distance between a point in the second neighborhood and a pixel. grayscale image Pixel value at; for The first neighbor point; for The second neighbor point; The mean distance; grayscale image Pixel values ​​and median output at [location] The absolute difference between them; grayscale image Output the pixel median at that location; S342: Set the first filter threshold Second filter threshold The value is 10, as shown below: ； in, The first filtering threshold; This is the second filtering threshold; The median gray level within the filtering window; A statistical function for calculating the median of a set of data; for The absolute difference between the gray value of a point and its median gray value; For pixels within the sliding pane; For the sake of A 5x5 sliding pane centered on the center; S343: If ,and The median filtering algorithm is used; if ,and The mean filtering algorithm is used; if The mean filtering algorithm is used to complete the filtering process of the grayscale image.

6. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: In step S41, the grayscale image is segmented by binarization, and the resulting grayscale image is then subjected to mathematical morphology processing and Canny edge detection processing, specifically: S411: Image segmentation is performed using binarization; threshold-based segmentation divides all pixels in the image into two categories based on grayscale changes, using a threshold as the boundary. A threshold is determined between the maximum and minimum thresholds in the image, resulting in two categories: those greater than the threshold and those less than the threshold. This yields the image binarization result. The expression for image binarization is shown below: ； in, It is a binary function; The set threshold; S412: Mathematical morphology algorithms include four basic operations: erosion, dilation, opening, and closing. Erosion and dilation can be combined to perform opening and closing operations. The expansion calculation process is as follows: ； in, for The first set; for The second set; It is an empty set; For the first set by the second set The expansion; For the dilation operator; ) represents the second set The position after the origin is translated; The corrosion calculation process is as follows: ； in, For the first set by the second set corrosion; For corrosion operators; S413: Using the Canny operator, Gaussian transform is used to denoise and smooth the image, preserving appropriate edges. Non-maximum suppression is employed, and high and low thresholds are set to effectively preserve edges. In a grayscale image, let it be... The Gaussian filtered image is The normal distribution equation in two-dimensional space is: ； in, The image is a Gaussian filtered image; The standard deviation is the normal distribution. The convolution operation is performed on the grayscale image as follows: ； in, This is the image obtained after performing a convolution operation on a grayscale image; The original grayscale image; After the convolution operation, the maximum value is used as the new grayscale value of the image center point; the orientation of the grayscale image is calculated through difference. and the amplitude of grayscale images As shown below: ； in, The orientation of the grayscale image; The amplitude of the grayscale image; This represents the gradient along the x-direction; Let be the gradient along the y-direction.

7. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: In step S42, wavelet fusion is performed on the edge grayscale image to extract the edge grayscale image curve, specifically as follows: S421: Perform wavelet transform on the edge images obtained by the two algorithms and conduct multi-scale analysis. The decomposition expression is shown below: ； in, It is a wavelet transform function; As a scale; This is the translation amount; It is a time variable; This is the first sub-function of the wavelet transform; This is the second sub-function of the wavelet transform; S422: Combine the transformation coefficients according to the local variance criterion, let... and These are the high-frequency sub-image data values ​​obtained by mathematical morphology algorithm and Canny algorithm, respectively. For the corresponding high-frequency sub-image fusion values, and Divided into several Sub-block images; perform numerical distribution statistics on each sub-block image and calculate its variance; determine the weighting coefficients for each sub-block image of the O and P images. and If the variance of the O image sub-block is greater than the variance of the P image sub-block, then ≥ ,otherwise < Determine the data fusion values ​​for each sub-block image as follows: ； In the formula, These are the corresponding high-frequency sub-image fusion values; These are the high-frequency sub-image data values ​​obtained by mathematical morphology algorithms; High-frequency sub-image data values ​​obtained by the Canny algorithm; These are the weighting coefficients for each sub-block of the O-image; These are the weighting coefficients for each sub-block of the P-image; S423: Perform inverse wavelet transform on the merged coefficients to obtain the final fused image, as shown in the following expression: ； in, The final fused image is obtained by using the wavelet transform function; This is the third sub-function of the wavelet transform.

8. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 1, characterized in that: The optimal function of the fitted curve function in step S5 To determine the function type, the curve is divided into 8 equal segments, and the radius of curvature of each segment is calculated. It can determine the angular bending angle of a mechanical structure. As shown below: ； In the formula, The radius of curvature; The arc length of the curve; () represents the range of the curve length along the X-axis in the camera coordinate system; This refers to the bending angle of the mechanical structure.

9. The machine vision-based jaw shaping control method for longitudinal skin stretching machine according to claim 8, characterized in that: Step S5 implements the jaw shaping control of the longitudinal skin stretching machine, specifically including: S51: The expression for calculating the upward bend angle is as follows: ； in, The upward bend angle; It is a horizontal bend. For mechanical structure angles; The downward bend angle; This is the initial position of the adjusting hydraulic cylinder; This is the first parameter of the mechanical structure; This is the second parameter of the mechanical structure; From the above expression, the mechanical structure angular bending angle is extracted. Calculate the mechanical structure angle ; S52: Mechanical structure angles obtained through solution The stroke of the hydraulic cylinder is derived. The relationship is: ； S53: The maximum and minimum stroke of each jaw are known; verify the stroke of the hydraulic cylinder. Is it in and between; S54: Calculate the stroke of the hydraulic cylinder for adjusting each jaw and transmit it to the control system for control, thereby enabling the jaw adjustment of the skin stretching machine.

Images