A visual quality inspection and machining precision closed-loop control method and system of a smart manufacturing unit

Abstract

The application provides a kind of visual quality inspection and machining precision closed-loop control method and system of intelligent manufacturing unit, specifically collect the image of workpiece to be inspected, obtain initial target frame by target detection network, segment workpiece image to obtain pixel mask and contour, calculate the minimum area of contour as size reference frame as circumscribed quadrilateral;Based on the matching relationship between the actual feature points in the reference frame and the preset template, calculate the homography matrix to correct the perspective distortion, get the corrected contour and the standard size frame;Sample along the normal direction of each side of the standard size frame to get edge point set, fit high-precision edge with random sampling consistency algorithm and calculate the proportion of in-point, if the proportion does not meet the standard, increase the sampling density and re-fit, compare the fitted workpiece size with the planned size, and construct the machining deviation state matrix;Combine historical deviation matrix and compensation parameters to update the transfer matrix of machining process online, generate compensation parameters through current deviation matrix and send to unit controller.

Description

A visual quality inspection and closed-loop control method and system for processing accuracy in intelligent manufacturing units. Technical Field

[0001] This application belongs to the field of control, and in particular relates to a closed-loop control method and system for visual quality inspection and processing accuracy of intelligent manufacturing units. Background Technology

[0002] Conventional quality inspection methods rely on offline sampling using contact gauges or coordinate measuring machines (CMMs). Offline inspections have long feedback cycles, making real-time intervention and compensation during processing difficult. In practical industrial applications, image-based non-contact online inspection faces challenges. For example, camera installation angles and lens distortion can cause perspective distortion, distorting the dimensional information extracted from 2D images. Reflections, oil stains, burrs on the workpiece surface, and changes in ambient lighting can interfere with edge feature extraction. Constructing a closed-loop quality control system for processing accuracy, which provides real-time feedback of online inspection results to the processing equipment controller to adjust processing parameters, can suppress processing deviations at their inception. However, existing closed-loop control strategies rely on simple PID control or static error compensation models, calculating a fixed compensation amount based on the current single measurement deviation. Static compensation models are ineffective when faced with continuous, gradual error accumulation and may even lead to oscillations. Therefore, integrating visual precision measurement technology with an online-updable process control model to establish a closed-loop control method capable of sensing and proactively compensating for changes in the processing process is a critical technical challenge for achieving stable production in intelligent manufacturing units. Summary of the Invention

[0003] This invention proposes a closed-loop control method for visual quality inspection and processing accuracy in intelligent manufacturing units, which addresses the problem that existing technologies struggle to integrate visual precision measurement technology with online-updable process control models, and are unable to perceive and proactively compensate for changes in the processing process. The method includes:

[0004] An image of the workpiece to be inspected is acquired, an initial target bounding box is obtained through a target detection network, and the workpiece is segmented within the initial target bounding box to obtain a pixel mask and contour; the minimum area circumscribed quadrilateral of the contour is calculated as the size reference box.

[0005] Based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template, a homography matrix for correcting perspective distortion is calculated; the homography matrix is ​​then used to perform perspective transformation on the contour and dimensional reference frame to obtain the corrected contour and standard dimensional frame.

[0006] For each edge of the standard size frame, edge point sets are obtained by sampling along the edge normal direction on the correction contour. The edge point sets are then fitted with straight lines using a random sampling consensus algorithm to obtain fitted edges, and the proportion of inlier points after fitting is calculated. When the proportion of inlier points on any side is lower than a first threshold, the sampling point density along the edge normal direction is increased and the straight line fitting is re-executed. The workpiece dimensions determined by all fitted edges are compared with the planned nominal dimensions to construct a processing deviation state matrix.

[0007] Based on the historical cycle's processing deviation state matrix and processing accuracy compensation parameters, the transfer matrix used to represent the characteristics of the processing process is updated online; the current cycle's processing deviation state matrix and the updated transfer matrix are calculated to generate the current cycle's processing accuracy compensation parameters, which are then sent to the controller of the intelligent manufacturing unit.

[0008] Furthermore, this invention also relates to a closed-loop control system for visual quality inspection and processing accuracy of an intelligent manufacturing unit, comprising the following modules:

[0009] The first calculation module is used to acquire an image of the workpiece to be inspected, obtain an initial target box through a target detection network, and segment the workpiece within the initial target box to obtain a pixel mask and contour; and calculate the minimum area circumscribed quadrilateral of the contour as a size reference box.

[0010] The second calculation module is used to calculate the homography matrix for correcting perspective distortion based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template; and to perform perspective transformation on the contour and dimensional reference frame using the homography matrix to obtain the corrected contour and standard dimensional frame.

[0011] The module is used to sample edge point sets on the correction contour along the edge normal direction for each edge of the standard size frame, perform straight line fitting on the edge point set using a random sampling consensus algorithm to obtain fitted edges, and calculate the proportion of inlier points after fitting; when the proportion of inlier points on any side is lower than a first threshold, the sampling point density along the edge normal direction is increased and the straight line fitting is re-executed; the workpiece size determined by all fitted edges is compared with the planned nominal size to construct a processing deviation state matrix;

[0012] The sending module is used to update the transfer matrix representing the characteristics of the machining process online based on the machining deviation state matrix and machining accuracy compensation parameters of the historical cycle; it performs calculations on the machining deviation state matrix of the current cycle and the updated transfer matrix to generate the machining accuracy compensation parameters of the current cycle, and sends them to the controller of the intelligent manufacturing unit.

[0013] This invention corrects perspective distortion caused by camera shooting angle by constructing a dimensional reference frame and calculating the homography matrix. It uses a random sampling consensus algorithm to fit edge points and adjusts the sampling strategy based on the fitting quality to extract high-precision workpiece edges from noisy contours. In the control stage, a transfer matrix model combining historical processing deviations and compensation effects is established based on the current measurement deviation. By continuously updating this matrix, the generation of compensation parameters can take into account the influence of slowly changing factors on the processing, generating more predictive compensation instructions and improving the processing accuracy and product consistency of the intelligent manufacturing unit. Attached Figure Description

[0014] Figure 1 is a flowchart of the first embodiment;

[0015] Figure 2 is a schematic diagram for calculating the circumscribed quadrilateral with the minimum area;

[0016] Figure 3 is a schematic diagram of linear fitting using RANSAC;

[0017] Figure 4 is a schematic diagram of the generation and closed-loop control of machining accuracy compensation parameters. Detailed Implementation

[0018] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0019] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0020] In the first embodiment, the present invention proposes a closed-loop control method for visual quality inspection and processing accuracy of an intelligent manufacturing unit, as shown in Figure 1, including:

[0021] S1. Acquire an image of the workpiece to be inspected, obtain an initial target bounding box through a target detection network, and segment the workpiece within the initial target bounding box to obtain a pixel mask and contour; calculate the minimum area circumscribed quadrilateral of the contour as the size reference box.

[0022] A camera is installed above the production line to capture grayscale or color images of the workpieces. Using a pre-trained target detection model on a large number of workpiece samples, the workpieces in the images are quickly located, and an initial bounding box is output. The image region within this initial bounding box is taken as the region of interest and input into a semantic segmentation network. This network outputs a binary pixel mask, where the pixel value of the workpiece region is 1, and the background is 0. The Suzuki contour finding algorithm is applied to this pixel mask to extract the outermost closed contour point set. For example, in a workpiece binarized pixel mask with a resolution of 1920×1080, the pixel value of the workpiece area is 1 and the background area is 0. First, a 4-neighborhood connectivity rule is set. Starting from the top left corner (0,0) of the mask image, all pixels are traversed row by row and column by column. When the first boundary pixel with a value of 1 that has not been marked for access is detected, it is used as the starting point for contour tracing. According to the boundary pixel judgment rule of the Suzuki algorithm, the state of the 8 neighboring pixels of the current pixel is checked in turn. The continuous boundary pixels are tracked in a clockwise direction, and the coordinates (x,y) of each pixel are recorded in real time until the tracking path returns to the starting point to form a closed contour. After the traversal is completed, the secondary contours caused by small noise or holes inside the mask are removed by the contour hierarchy judgment mechanism of the algorithm, and only the outermost closed contour with the largest area is retained.

[0023] Using the obtained contour point set as input, the rotation caliper algorithm is applied to... The angle step is iteratively rotated to fit the parallel caliper line to the contour edge. The area of ​​the circumscribed rectangle at each angle is calculated and the minimum value is selected. The coordinates of the four vertices of the circumscribed rectangle with the minimum area are output as the size reference box, as shown in Figure 2.

[0024] In an optional embodiment, acquiring an image of the workpiece to be inspected, obtaining an initial target bounding box through a target detection network, and segmenting the workpiece within the initial target bounding box to obtain a pixel mask and contour includes:

[0025] The acquired image of the workpiece to be inspected is input into the YOLOv5 target detection network, and the target bounding boxes with a confidence score greater than 0.9 are output as the initial target bounding boxes;

[0026] The image region within the initial target bounding box is input into the U-Net segmentation network, which outputs a grayscale probability map of the workpiece.

[0027] The grayscale probability map is binarized with a threshold of 128 to obtain a pixel mask, and the Suzuki algorithm is used to extract the outermost contour of the pixel mask.

[0028] Specifically, a 1920×1080 pixel image of the rectangular workpiece to be inspected, captured by a camera, is input into a pre-trained YOLOv5s object detection network model. This network, through a backbone network, neck network, and head detection network structure, quickly processes the image and outputs the detection results. For example, the network identifies the workpiece and provides a target bounding box with a confidence score of 0.95 and coordinates [450, 300, 1050, 800]. This bounding box serves as the initial target bounding box, roughly locating the workpiece. The YOLOv5s object detection network structure mainly consists of three parts: a backbone network, a neck network, and a head network. The backbone network uses a CSPDarknet53 structure to extract feature maps at different levels from the input image. The neck network uses a PANet path aggregation network structure, fusing the multi-scale feature maps output by the backbone network to enhance the model's feature representation capability. The head network, or YOLO Head, predicts on the fused feature maps, outputting the bounding box's position, confidence score, and class probability. The network model was pre-trained on the COCO general dataset and fine-tuned on a dataset containing 5000 workpiece images acquired under different lighting and angles. The training process used the Adam optimizer with an initial learning rate of 0.001, a batch size of 16, and a total of 100 iterations. The loss function included bounding box regression loss, confidence loss, and classification loss. The network input was an RGB image of the workpiece to be inspected, normalized to 640×640 pixels. The network output was a series of detection boxes, each containing four coordinate values, a target confidence score, and a class label.

[0029] A 600×500 pixel image region defined by an initial bounding box is cropped from the original image and scaled to a standard size of 256×256 pixels. This image is then fed into a U-Net segmentation network, which consists of an encoder downsampling path for feature extraction and a decoder upsampling path for pixel location recovery, fusing deep and shallow features through skip connections. The network outputs a 256×256 grayscale probability map, where each pixel's grayscale value ranges from 0 to 255, representing the probability that the pixel belongs to the workpiece. This probability map is binarized using a fixed threshold of 128; pixels with values ​​greater than 128 are converted to 255 (foreground), and others to 0 (background), forming a clear binary pixel mask. The Suzuki algorithm is applied to this mask, employing a 4-neighborhood connectivity rule and a filtering strategy that removes secondary contours while retaining the outermost closed contour. Only the outermost closed contour is tracked and extracted to obtain the workpiece edge pixel coordinate sequence.

[0030] In some embodiments, the U-Net segmentation network employs a classic encoder-decoder symmetric structure. The encoder path consists of multiple downsampling modules, each containing two 3×3 convolutional layers followed by a 2×2 max-pooling layer for detecting contextual features of the image. The decoder path progressively restores the image resolution and details through multiple upsampling modules, each containing a 2×2 deconvolutional layer, a skip connection concatenating the feature map of the corresponding encoder layer, and two 3×3 convolutional layers. The skip connection structure combines deep semantic information with shallow detail information. The final layer of the network is a 1×1 convolutional layer that outputs a pixel-level segmentation probability map through a sigmoid activation function. The network is trained using a dataset containing 2000 slices of workpiece images and their corresponding binary masks. Training uses the Adam optimizer with a learning rate of 0.0001 and the Dice loss function. The network input is the image region within the target bounding box output by YOLOv5, scaled to 256×256 pixels. The network output is a single-channel grayscale probability map with the same size as the input.

[0031] S2, Based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template, calculate the homography matrix used to correct perspective distortion; use the homography matrix to perform perspective transformation on the contour and dimensional reference frame to obtain the corrected contour and standard dimensional frame.

[0032] Extract the ideal coordinates of four or more stable feature points from the workpiece's CAD drawing, such as the four corner points of the workpiece, to form a feature point template. Within the area defined by the dimensional reference frame of the camera-acquired image, use Harris or Shi-Tomasi corner detection algorithms to locate the actual workpiece feature points. Establish a one-to-one correspondence between the actually detected feature points and the template feature points through positional relationships or feature descriptors. Using the at least four pairs of matched two-dimensional coordinate points, solve for a 3×3 homography matrix using a linear transformation algorithm.

[0033] Each point on the original contour and the coordinates of the four vertices of the dimensional reference box are represented as column vectors in homogeneous coordinate form. Each column vector is multiplied on the left by the homography matrix calculated in the previous step to obtain the transformed homogeneous coordinate vector. The transformed homogeneous coordinate vector is then converted into two-dimensional Cartesian coordinates to obtain a corrected contour with shape and pose correction and no perspective distortion, as well as a rectangle aligned with the image coordinate axes, i.e., the standard dimensional box.

[0034] In an optional embodiment, the step of calculating the homography matrix for correcting perspective distortion based on the matching relationship between actual feature points detected within the dimensional reference frame and a preset workpiece feature point template includes:

[0035] The ORB operator is used to detect at least 50 actual feature points within the dimensional reference frame and to perform Hamming distance matching with the workpiece feature point template.

[0036] Lowe's ratio test was used to eliminate mismatched pairs and select high-quality matching pairs with a matching distance ratio of less than 0.75.

[0037] A 3×3 homography matrix is ​​solved iteratively using the RANSAC algorithm with at least 4 high-quality matching point pairs.

[0038] A pre-defined workpiece feature point template is loaded. This template is extracted from an ideal image generated from the workpiece's CAD drawing and contains 100 ORB feature points and their corresponding binary descriptors. After acquiring the workpiece image, the ORB operator is applied within the dimensional reference frame region to detect actual feature points and calculate descriptors. For example, using an initialized ORB feature detector, the maximum number of feature points is set to 100, the scale pyramid layer number to 8, the scale factor to 1.2, and the edge threshold to 15. This detector uses the FAST algorithm to locate corner points and determine their principal direction, and then uses an improved BRIEF algorithm to generate a 256-bit binary descriptor for each feature point. To improve feature point stability, feature points located within 5 pixels of the dimensional reference frame edge and false feature points misdetected into the background mask region are further removed, resulting in actual feature points. Each feature point contains two-dimensional coordinates and a corresponding 256-bit descriptor.

[0039] Calculate the Hamming distance between each actual feature point descriptor and all 100 descriptors in the template, which is the number of different bits in the binary string, and find the two closest and second closest template feature points for each actual feature point.

[0040] To ensure the reliability of the matching, the Lowe ratio test is used for screening. For each match, the Hamming distance between the nearest and second nearest neighbors is divided to obtain a ratio value. For example, if the best matching distance for an actual feature point is 30 and the second best matching distance is 45, the ratio is approximately 0.67, which is less than the threshold of 0.75. Therefore, this match is considered a high-quality match and is retained. If the ratio is 0.8, the match is discarded. After screening, assuming 40 high-quality match pairs are obtained, the homography matrix is ​​estimated using the RANSAC algorithm. In multiple iterations, the algorithm randomly selects 4 match pairs each time to calculate a candidate 3×3 homography matrix and uses this matrix to test the projection error of all match pairs. The homography matrix that satisfies the projection error threshold for the most match pairs is selected as the solution for correcting perspective distortion.

[0041] In an optional embodiment, the obtained 3×3 homography matrix H is used to perform geometric correction on the original workpiece contour and dimensional reference frame. Specifically, the coordinates of all pixels on the contour (a total of N points) and the coordinates of the four vertices of the dimensional reference frame are formatted as homogeneous coordinates, i.e., for a point (x, y), its homogeneous coordinates are a column vector of (x, y, 1). The homography matrix H is then multiplied by each homogeneous coordinate column vector. , These are the original two-dimensional points. The position in the homogeneous coordinate system after perspective transformation. Then, the corrected two-dimensional coordinates are calculated through back projection from homogeneous coordinates to Cartesian coordinates: , These are the corrected actual two-dimensional coordinates. After this transformation, all N contour points form a corrected contour with corrected pose and eliminated perspective distortion. Simultaneously, the four vertices of the dimension reference box, after the same transformation, will form a standard rectangle that is strictly parallel to the image coordinate axes, i.e., the standard dimension box.

[0042] S3, For each edge of the standard size frame, sample the edge point set on the correction contour along the edge normal direction to obtain the edge point set, use the random sampling consensus algorithm to perform straight line fitting on the edge point set to obtain the fitted edge, and calculate the proportion of the fitted interior points; when the proportion of the interior points on any side is lower than the first threshold, increase the sampling point density along the edge normal direction and re-execute the straight line fitting; compare the workpiece size determined by all fitted edges with the planned nominal size to construct the processing deviation state matrix;

[0043] Taking the top edge of a standard-sized frame as an example, this edge is a horizontal line. Along this horizontal line, at fixed pixel intervals (e.g., 5 pixels), a search line is emitted in the normal direction (vertically downwards). The first intersection point of this search line with the corrected contour is recorded, and all intersection points are collected to form the point set of the top edge. A random sampling consensus algorithm is applied, and two points are randomly selected from this point set to form a candidate line. The number of points in the point set whose distance to this line is less than a preset small tolerance is calculated; these points are called inliers. This process is repeated hundreds of times, and the candidate line with the most inliers is selected as the optimal fit. All inliers are used to refit the edge using the least squares method. The proportion of inliers to the total number of points in the point set is calculated. If this proportion is lower than a preset first threshold (e.g., 0.9), the sampling interval is reduced from 5 pixels to 2 pixels to obtain a denser edge point set, and the above fitting process is repeated.

[0044] Based on the equations of the four fitted sides, the distances between them are calculated to obtain the actual measured length and width of the workpiece. These measured lengths and widths are then subtracted from the nominal lengths and widths obtained from the CAD drawings to obtain the length deviation and width deviation values. These two deviation values ​​are then used as a two-dimensional column vector to form the machining deviation state matrix for the current machining cycle.

[0045] In an optional embodiment, the step of sampling edge points along the normal direction of the edge on the corrected contour for each edge of the standard-sized frame to obtain an edge point set, performing line fitting on the edge point set using a random sampling consensus algorithm to obtain a fitted edge, and calculating the proportion of the fitted interior points includes:

[0046] For each edge of the standard size frame, within a strip area of ​​±5 pixels on both sides of the edge, samples are taken at 1-pixel intervals along the normal direction of the edge, and the intersection points with the correction contour are recorded to form an edge point set.

[0047] The random sampling consensus algorithm is used on the edge point set. Each time, two points are randomly selected to form a candidate line. The distance from other points in the point set to the line is calculated. Points with a distance of less than 0.5 pixels are recorded as interior points. After 100 iterations, the candidate line with the most interior points is selected as the fitted edge.

[0048] Specifically, taking the top edge of a standard-sized frame as an example, let's assume it's a horizontal line segment from point (100, 50) to point (300, 50). Define a search area, a horizontal strip with y-coordinates ranging from 45 to 55. Starting at x=100, scan along the direction perpendicular to the normal of the edge, i.e., the y-axis direction, at 1-pixel intervals until x=300. At each x-coordinate position, for example, x=152, search for the intersection point of the corrected workpiece contour within the range of y=45 to y=55. If an intersection point with coordinates (152, 51.5) is found, add that point to the edge point set. After scanning the entire edge, an edge point set containing approximately 201 contour points is obtained.

[0049] The RANSAC algorithm is applied to the edge point set for line fitting. This process is repeated 100 times. In each iteration, two points are randomly selected from the point set, for example, (125, 51.3) and (280, 51.8), and a candidate line equation is determined based on these two points. The perpendicular distance from all other points in the edge point set to this candidate line is calculated. All points with a distance less than 0.5 pixels are marked as inliers. After 100 iterations, the number of inliers in the candidate lines generated in each iteration is compared, and the line with the most inliers, for example, the line with 198 inliers, is selected as the fitted edge, as shown in Figure 3.

[0050] Suppose that after fitting a straight line to an edge, the initial sampling yields 200 edge points. After the RANSAC algorithm finishes running, the best-fit line finds 160 interior points. The proportion of interior points is calculated to be 80%.

[0051] In an optional embodiment, the step of increasing the sampling point density in the edge normal direction and re-performing line fitting when the proportion of interior points on any side is lower than a first threshold includes:

[0052] When the calculated number of interior points is less than the first threshold, the sampling interval of the edge normal direction is reduced from 1 pixel to 0.5 pixels, a new edge point set is obtained by resampling, and line fitting is performed again. This process is repeated up to 3 times.

[0053] Assuming the inlier ratio is 80%, since this ratio is below a preset first threshold of 85%, the fitting quality is considered poor, possibly due to sparse sampling points caused by edge jaggedness or slight curvature, triggering a retry mechanism. The current edge point set and fitting result are discarded, and the sampling process is re-executed on the same edge. In this new sampling, the sampling interval is reduced from 1 pixel to 0.5 pixels. New sampling lines are added between the previously sampled points, doubling the total number of sampling points to obtain a new edge point set of approximately 400 points. RANSAC line fitting is then performed again on this dense point set. If the inlier ratio is still below 85%, this process is repeated, with the sampling interval further reduced, limiting the total number of retries to 3 to ensure algorithm efficiency.

[0054] In an optional embodiment, the step of comparing the workpiece dimensions determined by all fitted edges with the planned nominal dimensions to construct a machining deviation state matrix includes:

[0055] Based on the four fitted edges, calculate the actual length, width, and four interior angles of the workpiece;

[0056] The actual length, width, and four interior angles are subtracted from the nominal length, width, and nominal angles on the planning drawings to obtain the length deviation, width deviation, and four angle deviations. These six deviation values ​​are then used as a 6×1 column vector to form the processing deviation state matrix for the current cycle. .

[0057] Specifically, after obtaining the equations of the lines representing the four sides of the workpiece, the intersection points of these lines are calculated to determine the coordinates of the four vertices of the workpiece. Based on these vertex coordinates, the actual dimensions of the workpiece can be calculated. For example, by calculating the average distance between two opposite sides, the actual length is found to be 100.12 mm and the actual width to be 49.95 mm. By calculating the included angle between the equations of adjacent lines, the actual angle values ​​of the four interior angles are obtained, for example, 90.05°, 89.98°, 90.02°, and 89.95°.

[0058] The planned nominal dimensions of the workpiece are retrieved from the database, such as a nominal length of 100.00 mm, a nominal width of 50.00 mm, and all nominal interior angles of 90.00°. By subtracting the nominal values ​​from the actual measured values, the following deviations are calculated: length deviation is +0.12 mm, width deviation is -0.05 mm, and the four angle deviations are +0.05°, -0.02°, +0.02°, and -0.05°, respectively. These six deviation values ​​are organized into a 6×1 column vector, which is the machining deviation state matrix for the current period k. .

[0059] S4. Based on the historical cycle's processing deviation state matrix and processing accuracy compensation parameters, the transfer matrix used to represent the characteristics of the processing process is updated online; the current cycle's processing deviation state matrix and the updated transfer matrix are calculated to generate the current cycle's processing accuracy compensation parameters, which are then sent to the controller of the intelligent manufacturing unit.

[0060] A recursive least squares algorithm with a forgetting factor is employed. The machining deviation state matrix from the previous cycle, the compensation parameters sent in the previous cycle, and the machining deviation state matrix measured in the current cycle are used as inputs to iteratively update a transfer matrix representing the relationship between compensation and deviation. This forgetting factor allows the algorithm to focus on recent machining data, thus tracking slow changes. The machining deviation state matrix of the current cycle is multiplied by the inverse or pseudo-inverse of the updated transfer matrix, and the result is negative to calculate the compensation parameters that can offset the current deviation, such as tool offset compensation values ​​for the X and Z axes. These compensation values ​​are then sent to the machine tool's CNC system via an industrial communication protocol for execution in the machining program of the next workpiece, as shown in Figure 4.

[0061] In an optional embodiment, the online updating of the transfer matrix representing the characteristics of the machining process, based on the historical cycle-based machining deviation state matrix and machining accuracy compensation parameters, includes:

[0062] The recursive least squares algorithm with a forgetting factor is used, based on the processing deviation state matrix of the current cycle. Machining accuracy compensation parameters from the previous cycle For the transfer matrix Perform online updates;

[0063] The forgetting factor is set to enhance the weight of new data in the update process.

[0064] It is an m×n dimensional real matrix, where m is the processing deviation state matrix. The dimension of n is the machining accuracy compensation parameter vector. The dimension; each element in the matrix The physical meaning is the influence coefficient of the unit adjustment of the j-th compensation parameter on the i-th processing deviation. Transfer matrix This illustrates how machining accuracy compensation parameters applied to the machine tool affect the dimensional and angular deviations of the workpiece. Since the machine tool's condition changes slowly due to tool wear and temperature variations, the transfer matrix needs continuous updating to maintain accuracy. This embodiment employs a recursive least squares method with a forgetting factor to implement the online update process. This algorithm does not require storing all historical data; instead, it iterative updates are based on the model from the previous period and the new data from the current period, resulting in high computational efficiency.

[0065] At the end of processing cycle k, the processing deviation state matrix of the current workpiece is obtained. Simultaneously, the machining accuracy compensation parameters from the previous cycle used before machining the workpiece were also recorded. . use and The two latest input-output data pairs, and the transfer matrix of the previous cycle. The recursive least squares algorithm calculates a new transfer matrix. Among them, the forgetting factor Set to 0.98, during the update process, the latest data... It will receive the largest weight, while the influence of historical data will decay exponentially at a rate of 0.98, allowing the model to adapt to changes in the processing.

[0066] In an optional embodiment, the step of performing calculations on the current cycle's processing deviation state matrix and the updated transfer matrix to generate processing accuracy compensation parameters for the current cycle, and sending these parameters to the controller of the intelligent manufacturing unit, includes:

[0067] Based on the following formula, combined with the machining accuracy compensation parameters from the previous cycle... Calculate the machining accuracy compensation parameters for the current cycle. :

[0068]

[0069] in, For the updated transfer matrix The pseudo-inverse matrix, This is the processing deviation state matrix for the current cycle;

[0070] The machining accuracy compensation parameters It is a vector containing compensation values ​​for the X-axis, Y-axis, and rotary axes of the machine tool, which is sent to the PLC controller of the intelligent manufacturing unit via the Modbus TCP / IP protocol.

[0071] Specifically, using the processing deviation state matrix that was just measured... For example, a vector containing a length deviation of +0.12mm and a width deviation of -0.05mm, and the online updated transfer matrix. To calculate the compensation parameters for the next processing cycle, the first step in the calculation process is to solve for the transfer matrix. pseudo-inverse matrix Even Even if the matrix is ​​not square or non-invertible, an optimal linear solution can still be found.

[0072] pseudo-inverse matrix Deviation matrix Multiplying them together yields a correction vector. This vector represents the vector used to eliminate the current bias. The required ideal compensation adjustment amount. Based on the formula, the compensation parameters from the previous cycle... Subtract the correction vector to calculate the new machining accuracy compensation parameters. For example, if Given [X:0.1, Y:-0.2], and the calculated correction value is [X:0.12, Y:-0.05], then the new compensation parameters are... The new compensation parameter vector will be [X:-0.02, Y:-0.15]. This data, containing specific compensation values ​​for the machine tool's X, Y, and rotary axes, is encapsulated into data packets conforming to the Modbus TCP / IP protocol and sent to the PLC controller in the production unit via Ethernet. Upon receiving this data, the PLC applies these values ​​as offsets to the corresponding coordinate axis control when executing the machining program for the next workpiece, achieving closed-loop compensation for machining accuracy.

[0073] In the second embodiment, the present invention also proposes a closed-loop control system for visual quality inspection and processing accuracy of an intelligent manufacturing unit, comprising the following modules:

[0074] The first calculation module is used to acquire an image of the workpiece to be inspected, obtain an initial target box through a target detection network, and segment the workpiece within the initial target box to obtain a pixel mask and contour; and calculate the minimum area circumscribed quadrilateral of the contour as a size reference box.

[0075] The second calculation module is used to calculate the homography matrix for correcting perspective distortion based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template; and to perform perspective transformation on the contour and dimensional reference frame using the homography matrix to obtain the corrected contour and standard dimensional frame.

[0076] The module is used to sample edge point sets on the correction contour along the edge normal direction for each edge of the standard size frame, perform straight line fitting on the edge point set using a random sampling consensus algorithm to obtain fitted edges, and calculate the proportion of inlier points after fitting; when the proportion of inlier points on any side is lower than a first threshold, the sampling point density along the edge normal direction is increased and the straight line fitting is re-executed; the workpiece size determined by all fitted edges is compared with the planned nominal size to construct a processing deviation state matrix;

[0077] The sending module is used to update the transfer matrix representing the characteristics of the machining process online based on the machining deviation state matrix and machining accuracy compensation parameters of the historical cycle; it performs calculations on the machining deviation state matrix of the current cycle and the updated transfer matrix to generate the machining accuracy compensation parameters of the current cycle, and sends them to the controller of the intelligent manufacturing unit.

[0078] In an optional embodiment, acquiring an image of the workpiece to be inspected, obtaining an initial target bounding box through a target detection network, and segmenting the workpiece within the initial target bounding box to obtain a pixel mask and contour includes:

[0079] The acquired image of the workpiece to be inspected is input into the YOLOv5 target detection network, and the target bounding boxes with a confidence score greater than 0.9 are output as the initial target bounding boxes;

[0080] The image region within the initial target bounding box is input into the U-Net segmentation network, which outputs a grayscale probability map of the workpiece.

[0081] The grayscale probability map is binarized with a threshold of 128 to obtain a pixel mask, and the Suzuki algorithm is used to extract the outermost contour of the pixel mask.

[0082] In an optional embodiment, the step of calculating the homography matrix for correcting perspective distortion based on the matching relationship between actual feature points detected within the dimensional reference frame and a preset workpiece feature point template includes:

[0083] The ORB operator is used to detect at least 50 actual feature points within the dimensional reference frame and to perform Hamming distance matching with the workpiece feature point template.

[0084] Lowe's ratio test was used to eliminate mismatched pairs and select high-quality matching pairs with a matching distance ratio of less than 0.75.

[0085] A 3×3 homography matrix is ​​solved iteratively using the RANSAC algorithm with at least 4 high-quality matching point pairs.

[0086] In an optional embodiment, the step of sampling edge points along the normal direction of the edge on the corrected contour for each edge of the standard-sized frame to obtain an edge point set, and then using a random sampling consensus algorithm to perform line fitting on the edge point set to obtain a fitted edge, includes:

[0087] For each edge of the standard size frame, within a strip area of ​​±5 pixels on both sides of the edge, samples are taken at 1-pixel intervals along the normal direction of the edge, and the intersection points with the correction contour are recorded to form an edge point set.

[0088] The random sampling consensus algorithm is used on the edge point set. Each time, two points are randomly selected to form a candidate line. The distance from other points in the point set to the line is calculated. Points with a distance of less than 0.5 pixels are recorded as interior points. After 100 iterations, the candidate line with the most interior points is selected as the fitted edge.

[0089] In an optional embodiment, the step of increasing the sampling point density in the edge normal direction and re-performing line fitting when the proportion of interior points on any side is lower than a first threshold includes:

[0090] When the calculated number of interior points is less than the first threshold, the sampling interval of the edge normal direction is reduced from 1 pixel to 0.5 pixels, a new edge point set is obtained by resampling, and line fitting is performed again. This process is repeated up to 3 times.

[0091] In an optional embodiment, the step of comparing the workpiece dimensions determined by all fitted edges with the planned nominal dimensions to construct a machining deviation state matrix includes:

[0092] Based on the four fitted edges, calculate the actual length, width, and four interior angles of the workpiece;

[0093] The actual length, width, and four interior angles are subtracted from the nominal length, width, and nominal angles on the planning drawings to obtain the length deviation, width deviation, and four angle deviations. These six deviation values ​​are then used as a 6×1 column vector to form the processing deviation state matrix for the current cycle. .

[0094] In an optional embodiment, the online updating of the transfer matrix representing the characteristics of the machining process, based on the historical cycle-based machining deviation state matrix and machining accuracy compensation parameters, includes:

[0095] The recursive least squares algorithm with a forgetting factor is used, based on the processing deviation state matrix of the current cycle. Machining accuracy compensation parameters from the previous cycle For the transfer matrix Perform online updates;

[0096] The forgetting factor is set to enhance the weight of new data in the update process.

[0097] In an optional embodiment, the step of performing calculations on the current cycle's processing deviation state matrix and the updated transfer matrix to generate processing accuracy compensation parameters for the current cycle, and sending these parameters to the controller of the intelligent manufacturing unit, includes:

[0098] Based on the following formula, combined with the machining accuracy compensation parameters from the previous cycle... Calculate the machining accuracy compensation parameters for the current cycle. :

[0099]

[0100] in, For the updated transfer matrix The pseudo-inverse matrix, This is the processing deviation state matrix for the current cycle;

[0101] The machining accuracy compensation parameters It is a vector containing compensation values ​​for the X-axis, Y-axis, and rotary axes of the machine tool, which is sent to the PLC controller of the intelligent manufacturing unit via the Modbus TCP / IP protocol.

[0102] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A closed-loop control method for visual quality inspection and processing accuracy of an intelligent manufacturing unit, characterized in that, Includes the following steps: An image of the workpiece to be inspected is acquired, an initial target bounding box is obtained through a target detection network, and the workpiece is segmented within the initial target bounding box to obtain a pixel mask and contour. The minimum area circumscribed quadrilateral of the contour is calculated as the dimensional reference frame; Based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template, a homography matrix for correcting perspective distortion is calculated. The homography matrix is ​​used to perform perspective transformation on the contour and dimensional reference frame to obtain a corrected contour and a standard dimensional frame. For each edge of the standard dimensional frame, edge point sets are sampled along the edge normal direction on the corrected contour. A random sampling consensus algorithm is used to perform straight line fitting on the edge point sets to obtain fitted edges, and the proportion of inlier points after fitting is calculated. When the proportion of inlier points on any side is lower than a first threshold, the sampling point density along the edge normal direction is increased, and straight line fitting is re-executed. The workpiece dimensions determined by all fitted edges are compared with the planned nominal dimensions to construct a processing deviation state matrix. Based on the processing deviation state matrix and processing accuracy compensation parameters of historical cycles, the transfer matrix representing the processing characteristics is updated online. The processing deviation state matrix of the current cycle is calculated with the updated transfer matrix to generate the processing accuracy compensation parameters for the current cycle, and then sent to the controller of the intelligent manufacturing unit.

2. The method according to claim 1, characterized in that, The process of acquiring an image of the workpiece to be inspected, obtaining an initial target bounding box through a target detection network, and segmenting the workpiece within the initial target bounding box to obtain a pixel mask and contour includes: inputting the acquired image of the workpiece to be inspected into a YOLOv5 target detection network, outputting a target bounding box with a confidence score greater than 0.9 as the initial target bounding box; inputting the image region within the initial target bounding box into a U-Net segmentation network, outputting a grayscale probability map of the workpiece; performing binarization processing on the grayscale probability map with a threshold of 128 to obtain a pixel mask, and using the Suzuki algorithm to extract the outermost contour of the pixel mask.

3. The method according to claim 1, characterized in that, The calculation of the homography matrix for correcting perspective distortion based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template includes: using the ORB operator to detect at least 50 actual feature points within the dimensional reference frame and performing Hamming distance matching with the workpiece feature point template; using the Lowe ratio test to eliminate mismatched point pairs and selecting high-quality matching point pairs with a matching distance ratio of less than 0.75; and using at least 4 high-quality matching point pairs to iteratively solve a 3×3 homography matrix using the RANSAC algorithm.

4. The method according to claim 1, characterized in that, The step of sampling edge points along the normal direction of the edge on the correction contour for each edge of the standard size frame to obtain an edge point set, and then using a random sampling consensus algorithm to perform line fitting on the edge point set to obtain a fitted edge, includes: for each edge of the standard size frame, sampling is performed at 1-pixel intervals along the normal direction of the edge within a strip region of ±5 pixels on both sides of the edge, and the intersection points with the correction contour are recorded to form an edge point set; the random sampling consensus algorithm is applied to the edge point set, and two points are randomly selected each time to form a candidate line, the distance from other points in the point set to the line is calculated, and points with a distance less than 0.5 pixels are recorded as inliers. After 100 iterations, the candidate line with the most inliers is selected as the fitted edge.

5. The method according to claim 1, characterized in that, When the proportion of interior points on any side is lower than the first threshold, the sampling point density in the edge normal direction is increased and the line fitting is re-executed. This includes: when the proportion of the calculated number of interior points to the total number of points in the edge point set is lower than the first threshold, the sampling interval in the edge normal direction is reduced from 1 pixel to 0.5 pixels, a new edge point set is obtained by resampling, and the line fitting is executed again. This process is repeated up to 3 times.

6. The method according to claim 1, characterized in that, The step of comparing the workpiece dimensions determined by all fitted edges with the planned nominal dimensions to construct a machining deviation state matrix includes: calculating the actual length, width, and four interior angles of the workpiece based on the four fitted edges; subtracting the actual length, width, and four interior angles from the nominal length, width, and nominal angles on the planning drawings to obtain the length deviation, width deviation, and four angle deviations; and using the six deviation values ​​as a 6×1 column vector to form the machining deviation state matrix for the current cycle. 。 7. The method according to claim 6, characterized in that, The online updating of the transfer matrix representing the characteristics of the machining process, based on the historical cycle-based machining deviation state matrix and machining accuracy compensation parameters, includes: employing a recursive least squares algorithm with a forgetting factor, based on the machining deviation state matrix of the current cycle... Machining accuracy compensation parameters from the previous cycle For the transfer matrix Perform online updates; set the forgetting factor to enhance the weight of new data during the update process.

8. The method according to claim 7, characterized in that, The step of performing calculations on the current cycle's processing deviation state matrix and the updated transfer matrix to generate the current cycle's processing accuracy compensation parameters, and sending these parameters to the controller of the intelligent manufacturing unit, includes: calculating the processing accuracy compensation parameters from the previous cycle according to the following formula. Calculate the machining accuracy compensation parameters for the current cycle. ： ;in, For the updated transfer matrix The pseudo-inverse matrix, This is the machining deviation state matrix for the current cycle; the machining accuracy compensation parameters It is a vector containing compensation values ​​for the X-axis, Y-axis, and rotary axes of the machine tool, which is sent to the PLC controller of the intelligent manufacturing unit via the Modbus TCP / IP protocol.

9. A closed-loop control system for visual quality inspection and processing accuracy of an intelligent manufacturing unit, characterized in that, It includes the following modules: a first calculation module, used to acquire an image of the workpiece to be inspected, obtain an initial target box through a target detection network, and segment the workpiece within the initial target box to obtain a pixel mask and contour; calculate the minimum area circumscribed quadrilateral of the contour as a size reference box; The second calculation module is used to calculate the homography matrix for correcting perspective distortion based on the matching relationship between the actual feature points detected within the dimensional reference frame and the preset workpiece feature point template; to perform perspective transformation on the contour and dimensional reference frame using the homography matrix to obtain the corrected contour and the standard dimensional frame; the construction module is used to sample edge point sets on the corrected contour along the edge normal direction for each edge of the standard dimensional frame, to perform straight line fitting on the edge point set using a random sampling consensus algorithm to obtain fitted edges, and to calculate the proportion of inlier points after fitting; when the proportion of inlier points on any side is lower than a first threshold, the sampling point density in the edge normal direction is increased and the straight line fitting is re-executed; the workpiece dimensions determined by all fitted edges are compared with the planned nominal dimensions to construct a processing deviation state matrix; the sending module is used to update the transfer matrix representing the characteristics of the processing process online based on the processing deviation state matrix and processing accuracy compensation parameters of the historical cycle; to perform calculations on the processing deviation state matrix of the current cycle and the updated transfer matrix to generate the processing accuracy compensation parameters of the current cycle, and to send them to the controller of the intelligent manufacturing unit.

10. The system according to claim 9, characterized in that, The process of acquiring an image of the workpiece to be inspected, obtaining an initial target bounding box through a target detection network, and segmenting the workpiece within the initial target bounding box to obtain a pixel mask and contour includes: inputting the acquired image of the workpiece to be inspected into a YOLOv5 target detection network, outputting a target bounding box with a confidence score greater than 0.9 as the initial target bounding box; inputting the image region within the initial target bounding box into a U-Net segmentation network, outputting a grayscale probability map of the workpiece; performing binarization processing on the grayscale probability map with a threshold of 128 to obtain a pixel mask, and using the Suzuki algorithm to extract the outermost contour of the pixel mask.

Images