Workpiece image motion distortion correction and accurate segmentation method based on directed potential energy field

Abstract

The invention discloses a workpiece image motion distortion correction and accurate segmentation method based on a directed potential energy field. The method comprises the following steps: 1) obtaining a target original image to be segmented; 2) analyzing the periodic change of the distorted image, and establishing a fluctuation model; 3) image adaptive preprocessing: removing an image background and reducing interference; 4) carrying out self-adaptive initialization on geometric contour vertexes based on directed potential energy field integrated clustering; according to the method, the real-time motion distortion correction, the self-adaptive correction and the dynamic segmentation of the image obtained by the linear array camera are realized, so that the problems of motion distortion, self-adaptive correction and dynamic cutting of the metal plate in image processing are solved.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and image processing, and in particular to a workpiece image motion distortion correction and precise segmentation method based on a directed potential energy field. Background Art

[0002] Motion distortion refers to the inaccurate and unrealistic visual representation in the image caused by the movement of the camera or the object during the shooting process, such as rotation, sliding, up and down shaking, etc. Specifically, motion distortion can be divided into the following types: (1) Geometric distortion: refers to the distortion and deformation of the shape of the object in the image due to the up and down movement or shaking of the object or the shooting camera during the shooting process. (2) Uneven stretching or compression: refers to the stretching or compression of the object in the image due to the uneven or fluctuating speed of the object or the shooting camera during the shooting process, making some parts appear to be out of proportion.

[0003] Since the line array camera scans line by line, if the object fluctuates vertically during the shooting process, the image parts captured by different scan lines will have vertical position differences, and the final combined image will have geometric distortion. If the object moves at an uneven speed, or the speed of some parts changes due to fluctuations, the distance between different scan lines will change, causing some parts of the image to be stretched or compressed, such as Figure 2 As shown in the figure. Traditional distortion correction methods are mainly used for images captured by area array cameras and are suitable for overall distortion correction, while line array images need to be processed line by line, which increases the complexity of correction. Since line array cameras are usually used in real-time applications, the correction algorithm must be completed in a very short time, and traditional distortion correction algorithms may not meet real-time requirements.

[0004] Image segmentation refers to the process of dividing an image into non-overlapping and meaningful sub-regions. Pixels in the same region have certain correlations, and pixels in different regions have certain differences. That is, the same label is assigned to pixels with the same properties in the image. General image segmentation includes the following steps: image preprocessing, feature extraction, segmentation processing, post-processing, verification and evaluation, output and application, etc.

[0005] Existing adaptive binarization methods may not be able to fully handle the diverse features in complex backgrounds. For example, when there are different textures, colors, and illumination changes in the background, the adaptive algorithm may misidentify part of the background as the foreground. This situation will lead to inaccurate artifact extraction and affect the effect of subsequent processing. Although some adaptive methods are able to handle local illumination changes, the performance of existing methods is still limited when faced with images with severe uneven exposure. Uneven exposure may also cause large changes in the grayscale distribution in the image, and traditional adaptive binarization methods may not be able to effectively handle these changes, thus affecting the binarization quality of the image. Efficient adaptive binarization algorithms usually need to process a large amount of image data and calculate complex parameters, which may result in long processing time, especially when processing high-resolution images. This computational complexity may limit the applicability of adaptive binarization methods in real-time applications.

[0006] In the field of industrial image processing, obtaining the vertices of workpiece geometry is an important step in image segmentation, especially in workpiece positioning and automated inspection. Accurately extracting geometric vertices can help identify the shape, size, and position of the workpiece, providing key information for subsequent processing, assembly, and quality control. However, the adaptive binarization method in the image preprocessing step can only simply obtain the contour of the workpiece. Especially when the workpiece is in an uncertain position on the production line, the contour obtained by preprocessing may have shape distortion and unclear edges, making it difficult to provide accurate vertex positions. Therefore, more complex algorithms and techniques are needed to further refine the acquisition of contour points. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the prior art and provide a method for motion distortion correction and precise segmentation of workpiece images based on a directed potential field, which realizes real-time motion distortion correction, adaptive correction and dynamic segmentation of images obtained by a linear array camera, so as to solve the problems of motion distortion, adaptive correction and dynamic cropping of metal plates in image processing.

[0008] The object of the present invention is achieved by: a method for correcting motion distortion and accurately segmenting a workpiece image based on a directed potential field, comprising the following steps:

[0009] Step 1) Obtaining the original image of the target to be segmented: Using a linear array industrial camera in conjunction with a heavy-duty conveyor, the image of the target object is captured to provide the original image to be processed for subsequent distortion correction and image segmentation;

[0010] Step 2) Analyze the periodic changes of the distorted image and establish a fluctuation model: by reading and converting the image into the frequency domain, analyzing the spectrum to detect the frequency of the distortion, and generating a fluctuation model based on the detected frequency and amplitude;

[0011] Step 3) Image adaptive preprocessing: The corrected image is preprocessed by an adaptive binarization method based on minimum fusion spatial correlation to remove image background and reduce interference;

[0012] Step 4) Adaptive initialization of geometric contour vertices based on directed potential field integrated clustering: obtaining the vertices of the workpiece geometry through edge detection and integrated clustering based on directed potential field;

[0013] Step 5) Clustering, finding vertices: Determine vertex coordinates by calculating the distance between each corner point and the geometric center in each class;

[0014] Step 6) Calculate the rotation angle, create a rotation matrix, and rotate and correct: select two adjacent vertices from the vertices to form a straight line, and use the inverse tangent function to calculate the straight line to generate a rotation matrix;

[0015] Step 7) Set boundary dynamic cropping: Determine the cropping area by recalculating the vertex coordinates of the rotated image and dynamically crop the image.

[0016] As a further limitation of the present invention, the step 2) specifically comprises:

[0017] 2-1) By reading the image data, converting it into a grayscale image and storing it, and then performing a two-dimensional discrete Fourier transform, a spectrum diagram containing the intensity and phase information of various frequency components in the image is obtained; its mathematical expression is as follows:

[0018]

[0019] Where f(x,y) is the pixel value of the input image, F(u,v) is the frequency domain representation of the output, and M and N are the width and height of the image; are sine and cosine waves represented by complex exponentials;

[0020] 2-2) Rearrange the frequency domain: Rearrange the frequency domain, move the low-frequency components to the center of the spectrum, and move the high-frequency components to the edge;

[0021] 2-3) Compress the spectrum amplitude through logarithmic transformation. Compress the range of the spectrum amplitude through logarithmic transformation, take the natural logarithm of each element of the amplitude spectrum, and adjust it to a suitable visualization scale;

[0022] 2-4) Detect the main frequency components and amplitudes of the image by analyzing the central area of ​​the image spectrum; analyze the main frequency components of the image and find the peak position; calculate the coordinates of the frequency peak in the entire spectrum to determine the main frequency components in the image;

[0023] 2-5) Calculate the frequency of the periodic structure in the image and set an amplitude value for subsequent image correction;

[0024] 2-6) Generate a wave model; simulate the periodic distortion in the image through a sine wave and generate a wave model based on a sine function to correct the image distortion; generate a set of horizontal offsets according to the height of the image for subsequent image correction; if the image height is rows, each row in the image is y i , the i-th element is x i , the adjusted amplitude is A, the frequency is f, and the fluctuation model is expressed as:

[0025] y i =i,i∈{0,1,2,…,rows-1}

[0026]

[0027] 2-7) Perform inverse transformation correction; calculate the horizontal offset through the wave model, and horizontally shift the pixels of each row to reverse the periodic distortion in the image to achieve image correction.

[0028] As a further limitation of the present invention, the step 3) specifically comprises:

[0029] 3-1) Weighted grayscale conversion: the M×N resolution, RGB three-channel image I clb The pixel value of the k channel with coordinates (i, j) is recorded as Then its weighted grayscale image I gray The gray value of the pixel with coordinates (i, j) is

[0030]

[0031] Among them, {α1, α2, α3} are weighting coefficients;

[0032] 3-2) The threshold is traversed by the adaptive binarization method that maximizes the continuity of foreground pixel coordinates. The binarization threshold is G k , get the binary image I bin Gray value of the pixel with coordinates (i, j) The process is:

[0033]

[0034] By the current threshold G k Obtained n k foreground pixels, calculate the geometric center c of the current foreground k :

[0035]

[0036] The envelope area of ​​all two-dimensional space points (i, j) is represented by ch(i, j), and the quotient of the average second-order central moment of the foreground pixels and ch(i, j) is used to define the fusion spatial correlation index fco of the current image foreground:

[0037]

[0038] According to the segmentation threshold, the average second-order central moment of the foreground pixels is calculated, the envelope gradient is calculated, and the minimum value of the fusion spatial correlation index fco is determined;

[0039] Traverse G k The parameter space of the fusion space correlation index fco can be minimized to obtain the appropriate binarization threshold G, that is:

[0040]

[0041] The obtained threshold value G is used to perform binarization, and a binarized image is obtained as the final result of step 3).

[0042] As a further limitation of the present invention, it is characterized in that the step 4) specifically comprises:

[0043] 4-1) Use the canny operator and grayscale image convolution to obtain the image edge point set I edge ; Extend the grayscale image using periodic mirroring;

[0044] 4-2) Adding an initial particle set: Add an initial particle set P with equal intervals and uniform distribution in the space where the image is located. e ; If the coordinates of the starting particle in the image coordinate system are and the particle interval is Δ, then the initial particle set P e It is expressed as:

[0045]

[0046] 4-3) Obtaining the internal particle set: The initial particle set P e The binary image I obtained by preprocessing in step 3-2) bin Perform convolution to obtain the binary image I bin The particle set P inside the envelope:

[0047]

[0048] 4-4) Use the internal particle set P to the edge point set I edge Construct a directed potential energy field; solve the resultant force magnitude, direction and potential energy value of all points in the edge point set;

[0049] 4-5) Based on the three indicators of the magnitude of the resultant force, the direction of the resultant force and the potential energy value of the resultant force, three base clusters are formed using the Kmeans method; the cluster number K is set to an integer K less than the square root of the number of edge points. ini ,Right now:

[0050]

[0051] 4-6) After completing the construction of the three base clusters, cluster integration is achieved through cluster merging and sample allocation; cluster merging is achieved by calculating the similarity between clusters, and any point in the edge point set is Class L i With class L j The similarity is measured by the coefficient J i,j Characterization:

[0052]

[0053] Merge the two classes with the highest similarity and make the number of clusters K in the i-th iteration i =K i-1 -1; if class L i With class L j When merging, any data point Appear and When , the voting method is used for sample allocation; data points New label L new satisfy:

[0054]

[0055] Recursively perform cluster merging and sample allocation until the similarity between any two classes is no higher than the threshold γ; at this point, the number of clusters is K end ; Select four classes that meet the conditions in turn Initialize the workpiece geometry contour vertex class:

[0056]

[0057] As a further limitation of the present invention, the step 4-4) of solving the resultant force magnitude, resultant force direction and resultant force potential energy value of all points in the edge point set specifically includes:

[0058] Any point in the particle set P (P x1 ,P y1 ) for the edge point set I edge Any point in Generate repulsive force Its size is:

[0059]

[0060] Among them, k0 is the potential energy coefficient, R0 is the limit range of repulsive force; when the particle (P x1 ,P y1 ) and dot When the second-order moment exceeds the limit range R0, there is no repulsive force between the two; the repulsive force is a vector, and its direction points from the point of occurrence to the point of action. The sum of the forces on a single point of action is the vector sum; any point in the edge point concentration The resultant force vector of all particles is

[0061]

[0062] The angle between the resultant force vector and the unit vector (1,0) in the positive direction of the x-axis of the image coordinate system is:

[0063]

[0064] By particle (P x1 ,P y1 ) in the potential field generated by the edge point set I edge Any point in The potential energy express:

[0065]

[0066] The potential field generated by all particles in the internal particle set P is at point The sum of the potential energy at is:

[0067]

[0068] As a further limitation of the present invention, the step 5) specifically comprises:

[0069] 5-1) Calculate the Euclidean distance from the initial cluster center to all corner points, and assign the corner points to the category to which the nearest cluster center belongs according to the calculated distance; if there are two corner points x = (x1, x2, ..., x n ) and y=(y1,y2,…,y n ), then their Euclidean distance d(x,y) is expressed as:

[0070]

[0071] 5-2) Detect all contours in the image: Binarize the image, identify and track edge points to determine the boundaries of objects in the image, extract external contours, ignore internal nested contours, and remove redundant points; if the binary image is denoted as I(x,y), where the pixel value I(x,y)∈{0,255}, the binary image is represented as:

[0072]

[0073] The pixel coordinates on the contour are marked as (x i ,y i ), then the kth contour C k It is expressed as:

[0074] C k ={(x1,y1),(x2,y2),…,(x n ,y n )}

[0075] Use linked list encoding to record the points on the contour (x i+1 ,y i+1 ), the displacement corresponding to the direction code is recorded as Δ(x,y), then:

[0076] (x i+1 ,y i+1 )=(x i ,y i )+Δ(x,y)

[0077] Then the set of all contours C is:

[0078] C={C1,C2,…,C k}

[0079] 5-3) Select the largest contour. The contour with the largest area among all contours is the contour of the metal plate. If (x i ,y i ) is the contour C i The jth point on the contour, m is the total number of points on the contour, then the contour C i The area A(C i ) is expressed as:

[0080]

[0081] 5-4) Calculate the geometric moment of the contour to determine the geometric center of the contour, that is, the centroid; if the zero-order moment is denoted by M 00 , M 10 and M 01 is the first-order moment, which is used to calculate the x and y coordinates of the center of mass. The calculation formula of the center of mass is as follows:

[0082] The x-coordinate of the center of mass

[0083] The y coordinate of the center of mass From each corner point list, select the corner point farthest from the geometric center, that is, the vertex; in the corner point set P i In the example, the corner point is denoted by (x ij ,y ij ), then the vertex (x i,max ,y i,max ):

[0084]

[0085] Arrange them in the order of upper left, upper right, lower left, and lower right to obtain the four vertices of the metal plate.

[0086] As a further limitation of the present invention, the step 6) specifically comprises:

[0087] 6-1) Calculate the rotation angle. If there are two adjacent corner points p1 = (x1, y1) and p2 = (x2, y2), the rotation angle θ is expressed as:

[0088] θ=arctan(y2-y1,x2-x1)

[0089] Then convert the radian value to an angle value:

[0090]

[0091] 6-2) Create a rotation matrix. If a point (x, y) is rotated around a center point (cx, cy) by an angle θ to obtain a new coordinate (x', y'), then the translation tx and ty are expressed as:

[0092] tx=(1-cosθ)·cx-sinθ·cy

[0093] ty=sinθ·cx+(1-cosθ)·cy

[0094] The final rotation matrix M is expressed as:

[0095]

[0096] 6-3) Rotation normalization: apply a rotation matrix to the original image around the center point of the image to rotate it to a specific angle, rotate the image to the desired position while maintaining the original size of the image; perform a rotation transformation on the original image according to the rotation matrix to generate a new rotated image, that is, use the affine transformation function to move each pixel in the source image to a new position in the target image by applying the affine transformation matrix.

[0097] As a further limitation of the present invention, the step 7) specifically comprises:

[0098] 7-1) Set the border value, which is the minimum distance between the edge of the cropped area and the vertex of the rotated image. Set the border value to ensure that the cropped image can contain the target area;

[0099] border=α·min(w target ,h target )

[0100] Among them, w target is the width of the metal plate in the image, h target is the height of the metal plate in the image, in pixels, and the proportionality coefficient α is a small quantity satisfying 0<α<1;

[0101] 7-2) Determine the cropping area, recalculate the four vertices of the metal plate after the image is rotated and normalized, calculate the minimum and maximum values ​​of the x-coordinates and y-coordinates of these vertices respectively, and set the image width as width, height as height, and boundary value as border. The boundary coordinates of the cropping area are expressed as:

[0102] min x =max(0,min(x i )+border)

[0103] max x =min(width,max(x i )-border)

[0104] min y =max(0,min(y i )+border)

[0105] max y =min(height,max(y i )-border)

[0106] 7-3) To crop the image, use array slicing to extract a sub-rectangular area from the two-dimensional array according to the specified vertex coordinates, expressed as:

[0107] cropped_image=rotated_image[min y :max y ,min x :max x ].

[0108] The present invention adopts the above technical solution, and compared with the prior art, the beneficial effects are as follows: the corrected image is preprocessed by an adaptive binarization method based on minimum fusion spatial correlation to remove the image background and reduce interference, and the rectangular workpiece image can be segmented more accurately and effectively in the case of reflective material, uneven lighting, and complex background, thereby realizing adaptive preprocessing that initially retains geometric contour features. A directed potential energy field is constructed by adding a uniformly distributed particle set to the preprocessed image, and the repulsive force size, direction, and potential energy value of the edge points are integrated and clustered, and the initial point set containing the vertices of the outer contour of the image is adaptively obtained, thereby overcoming the influence of the uncertain posture of the workpiece on the production line on the image segmentation. The geometric center and vertex position of the metal plate are accurately calculated, and the skewed metal plate image is normalized by the rotation matrix. After the correction, vertex detection and dynamic cropping are performed to achieve accurate correction and effective cropping of the metal plate image, which is particularly suitable for rectangular metal plates with large rotation angles, non-centered and narrow and long; the image is read and converted to the frequency domain, the spectrum diagram is analyzed to detect the frequency of distortion, a wave model is generated according to the detected frequency and amplitude, the image is inversely transformed to correct the distortion, and the corrected image is displayed and saved; the image segmentation method of the present invention can significantly improve the accuracy and flexibility of metal plate image processing, adapt to a wider range of rotation angles and metal plates of various forms, and provide an efficient and reliable solution for online automatic detection and quality control of metal plates. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 Schematic diagram of the application environment of the image segmentation method of the present invention.

[0110] Figure 2 Schematic diagram of traditional image motion distortion.

[0111] Figure 3 It is a schematic diagram of the process of the present invention.

[0112] Figure 4 This is a comparison chart of the image adaptive preprocessing effects of the present invention.

[0113] Figure 5 This is an effect diagram of the adaptive initialization of geometric contour vertices based on directed potential field integrated clustering of the present invention.

[0114] Figure 6 This is an example effect diagram of the image rotation, correction and cropping method of the present invention. DETAILED DESCRIPTION

[0115] The image segmentation method provided in this example can be applied to Figure 1 In the application environment shown.

[0116] Among them, industrial cameras refer to linear array cameras with relative motion errors. Linear array cameras scan line by line and obtain complete images by constantly moving objects or the camera itself. Since it is line-by-line scanning, if the object fluctuates up and down during the scanning process (such as metal plates during transportation), the image parts captured by different scanning lines will have different vertical positions, and the images captured in each line will be slightly different; if the speed of the object is uneven during transportation, or the movement speed of some parts changes due to fluctuations, the distance between the scanning lines will change, which will cause motion distortion. This distortion appears as wavy distortion or periodic deformation in the image.

[0117] The conveying device here refers to the device that can transport large-area heavy objects in the industrial field, including belt conveyors, roller conveyors, chain conveyors, chain plate conveyors, etc., which can transport objects from one location to another, satisfying the material transportation of the production line while also providing convenience for the line array camera to obtain images. The up and down jitter caused by the conveying device due to its own accuracy will also cause the object to jitter up and down. The motion distortion caused by this jitter is similar to the motion distortion caused by the up and down fluctuation of the object during the transportation process, which will not be repeated here.

[0118] The terminal can be an industrial computer, or an ordinary desktop, laptop, tablet computer, etc., and it only needs to provide the code running environment and software running conditions.

[0119] like Figure 3 The workpiece image motion distortion correction and precise segmentation method based on directed potential energy field shown in the figure comprises the following steps:

[0120] Step 1) Obtaining the original image of the target to be segmented: For the image acquisition and detection problem of large-size three-dimensional profiles, a linear array industrial camera is used in conjunction with a heavy-duty conveyor to capture the image of the target object, providing the original image to be processed for subsequent distortion correction and image segmentation;

[0121] Step 2) Analyze the periodic changes of the distorted image and establish a fluctuation model: by reading and converting the image into the frequency domain, analyzing the spectrum to detect the frequency of the distortion, and generating a fluctuation model based on the detected frequency and amplitude;

[0122] 2-1) By reading the image data, converting it into a grayscale image and storing it, and then performing a two-dimensional discrete Fourier transform, a spectrum diagram containing the intensity and phase information of various frequency components in the image is obtained; its mathematical expression is as follows:

[0123]

[0124] Where f(x,y) is the pixel value of the input image; F(u,v) is the frequency domain representation of the output; M and N are the width and height of the image; are sine and cosine waves represented by complex exponentials.

[0125] 2-2) Rearrange the frequency domain. In the obtained frequency domain, low-frequency components and high-frequency components are usually distributed in different positions of the matrix. The default arrangement is that the low-frequency components are located at the four corners of the matrix and the high-frequency components are located in the central area of ​​the matrix. However, this default arrangement is not conducive to spectrum observation and analysis. Therefore, the frequency domain is rearranged to move the low-frequency components to the center of the spectrum and the high-frequency components to the edge.

[0126] 2-3) Compress the spectrum amplitude by logarithmic transformation. If the frequency of the spectrum graph is recorded as a complex number z=a+bi, its amplitude is The logarithmic transformation can compress the range of the spectrum amplitude to a smaller range. The natural logarithm log(x) is taken for each element of the amplitude spectrum and adjusted to a more appropriate visualization scale to enhance the visualization effect and make the details more obvious.

[0127] 2-4) Detect the main frequency components and amplitudes of the image by analyzing the central area of ​​the image spectrum; analyze the main frequency components of the image, find the peak position, and calculate its coordinates in the entire spectrum to determine the main frequency components in the image.

[0128] 2-5) Calculate the frequency of the periodic structure in the image and set an amplitude value for subsequent image correction.

[0129] 2-6) Generate a wave model, simulate the periodic distortion in the image through a sine wave and generate a wave model based on a sine function to correct the image distortion; generate a set of horizontal offsets according to the height of the image for subsequent image correction; if the image height is rows, each row in the image is y i , the i-th element is x i , the adjusted amplitude is A, the frequency is f, and the fluctuation model can be expressed as:

[0130] y i =i,i∈{0,1,2,…,rows-1}

[0131]

[0132] 2-7) Perform inverse transformation correction, calculate the horizontal offset through the wave model, and horizontally shift the pixels of each row to reverse the periodic distortion in the image to achieve image correction. If the pixel value of the corrected image in the i-th row and j-th column is recorded as corrected_img(i,j), the pixel value of the original image in the i-th row and j-th column is recorded as img(i,j), and the displacement of the i-th row is recorded as x i , the image width is C, then

[0133] corrected_img(i,j)=img(i,(j+x i )modC).

[0134] Step 3) Image adaptive preprocessing: To address the problems of cluttered background and uneven exposure of workpiece images that affect the accuracy of workpiece extraction, the corrected image is preprocessed by an adaptive binarization method based on minimum fusion spatial correlation to remove the image background and reduce interference;

[0135] 3-1) Weighted grayscale conversion: the M×N resolution, RGB three-channel image I clb The pixel value of the k channel with coordinates (i, j) is recorded as Then its weighted grayscale image I gray The gray value of the pixel with coordinates (i, j) is

[0136]

[0137] Wherein {α1, α2, α3} are weighted coefficients. In this embodiment, α1=0.299, α2=0.578, α3=0.114 are selected;

[0138] 3-2) The imaged workpiece is a single, continuous, and complete object. The pixels of the workpiece image have position correlation. Based on this characteristic, an adaptive binarization method that maximizes the continuity of foreground pixel coordinates is proposed. If the binarization threshold is G k , get the binary image I bin The gray value of the pixel with coordinates (i, j) The process is:

[0139]

[0140] By the current threshold G k Obtained n k foreground pixels, calculate the geometric center c of the current foreground k :

[0141]

[0142] The envelope area of ​​all two-dimensional space points (i, j) is represented by ch(i, j), and the quotient of the average second-order central moment of the foreground pixels and ch(i, j) is used to define the fusion spatial correlation index fco of the current image foreground:

[0143]

[0144] When the shooting background is messy, if the segmentation threshold is small, more background points are misjudged as foreground points, the segmentation result contains a large number of scattered background points, and the average second-order central moment of the foreground pixels remains basically unchanged. At this time, the envelope area is large, so the fco value remains at a small level. If the segmentation threshold is moderate, the foreground points will gradually be misjudged as the background, and the average second-order central moment of the foreground pixels will remain basically unchanged. As the threshold increases, the edge pixels away from the illumination center gradually decrease, the envelope area gradually decreases, and the fco value increases. If the segmentation threshold is large, a large number of foreground points are misjudged as the background, and the remaining foreground points are relatively concentrated. The average second-order central moment decreases rapidly, while the envelope area remains basically unchanged, so the fco value of the fusion spatial correlation indicator decreases.

[0145] Traverse G k The parameter space and the minimum fco value can obtain the appropriate binarization threshold G, that is:

[0146]

[0147] The obtained threshold value G is used to perform binarization, and a binarized image is obtained as the final result of this step.

[0148] In this embodiment, the grayscale image of the workpiece, the fco variation diagram with the threshold, the traditional OTSU method, the adaptive binarization method, and the comparison diagram of the method proposed in the present invention are shown in FIG. Figure 4 Obviously, the proposed method can segment the rectangular workpiece image more accurately and effectively in the case of reflective material, uneven lighting and complex background, and realize adaptive preprocessing that initially retains the geometric contour features.

[0149] Step 4) Adaptive initialization of geometric contour vertices based on directed potential field integrated clustering: The vertices of the workpiece geometry are obtained through edge detection and integrated clustering based on directed potential field; obtaining accurate vertices is an important step in image segmentation, but step 3) adaptive preprocessing can only simply obtain inaccurate image contours, so further processing of the image is still required to obtain the precise position of the vertices. Step 4) Construct a directed potential field by adding a uniformly distributed particle set to the preprocessed image, integrate and cluster the repulsive force size, direction, and potential energy value of the edge points, and adaptively obtain the initial point set containing the image outer contour vertices to overcome the influence of the uncertain posture of the workpiece on the production line on image segmentation.

[0150] 4-1) Use the canny operator and grayscale image convolution to obtain the image edge point set I edge ; To avoid edge effects caused by convolution operations in the image boundary area, the grayscale image is extended by using a periodic mirror. Since the canny edge detection technology is mature and stable, and is not the main content of the present invention, the present invention will not elaborate on it in detail.

[0151] 4-2) Add an initial particle set, add an initial particle set P with equal intervals and uniform distribution in the space where the image is located e ; If the coordinates of the starting particle in the image coordinate system are and the particle interval is Δ, then the initial particle set P e It can be expressed as:

[0152]

[0153] 4-3) Obtain the internal particle set and convert the initial particle set P e The binary image I obtained by preprocessing in step (3) bin Perform convolution to obtain a binary image I bin The particle set P inside the envelope:

[0154]

[0155] 4-4) Use the internal particle set P to the edge point set I edge Construct a directed potential energy field; solve the magnitude, direction and potential energy value of the resultant force of all points in the edge point set;

[0156] Any point (P x1 ,P y1 ) for the edge point set I edge Any point in Generate repulsive force Its size is:

[0157]

[0158] In the above formula, k0 is the potential energy coefficient, and R0 is the limit range of repulsive force. Obviously, when the particle (P x1 ,P y1 ) and dot When the second-order moment of exceeds the limit range R0, there is no repulsive force between the two. The repulsive force is a vector, and its direction is from the point of occurrence to the point of action. The sum of the forces on a single point of action is the vector sum; any point in the edge point concentration The resultant force vector of all particles is

[0159]

[0160] The angle between the resultant force vector and the unit vector (1,0) in the positive direction of the x-axis of the image coordinate system is:

[0161]

[0162] By particle (P x1 ,P y1 ) in the potential field generated by the edge point set I edgeAny point in The potential energy express:

[0163]

[0164] Potential energy is a scalar, so the sum of potential energy values ​​is an algebraic sum; the potential field generated by all particles in the internal particle set P is at point The sum of the potential energy at is:

[0165]

[0166] 4-5) The Kmeans method is used to form three base clusters for these three indicators respectively; since the Kmeans clustering technology is already a mature algorithm, it will not be described in detail in this description.

[0167] Set the cluster number K to an integer K less than the square root of the number of edge points. ini ,Right now:

[0168]

[0169] 4-6) After completing the construction of the three base clusters, cluster integration is achieved through cluster merging and sample allocation. Cluster merging is achieved by calculating the similarity between clusters. Class L i With class L j The similarity is measured by the coefficient J i,j Characterization:

[0170]

[0171] Merge the two classes with the highest similarity and make the number of clusters K in the i-th iteration i =K i-1 -1. If class L i With class L j When merging, any data point Appear and When , the voting method is used for sample allocation; data points New label L new satisfy:

[0172]

[0173] Recursively perform cluster merging and sample allocation until the similarity between any two classes is no higher than the threshold γ; at this point, the number of clusters is K end In this embodiment, the threshold γ is set to 0.9; four categories that meet the following conditions are selected in turn: Initialize the workpiece geometry contour vertex class:

[0174]

[0175] In this embodiment, the grayscale image of the workpiece, the canny edge point set, the initial particle set, the internal particle set, the edge point set repulsion magnitude, the repulsion angle and potential energy cluster diagram, and the integrated cluster diagram are shown in Figure 5. Obviously, the three evaluation indices designed in the present invention can effectively distinguish the internal pattern edge, the straight edge and the geometric contour vertex. Specifically, the class where the geometric contour vertex belongs has the largest average resultant force and the smallest average potential energy, and the resultant force direction is used to distinguish different contour vertices. In real image processing, only these three indicators are used for discrimination, and their accuracy is greatly affected by image noise. Therefore, the present invention uses an integrated clustering method to further improve the accuracy, stability and robustness of geometric contour vertex type discrimination.

[0176] Step 5) Clustering, finding vertices: Determine vertex coordinates by calculating the distance between each corner point and the geometric center in each class;

[0177] 5-1) Calculate the Euclidean distance from the initial cluster center to all corner points, and assign the corner points to the category to which the nearest cluster center belongs according to the calculated distance; if there are two corner points x = (x1, x2, ..., x n ) and y=(y1,y2,…,y n ), then their Euclidean distance d(x,y) can be expressed as:

[0178]

[0179] The above corner points must include the four vertices of the metal plate. In order to find the vertices in the corner point list, it is necessary to obtain the geometric center of the metal plate.

[0180] 5-2) Detect all contours in the image, binarize the image, identify and track edge points to determine the boundaries of objects in the image, extract external contours, ignore internal nested contours, and remove redundant points; if the binary image is recorded as I(x,y), where the pixel value I(x,y)∈{0,255}, the binary image can be expressed as:

[0181]

[0182] The pixel coordinates on the contour are marked as (x i ,y i ), then the kth contour C k It is expressed as:

[0183] C k ={(x1,y1),(x2,y2),…,(x n ,y n )}

[0184] Use linked list encoding to record the points on the contour (x i+1 ,y i+1 ), the displacement corresponding to the direction code is recorded as Δ(x,y), then:

[0185] (x i+1 ,y i+1 )=(x i ,y i )+Δ(x,y)

[0186] Then the set of all contours C is:

[0187] C={C1,C2,…,C k}

[0188] 5-3) Select the largest contour. The contour with the largest area among all contours is the contour of the metal plate. If (x i ,y i ) is the contour C i The jth point on the contour, m is the total number of points on the contour, then the contour C i The area A(C i ) can be expressed as:

[0189]

[0190] 5-4) Calculate the geometric moment of the contour to determine the geometric center (center of mass) of the contour. If the zero-order moment is denoted by M 00 , M 10 and M 01 is the first-order moment, which is used to calculate the x and y coordinates of the center of mass. The calculation formula of the center of mass is as follows:

[0191] The x-coordinate of the center of mass

[0192] The y coordinate of the center of mass From each corner point list, select the corner point farthest from the geometric center, that is, the vertex; in the corner point set P i In the example, the corner point is denoted by (x ij ,y ij ), then the vertex (x i,max ,y i,max ):

[0193]

[0194] Arrange them in the order of upper left, upper right, lower left, and lower right, thus obtaining the four vertices of the metal plate.

[0195] Step 6) Calculate the rotation angle, create a rotation matrix, and rotate and correct: select two adjacent vertices from the vertices to form a straight line, and use the inverse tangent function to calculate the straight line to generate a rotation matrix;

[0196] 6-1) Calculate the rotation angle. If there are two adjacent corner points p1 = (x1, y1) and p2 = (x2, y2), the rotation angle θ can be expressed as:

[0197] θ=arctan(y2-y1,x2-x1)

[0198] Then convert the radian value to an angle value:

[0199]

[0200] 6-2) Create a rotation matrix. If a point (x, y) is rotated around a center point (cx, cy) by an angle θ to obtain a new coordinate (x', y'), the translation amounts tx and ty can be expressed as:

[0201] tx=(1-cosθ)·cx-sinθ·cy

[0202] ty=sinθ·cx+(1-cosθ)·cy

[0203] The final rotation matrix M can be expressed as:

[0204]

[0205] 6-3) Rotation normalization: Apply a rotation matrix to the original image around the center point of the image to rotate it by a specific angle, rotate the image to the desired position, and keep the original size of the image; perform a rotation transformation on the original image according to the rotation matrix to generate a new rotated image, that is, use the affine transformation function to move each pixel in the source image to a new position in the target image by applying the affine transformation matrix; specifically, for each pixel (x, y) in the source image, the transformation matrix will calculate its new position (x', y') in the target image:

[0206]

[0207] Among them, a 11 , a 12 , a 21 , a 22 is the element in the rotation matrix, b1 and b2 are the translation amounts.

[0208] Step 7) Setting boundary dynamic cropping: determining the cropping area by recalculating the vertex coordinates of the rotated image, and dynamically cropping the image;

[0209] 7-1) Set the border value, which is the minimum distance between the edge of the cropped area and the vertex of the rotated image. Set the border value to ensure that the cropped image can contain the target area while avoiding excessive background interference;

[0210] border=α·min(w target ,h target )

[0211] Among them, w target is the width of the metal plate in the image, h target is the height of the metal plate in the image, in pixels. The proportionality coefficient α is a small value satisfying 0<α<1. In this embodiment, α=0.01.

[0212] 7-2) Determine the cropping area, recalculate the four vertices of the metal plate after the image is rotated and normalized, calculate the minimum and maximum values ​​of the x-coordinates and y-coordinates of these vertices respectively, and set the image width as width, height as height and border value as border. The border coordinates of the cropping area are expressed as:

[0213] min x =max(0,min(x i )+border)

[0214] max x =min(width,max(x i )-border)

[0215] min y =max(0,min(y i )+border)

[0216] max y =min(height,max(y i )-border)

[0217] 7-3) To crop an image, use array slicing to extract a sub-rectangular area from a two-dimensional array according to the specified vertex coordinates, which can be expressed as:

[0218] cropped_image=rotated_image[min y :max y ,min x :max x ];

[0219] In this embodiment, the rotation correction and cropping diagram of the workpiece are as follows: Figure 6 As shown; obviously, the rotation correction method designed in the present invention can efficiently and accurately correct the target object in the image and separate it from the background, can cope with displacement and rotation in any direction and at any angle, and has high stability and strong robustness.

[0220] The present invention provides a workpiece image motion distortion correction and precise segmentation method based on a directed potential field. The corrected image is preprocessed by an adaptive binarization method based on minimum fusion spatial correlation to remove the image background and reduce interference. In the case of reflective material, uneven lighting, and complex background, the rectangular workpiece image can be segmented more accurately and effectively, and the adaptive preprocessing of preliminarily retaining geometric contour features is realized. A directed potential field is constructed by adding a uniformly distributed particle set to the preprocessed image, and the repulsive force size, direction, and potential energy value of the edge point are integrated and clustered, and the initial point set containing the image outer contour vertices is adaptively obtained to overcome the influence of the uncertain posture of the workpiece on the production line on the image segmentation. The geometric center and vertex position of the metal plate are accurately calculated, and the skewed metal plate image is normalized by a rotation matrix. After normalization, vertex detection and dynamic cropping are performed to achieve accurate normalization and effective cropping of the metal plate image, which is particularly suitable for rectangular metal plates with large rotation angles, non-centering, and narrow and long.

[0221] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A method for workpiece image motion distortion correction and accurate segmentation based on directed potential field, characterized in that: The following steps are involved: Step 1) Obtaining the original image of the target to be segmented: Using a linear array industrial camera in conjunction with a heavy-duty conveyor, the image of the target object is captured to provide the original image to be processed for subsequent distortion correction and image segmentation; Step 2) Analyze the periodic changes of the distorted image and establish a fluctuation model: by reading and converting the image into the frequency domain, analyzing the spectrum to detect the frequency of the distortion, and generating a fluctuation model based on the detected frequency and amplitude; Step 3) Image adaptive preprocessing: The corrected image is preprocessed by an adaptive binarization method based on minimum fusion spatial correlation to remove image background and reduce interference; Step 4) Adaptive initialization of geometric contour vertices based on directed potential field integrated clustering: obtaining the vertices of the workpiece geometry through edge detection and integrated clustering based on directed potential field; Step 5) Clustering, finding vertices: Determine vertex coordinates by calculating the distance between each corner point and the geometric center in each class; Step 6) Calculate the rotation angle, create a rotation matrix, and rotate and correct: select two adjacent vertices from the vertices to form a straight line, and use the inverse tangent function to calculate the straight line to generate a rotation matrix; Step 7) Set boundary dynamic cropping: Determine the cropping area by recalculating the vertex coordinates of the rotated image and dynamically crop the image.

2. The method for correcting motion distortion and accurately segmenting workpiece images based on directed potential energy field according to claim 1, characterized in that: The step 2) specifically includes: 2-1) By reading the image data, converting it into a grayscale image and storing it, and then performing a two-dimensional discrete Fourier transform, a spectrum diagram containing the intensity and phase information of various frequency components in the image is obtained; its mathematical expression is as follows: Where f(x,y) is the pixel value of the input image, F(u,v) is the frequency domain representation of the output, and M and N are the width and height of the image; are sine and cosine waves represented by complex exponentials; 2-2) Rearrange the frequency domain: Rearrange the frequency domain, move the low-frequency components to the center of the spectrum, and move the high-frequency components to the edge; 2-3) Compress the spectrum amplitude through logarithmic transformation. Compress the range of the spectrum amplitude through logarithmic transformation, take the natural logarithm of each element of the amplitude spectrum, and adjust it to a suitable visualization scale; 2-4) Detect the main frequency components and amplitudes of the image by analyzing the central area of ​​the image spectrum; analyze the main frequency components of the image and find the peak position; calculate the coordinates of the frequency peak in the entire spectrum to determine the main frequency components in the image; 2-5) Calculate the frequency of the periodic structure in the image and set an amplitude value for subsequent image correction; 2-6) Generate a wave model; simulate the periodic distortion in the image through a sine wave and generate a wave model based on a sine function to correct the image distortion; generate a set of horizontal offsets according to the height of the image for subsequent image correction; if the image height is rows, each row in the image is y i , the i-th element is x i , the adjusted amplitude is A, the frequency is f, and the fluctuation model is expressed as: y i =i,i∈{0,1,2,…,rows-1} 2-7) Perform inverse transformation correction; calculate the horizontal offset through the wave model, and horizontally shift the pixels of each row to reverse the periodic distortion in the image to achieve image correction.

3. The workpiece image motion distortion correction and precise segmentation method based on directed potential field according to claim 1 is characterized in that: The step 3) specifically includes: 3-1) Weighted grayscale conversion: the M×N resolution, RGB three-channel image I clb The pixel value of the k channel with coordinates (i, j) is recorded as Then its weighted grayscale image I gray The gray value of the pixel with coordinates (i, j) is Among them, {α1, α2, α3} are weighting coefficients; 3-2) The threshold is traversed by the adaptive binarization method that maximizes the continuity of foreground pixel coordinates. The binarization threshold is G k , get the binary image I bin Gray value of the pixel with coordinates (i, j) The process is: By the current threshold G k Obtained n k foreground pixels, calculate the geometric center c of the current foreground k : The envelope area of ​​all two-dimensional space points (i, j) is represented by ch(i, j), and the quotient of the average second-order central moment of the foreground pixels and ch(i, j) is used to define the fusion spatial correlation index fco of the current image foreground: According to the segmentation threshold, the average second-order central moment of the foreground pixels is calculated, the envelope gradient is calculated, and the minimum value of the fusion spatial correlation index fco is determined; Traverse G k The parameter space of the fusion space correlation index fco can be minimized to obtain the appropriate binarization threshold G, that is: The obtained threshold value G is used to perform binarization, and a binarized image is obtained as the final result of step 3).

4. The method for correcting motion distortion and accurately segmenting workpiece images based on directed potential energy field according to claim 1, characterized in that: The step 4) specifically includes: 4-1) Use the canny operator and grayscale image convolution to obtain the image edge point set I edge ; Extend the grayscale image using periodic mirroring; 4-2) Adding an initial particle set: Add an initial particle set P with equal intervals and uniform distribution in the space where the image is located. e ; If the coordinates of the starting particle in the image coordinate system are and the particle interval is Δ, then the initial particle set P e It is expressed as: 4-3) Obtaining the internal particle set: The initial particle set P e The binary image I obtained by preprocessing in step 3-2) bin Perform convolution to obtain the binary image I bin The particle set P inside the envelope: 4-4) Use the internal particle set P to the edge point set I edge Construct a directed potential energy field; solve the resultant force magnitude, direction and potential energy value of all points in the edge point set; 4-5) Based on the three indicators of the magnitude of the resultant force, the direction of the resultant force and the potential energy value of the resultant force, three base clusters are formed using the Kmeans method; the cluster number K is set to an integer K less than the square root of the number of edge points. ini ,Right now: 4-6) After completing the construction of the three base clusters, cluster integration is achieved through cluster merging and sample allocation; cluster merging is achieved by calculating the similarity between clusters, and any point in the edge point set is Class L i With class L j The similarity is measured by the coefficient J i,j Characterization: Merge the two classes with the highest similarity and make the number of clusters K in the i-th iteration i =K i-1 -1; if class L i With class L j When merging, any data point Appear and When , the voting method is used for sample allocation; data points New label L new satisfy: Recursively perform cluster merging and sample allocation until the similarity between any two classes is no higher than the threshold γ; at this point, the number of clusters is K end , select four classes that meet the conditions in turn Initialize the workpiece geometry contour vertex class:

5. The method for workpiece image motion distortion correction and precise segmentation based on directed potential field according to claim 4 is characterized in that: Step 4-4) of solving the resultant force magnitude, resultant force direction and resultant force potential energy value of all points in the edge point set specifically includes: Any point in the particle set P (P x1 ,P y1 ) for the edge point set I edge Any point in Generate repulsive force Its size is: Among them, k0 is the potential energy coefficient, R0 is the limit range of repulsion; when the particle (P x1 ,P y1 ) and dot When the second-order moment exceeds the limit range R0, there is no repulsive force between the two; the repulsive force is a vector, and its direction points from the point of occurrence to the point of action. The sum of the forces on a single point of action is the vector sum; any point in the edge point concentration The resultant force vector of all particles is The angle between the resultant force vector and the unit vector (1,0) in the positive direction of the x-axis of the image coordinate system is: By particle (P x1 ,P y1 ) in the potential field generated by the edge point set I edge Any point in The potential energy express: The potential field generated by all particles in the internal particle set P is at point The sum of the potential energy at is:

6. The method for workpiece image motion distortion correction and precise segmentation based on directed potential field according to claim 1, characterized in that: The step 5) specifically includes: 5-1) Calculate the Euclidean distance from the initial cluster center to all corner points, and assign the corner points to the category to which the nearest cluster center belongs according to the calculated distance; if there are two corner points x = (x1, x2, ..., x n ) and y=(y1,y2,…,y n ), then their Euclidean distance d(x,y) is expressed as: 5-2) Detect all contours in the image: Binarize the image, identify and track edge points to determine the boundaries of objects in the image, extract external contours, ignore internal nested contours, and remove redundant points; if the binary image is denoted as I(x,y), where the pixel value I(x,y)∈{0,255}, the binary image is represented as: The pixel coordinates on the contour are marked as (x i ,y i ), then the kth contour C k It is expressed as: C k ={(x1,y1),(x2,y2),…,(x n ,y n )} Use linked list encoding to record the points on the contour (x i+1 ,y i+1 ), the displacement corresponding to the direction code is recorded as Δ(x,y), then: (x i+1 ,and i+1 )=(x i ,and i )+Δ(x,y) Then the set of all contours C is: C={C1,C2,…,C k }; 5-3) Select the largest contour. The contour with the largest area among all contours is the contour of the metal plate. If (x i ,y i ) is the contour C i The jth point on the contour, m is the total number of points on the contour, then the contour C i The area A(C i ) is expressed as: 5-4) Calculate the geometric moment of the contour to determine the geometric center of the contour, that is, the centroid; if the zero-order moment is denoted by M 00 , M 10 and M 01 is the first-order moment, which is used to calculate the x and y coordinates of the center of mass. The calculation formula of the center of mass is as follows: The x-coordinate of the center of mass The y coordinate of the center of mass From each corner point list, select the corner point farthest from the geometric center, that is, the vertex; in the corner point set P i In the example, the corner point is denoted by (x ij ,y ij ), then the vertex (x i,max ,y i,max ): Arrange them in the order of upper left, upper right, lower left, and lower right to obtain the four vertices of the metal plate.

7. The workpiece image motion distortion correction and precise segmentation method based on directed potential field according to claim 1 is characterized in that: The step 6) specifically includes: 6-1) Calculate the rotation angle. If there are two adjacent corner points p1 = (x1, y1) and p2 = (x2, y2), the rotation angle θ is expressed as: θ=arctan(y2-y1,x2-x1) Then convert the radian value to an angle value: 6-2) Create a rotation matrix. If a point (x, y) is rotated around a center point (cx, cy) by an angle θ to obtain a new coordinate (x', y'), then the translation tx and ty are expressed as: tx=(1-cosθ)cx-sinθ·cy ty=sinθcx+(1-cosθ)·cy The final rotation matrix M is expressed as: 6-3) Rotation normalization: apply a rotation matrix to the original image around the center point of the image to rotate it to a specific angle, rotate the image to the desired position while maintaining the original size of the image; perform a rotation transformation on the original image according to the rotation matrix to generate a new rotated image, that is, use the affine transformation function to move each pixel in the source image to a new position in the target image by applying the affine transformation matrix.

8. The workpiece image motion distortion correction and precise segmentation method based on directed potential field according to claim 1 is characterized in that: The step 7) specifically includes: 7-1) Set the border value, which is the minimum distance between the edge of the cropped area and the vertex of the rotated image. Set the border value to ensure that the cropped image can contain the target area; border=α·min(w target ,h target ) Among them, w target is the width of the metal plate in the image, h target is the height of the metal plate in the image, in pixels, and the proportionality coefficient α is a small quantity satisfying 0<α<1; 7-2) Determine the cropping area, recalculate the four vertices of the metal plate after the image is rotated and normalized, calculate the minimum and maximum values ​​of the x-coordinates and y-coordinates of these vertices respectively, and set the image width as width, height as height, and boundary value as border. The boundary coordinates of the cropping area are expressed as: my x =max(0,min(x i )+border) max x =min(width,max(x i )-border) my y =max(0,min(y i )+border) max y =min(height,max(y i )-border) 7-3) To crop the image, use array slicing to extract a sub-rectangular area from the two-dimensional array according to the specified vertex coordinates, expressed as: cropped_image=rotated_image[min y :max y ,min x :max x ]。

Images