Workpiece image motion distortion correction and accurate segmentation method based on directed potential field
By employing the directional potential field method, combined with frequency domain analysis and rotation matrix correction, the problems of motion distortion and adaptive segmentation in linear scan cameras were solved, enabling real-time distortion correction and accurate segmentation of workpiece images, thus improving the accuracy and efficiency of workpiece inspection.
Patent Information
- Application Number
- CN202411723607.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-11-28
AI Technical Summary
Existing technologies struggle to achieve real-time motion distortion correction and adaptive image segmentation in linear array cameras, especially in complex backgrounds and uneven exposure conditions, leading to inaccurate workpiece extraction and affecting subsequent processing results.
A method based on directional potential energy field is adopted to obtain the vertices of the workpiece geometry through adaptive binarization and ensemble clustering. Combined with frequency domain analysis and rotation matrix correction, real-time distortion correction and accurate segmentation of the image are achieved.
It achieves real-time motion distortion correction and adaptive segmentation of linear scan camera images, improving the accuracy and flexibility of workpiece image processing, adapting to a wider range of rotation angles and various shapes of metal plates, and providing an efficient and reliable solution for online automatic inspection and quality control.
Smart Images

Figure CN119648591B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a method for workpiece image motion distortion correction and accurate segmentation based on a directional potential energy field. Background Technology
[0002] Motion distortion refers to the inaccurate and unrealistic visual representation in an image caused by the movement of the camera or the subject during the shooting process, such as rotation, sliding, or vertical shaking. Specifically, motion distortion can be divided into the following types: (1) Geometric distortion: This refers to the distortion and deformation of the shape of the object in the image caused by the vertical movement or shaking of the subject or camera during the shooting process. (2) Uneven stretching or compression: This refers to the stretching or compression of the object in the image caused by the uneven speed or fluctuation of the subject or camera during the shooting process, making some parts appear to be out of proportion.
[0003] Because line scan cameras scan line by line, if the object undulates vertically during shooting, the image portions captured by different scan lines will have differences in vertical position, resulting in geometric distortion in the final combined image. If the object's movement speed is uneven, or if fluctuations cause changes in the speed of certain parts, the distance between different scan lines will change, causing some parts of the image to be stretched or compressed, such as... Figure 2 As shown, traditional distortion correction methods are mainly for images captured by area scan cameras and are suitable for overall distortion correction. However, line scan images require line-by-line processing, increasing the complexity of the correction. Since line scan cameras are typically used in real-time applications, the correction algorithm must be completed in a very short time, and traditional distortion correction algorithms may not meet the real-time requirements.
[0004] Image segmentation refers to dividing an image into non-overlapping, meaningful sub-regions. Pixels within the same region have a certain correlation, while pixels in different regions exhibit certain differences. Essentially, it's the process of assigning the same label to pixels with similar properties. Typical image segmentation includes the following steps: image preprocessing, feature extraction, segmentation processing, post-processing, verification and evaluation, output, and application.
[0005] Existing adaptive binarization methods may not be able to fully handle the diverse features in complex backgrounds. For example, when there are different textures, colors, and lighting variations in the background, adaptive algorithms may misidentify a portion of the background as the foreground. This can lead to inaccurate artifact extraction and affect the effectiveness of subsequent processing. While some adaptive methods can handle local lighting variations, their performance remains limited when dealing with images with severe exposure unevenness. Uneven exposure can also cause significant changes in the grayscale distribution of an image, which traditional adaptive binarization methods may not be able to handle effectively, thus affecting the binarization quality of the image. Efficient adaptive binarization algorithms typically require processing large amounts of image data and calculating complex parameters, especially when processing high-resolution images, which can lead to long processing times. This computational complexity may limit the applicability of adaptive binarization methods in real-time applications.
[0006] In industrial image processing, acquiring the vertices of a workpiece's geometry is a crucial step in image segmentation, particularly vital for workpiece positioning and automated inspection. Accurate vertices help identify the workpiece's shape, size, and location, providing critical information for subsequent processing, assembly, and quality control. However, adaptive binarization methods in image preprocessing can only provide a simple outline of the workpiece. Especially when the workpiece's position on the production line is uncertain, the preprocessed outline may exhibit shape distortion and unclear edges, failing to provide precise vertex locations. Therefore, more sophisticated algorithms and techniques are needed to further refine the acquisition of contour points. Summary of the Invention
[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for workpiece image motion distortion correction and accurate segmentation based on a directional potential energy field. This method realizes real-time motion distortion correction, adaptive correction and dynamic segmentation of images obtained by a linear scan camera, so as to solve the problems of motion distortion, adaptive correction and dynamic cropping of metal plates in image processing.
[0008] The objective of this invention is achieved as follows: a method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field, comprising the following steps:
[0009] Step 1) Obtain the original image of the target to be segmented: Capture the image of the target object using a line scan industrial camera and a heavy-duty conveyor device to provide the original image to be processed for subsequent distortion correction and image segmentation;
[0010] Step 2) Analyze the periodic changes of the distorted image and establish a fluctuation model: By reading and converting the image to the frequency domain, analyze the spectrum to detect the frequency of distortion, and generate a fluctuation model based on the detected frequency and amplitude;
[0011] Step 3) Adaptive image preprocessing: The corrected image is preprocessed using an adaptive binarization method based on minimum fusion spatial correlation to remove the image background and reduce interference;
[0012] Step 4) Adaptive initialization of geometric contour vertices based on ensemble clustering of directed potential energy field: obtain the vertices of the workpiece geometry through edge detection and ensemble clustering based on directed potential energy field;
[0013] Step 5) Clustering to find vertices: Determine vertex coordinates by calculating the distance between each corner point in each class and the geometric center;
[0014] Step 6) Calculate the rotation angle, create the rotation matrix, and perform rotation correction: Select two adjacent vertices from the vertices and connect them with a straight line. Calculate the rotation matrix using the arctangent function.
[0015] Step 7) Set boundary dynamic cropping: Determine the cropping area by recalculating the vertex coordinates of the rotated image, and dynamically crop the image.
[0016] As a further limitation of the present invention, step 2) specifically includes:
[0017] 2-1) By reading the image data, converting it into a grayscale image and storing it, and then performing a two-dimensional discrete Fourier transform, a spectrum containing the intensity and phase information of various frequency components in the image is obtained; its mathematical expression is as follows:
[0018]
[0019] Where f(x,y) is the pixel value of the input image, F(u,v) is the frequency domain representation of the output, and M and N are the width and height of the image; For sine and cosine waves represented by complex exponents;
[0020] 2-2) Rearrange the frequency domain: Rearrange the frequency domain, moving low-frequency components to the center of the spectrum and high-frequency components to the edges;
[0021] 2-3) Compress the spectral amplitude through logarithmic transformation. The range of the spectral amplitude is compressed by taking the natural logarithm of each element of the amplitude spectrum and adjusting it to a suitable visualization scale.
[0022] 2-4) By analyzing the central region of the image spectrogram, detect the main frequency components and amplitudes of the image; analyze the main frequency components of the image to find the peak positions; calculate the coordinates of the frequency peaks in the entire spectrogram to determine the main frequency components in the image;
[0023] 2-5) Calculate the frequency of periodic structures in the image and set an amplitude value for subsequent image correction;
[0024] 2-6) Generate a wave model; simulate the periodic distortion in the image using a sine wave and generate a wave model based on a sine function to correct the image distortion; generate a set of horizontal offsets based on the image height for subsequent image correction; if the image height is rows, each row in the image is y i The i-th element is x i The adjusted amplitude is A, the frequency is f, and the wave model is expressed as:
[0025] y i = i, i∈{0,1,2,…,rows-1}
[0026]
[0027] 2-7) Perform inverse transformation correction; calculate the horizontal offset using the fluctuation model, and horizontally shift the pixels of each row to reverse the periodic distortion in the image, thereby achieving image correction.
[0028] As a further limitation of the present invention, step 3) specifically includes:
[0029] 3-1) Weighted grayscale conversion: Convert the distortion-corrected M×N resolution, RGB three-channel image I... clb The pixel value of the k-channel with coordinates (i,j) is denoted as... Then its weighted grayscale image I gray The pixel grayscale value at coordinate (i,j) is
[0030]
[0031] Where {α1,α2,α3} are weighting coefficients;
[0032] 3-2) The threshold is traversed using an adaptive binarization method that maximizes the continuity of foreground pixel coordinates. The binarization threshold is G. k Obtain the binarized image I bin The pixel grayscale value at coordinate (i,j) The process is as follows:
[0033]
[0034] Using the current threshold G k n was obtained k Given a number of foreground pixels, calculate the geometric center c of the current foreground. k :
[0035]
[0036] Let ch(i,j) represent the area of the envelope of all points (i,j) in two-dimensional space. The fusion spatial correlation index fco of the current image foreground is defined by the quotient of the average second-order central moments of the foreground pixels and ch(i,j).
[0037]
[0038] Based on the segmentation threshold, the average second-order central moment of the foreground pixels is calculated, the envelope gradient is calculated, and the minimum value of the fusion spatial correlation index fco is determined.
[0039] Traversing G k The appropriate binarization threshold G can be obtained by minimizing the correlation index fco in the fusion space within the parameter space, i.e.:
[0040]
[0041] The obtained threshold G is used for binarization to obtain the binarized image as the final result of step 3).
[0042] As a further limitation of the present invention, step 4) specifically includes:
[0043] 4-1) Obtain the image edge point set I by convolving the Canny operator with the grayscale image. edge Extending grayscale images using periodic mirroring;
[0044] 4-2) Add an initial particle set: Add an initial particle set P that is equally spaced and uniformly distributed in the space where the image is located. e If the initial particle's coordinates in the image coordinate system are... If the particle spacing is Δ, then the initial particle set P e Expressed as:
[0045]
[0046] 4-3) Obtain the internal particle set: Obtain the initial particle set P e The binarized image I obtained by preprocessing in step 3-2) bin Perform convolution to obtain the binarized image I. bin The particle set P inside the envelope:
[0047]
[0048] 4-4) Using the internal particle set P to compare the edge point set I edge Construct a directed potential energy field; solve for the magnitude, direction, and potential energy of the resultant force at all points in the edge point set;
[0049] 4-5) For the three indicators of resultant force magnitude, resultant force direction, and resultant potential energy, three basic clusters are formed using the K-means method; the number of clusters K is set to an integer K second only to the square root of the number of edge point sets. ini ,Right now:
[0050]
[0051] 4-6) After constructing the three basic clusters, cluster ensembles are achieved through cluster merging and sample allocation; cluster merging is achieved by calculating the inter-class similarity for any point in the edge point set. Class L i With class L j The similarity is determined by the coefficient J. i,j Characterization:
[0052]
[0053] Merge the two classes with the highest similarity, and in the i-th iteration, make the number of clusters K i =K i-1 -1; if class L i With class L j During merging, any data point Appear and In this case, a voting method is used for sample allocation; data points New label L new satisfy:
[0054]
[0055] The clustering and sample allocation are performed recursively until the similarity between any two classes is no higher than the threshold γ; at this point, the number of clusters is denoted as K. end Select the four classes that meet the conditions in sequence. Initialize the workpiece geometry vertex class:
[0056]
[0057] As a further limitation of the present invention, step 4-4) specifically includes solving for the magnitude, direction, and potential energy of the resultant force of all points in the edge point set:
[0058] Any point in particle set P (P x1 ,P y1 For edge point set I edge any point in the middle Generate repulsive force Its size is:
[0059]
[0060] Where k0 is the potential energy coefficient, and R0 is the limiting range of the repulsive force; when the particle (P x1 ,P y1 ) and point When the second moment exceeds the limit R0, there is no repulsive force between them; the repulsive force is a vector, its direction is from the point of origin to the point of application, and the sum of the forces at a single point of application is the vector sum; any point at the edge point concentration The vector of the resultant force of all particles is
[0061]
[0062] The angle between the resultant force vector and the unit vector (1,0) in the positive x-axis direction of the image coordinate system is:
[0063]
[0064] By particles (P) x1 ,P y1 In the potential field generated by ), the set of edge points I edge any point in the middle The magnitude of the potential energy is used express:
[0065]
[0066] The potential field generated by all particles in the internal particle set P at point The sum of potential energies at each location is:
[0067]
[0068] As a further limitation of the present invention, step 5) specifically includes:
[0069] 5-1) Calculate the Euclidean distance from the initial cluster centers to all corner points. Based on the calculated distances, assign the corner points to the clusters belonging to the nearest cluster centers. If there are two corner points x = (x1, x2, ..., x...), ... n ) and y = (y1, y2, ..., y n If x, y, then their Euclidean distance d(x,y) can be expressed as:
[0070]
[0071] 5-2) Detect all contours in the image: Binarize the image, identify and track edge points to determine the boundaries of objects in the image, extract the outer contours, ignore the nested inner contours, and remove redundant points; if the binary image is denoted as I(x,y), where the pixel values I(x,y)∈{0,255}, the binary image is represented as:
[0072]
[0073] The pixel coordinates on the contour are (x i ,y i If the k-th contour C is... k Expressed as:
[0074] C k ={(x1,y1),(x2,y2),…,(x n ,y n )}
[0075] Use linked list encoding to record the points (x) on the contour. i+1 ,y i+1 Let the displacement corresponding to the direction code be denoted as Δ(x,y), then:
[0076] (x i+1 ,y i+1 )=(x i ,y i )+Δ(x,y)
[0077] Then the set of all contours C:
[0078] C = {C1,C2,…,C} k}
[0079] 5-3) Select the largest contour. The contour with the largest area among all contours is the contour of the metal plate. If (x i ,y i ) is the outline C i Let the j-th point on the contour be m, and let m be the total number of points on the contour C. i Area A(C) i ) is represented as:
[0080]
[0081] 5-4) Calculate the geometric moments of the profile to determine its geometric center, i.e., its centroid; if the zeroth moment is denoted as M... 00 M 10 and M 01 Let be the first moment, used to calculate the x and y coordinates of the centroid. The formula for calculating the centroid is as follows:
[0082] x-coordinate of the centroid
[0083] y-coordinate of the centroid Select the corner point farthest from the geometric center from each list of corner points; that is, the vertex. In the corner point set P... i In the diagram, the corner point is denoted as (x... ij ,y ij ), then vertex (x) i,max ,y i,max ):
[0084]
[0085] Arrange them in the order of top left, top right, bottom left, and bottom right to obtain the four vertices of the metal plate.
[0086] As a further limitation of the present invention, step 6) specifically includes:
[0087] 6-1) Calculate the rotation angle. If there are two adjacent corner points p1 = (x1, y1) and p2 = (x2, y2), then the rotation angle θ is expressed as:
[0088] θ = arctan(y2-y1, x2-x1)
[0089] Then convert the radian value to an angle value:
[0090]
[0091] 6-2) Create a rotation matrix. If a point (x, y) is rotated by an angle θ around a center point (cx, cy) to obtain new coordinates (x', y'), then the translation amounts tx and ty are expressed as:
[0092] tx=(1-cosθ)·cx-sinθ·cy
[0093] ty = sinθ·cx + (1-cosθ)·cy
[0094] The final rotation matrix M is expressed as:
[0095]
[0096] 6-3) Rotation Correction: Apply a rotation matrix to the original image around the center point to rotate it by a specific angle, rotating the image to the desired position while maintaining the original size of the image; perform a rotation transformation on the original image according to the rotation matrix to generate a new rotated image, that is, use the affine transformation function to move each pixel in the source image to a new position in the target image by applying the affine transformation matrix.
[0097] As a further limitation of the present invention, step 7) specifically includes:
[0098] 7-1) Set the boundary value. The boundary value is the minimum distance between the edge of the cropped area and the vertex of the rotated image. Set the boundary value border to ensure that the cropped image can contain the target area.
[0099] border=α·min(w target ,h target )
[0100] Among them, w target h is the width of the metal plate in the image. target The height of the metal plate in the image is expressed in pixels, and the scaling factor α is a small quantity that satisfies 0 < α < 1.
[0101] 7-2) Determine the cropping region, recalculate the four vertices of the metal plate after image rotation and correction, and calculate the minimum and maximum values of the x and y coordinates of these vertices respectively. Let the image width be *width*, the height be *height*, and the boundary value be *border*. Then the boundary coordinates of the cropping region are represented as:
[0102] min x =max(0,min(x) i )+border)
[0103] max x =min(width,max(x) i )-border)
[0104] min y =max(0,min(y) i )+border)
[0105] max y =min(height,max(y) i )-border)
[0106] 7-3) Cropping the image: Based on the specified vertex coordinates, extract a sub-rectangular region from the two-dimensional array using array slicing, represented as:
[0107] cropped_image=rotated_image[min y :max y ,min x :max x ].
[0108] Compared with existing technologies, the beneficial effects of this invention, employing the above technical solutions, are as follows: By preprocessing the corrected image using an adaptive binarization method based on minimum fusion spatial correlation, the image background is removed and interference is reduced. This allows for more accurate and effective segmentation of rectangular workpiece images even under conditions of material reflection, uneven lighting, and complex backgrounds, achieving adaptive preprocessing that initially preserves geometric contour features. A directional potential energy field is constructed by adding a uniformly distributed particle set to the preprocessed image. The magnitude, direction, and potential energy value of the repulsive force at edge points are integrated and clustered to adaptively obtain an initial point set containing the vertices of the image's outer contour, overcoming the influence of uncertain workpiece poses on image segmentation during production lines. The geometric center and vertex positions of the metal plate are accurately calculated, and the skewed metal plate image is corrected using a rotation matrix. After correction, vertex detection and dynamic cropping are performed to achieve accurate correction and effective cropping of metal plate images, which is particularly suitable for rectangular metal plates with large rotation angles, non-centeredness, and narrow shape. The image is read and converted to the frequency domain, and the spectrum is analyzed to detect the frequency of distortion. A wave model is generated based on the detected frequency and amplitude, and the image is inversely transformed to correct the distortion. The corrected image is then displayed and saved. The image segmentation method of this invention can significantly improve the accuracy and flexibility of metal plate image processing, adapt to a wider range of rotation angles and various shapes of metal plates, and provide an efficient and reliable solution for online automatic inspection and quality control of metal plates. Attached Figure Description
[0109] Figure 1 This is a schematic diagram illustrating the application environment of the image segmentation method of the present invention.
[0110] Figure 2 This is a schematic diagram of motion distortion in traditional images.
[0111] Figure 3 This is a schematic diagram of the process of the present invention.
[0112] Figure 4 This is a comparison chart of the image adaptive preprocessing effects of the present invention.
[0113] Figure 5 This is a diagram illustrating the adaptive initialization effect of geometric contour vertices based on directed potential field integrated clustering, as described in this invention.
[0114] Figure 6 This is an example of the image rotation correction and cropping method of the present invention. Detailed Implementation
[0115] The image segmentation method provided in this example can be applied to, for example... Figure 1 The application environment shown.
[0116] Industrial cameras, in particular, refer to line scan cameras with relative motion errors. Line scan cameras scan line by line, acquiring a complete image by continuously moving the object or the camera itself. Because they scan line by line, if the object fluctuates vertically during the scanning process (such as a metal plate being transported), the image portions captured by different scan lines will have different vertical positions, resulting in slightly different images for each line. Furthermore, if the object's speed is uneven during transport, or if fluctuations cause changes in the speed of certain parts, the distance between scan lines will change, all leading to motion distortion. This distortion manifests in the image as wavy twists or periodic deformations.
[0117] The conveying devices mentioned here refer to industrial devices capable of transporting large areas of heavy objects, including belt conveyors, roller conveyors, chain conveyors, and chain plate conveyors. These devices move objects from one location to another, fulfilling material transport needs on the production line while also facilitating image acquisition by line scan cameras. The vertical vibrations caused by the inherent precision of the conveying device will also cause vertical vibrations in the objects themselves. The motion distortion resulting from this vibration is similar to the motion distortion caused by the vertical fluctuations of the object during transport, and will not be elaborated upon here.
[0118] The terminal can be an industrial computer, or a regular desktop computer, laptop computer, tablet computer, etc., as long as it can provide a code running environment and software running conditions.
[0119] like Figure 3 The workpiece image motion distortion correction and accurate segmentation method based on directional potential energy field, as shown, includes the following steps:
[0120] Step 1) Obtain the original image of the target to be segmented: For the problem of image acquisition and detection of large-size 3D profiles, a line scan industrial camera is used in conjunction with a heavy-duty conveyor to capture the image of the target object, providing the original image to be processed for subsequent distortion correction and image segmentation;
[0121] Step 2) Analyze the periodic changes of the distorted image and establish a fluctuation model: By reading and converting the image to the frequency domain, analyze the spectrum to detect the frequency of distortion, and generate a fluctuation model based on the detected frequency and amplitude;
[0122] 2-1) By reading the image data, converting it into a grayscale image and storing it, and then performing a two-dimensional discrete Fourier transform, a spectrum containing the intensity and phase information of various frequency components in the image is obtained; its mathematical expression is as follows:
[0123]
[0124] Where f(x,y) is the pixel value of the input image; F(u,v) is the frequency domain representation of the output; M and N are the width and height of the image; These are sine and cosine waves represented by complex exponents.
[0125] 2-2) Rearrange the frequency domain. In the resulting frequency domain, low-frequency components and high-frequency components are usually distributed in different positions of the matrix. The default arrangement is that the low-frequency components are located at the four corners of the matrix and the high-frequency components are located in the center of the matrix. However, this default arrangement is not conducive to spectrum observation and analysis. Therefore, the frequency domain is rearranged to move the low-frequency components to the center of the spectrum and the high-frequency components to the edge.
[0126] 2-3) Compressing the spectral amplitude through logarithmic transformation: If the frequency of the spectrum is denoted as a complex number z = a + bi, its amplitude is... Logarithmic transformation can compress the range of spectral amplitude to a smaller range. Taking the natural logarithm log(x) of each element of the amplitude spectrum adjusts it to a more suitable visualization scale, thereby enhancing the visualization effect and making the details more obvious.
[0127] 2-4) By analyzing the central region of the image spectrogram, detect the main frequency components and amplitudes of the image; analyze the main frequency components of the image, find the peak positions, and calculate their coordinates in the entire spectrogram to determine the main frequency components in the image.
[0128] 2-5) Calculate the frequency of the periodic structure in the image and set an amplitude value for subsequent image correction.
[0129] 2-6) Generate a wave model: Simulate periodic distortion in the image using a sine wave and generate a wave model based on a sine function to correct image distortion; Generate a set of horizontal offsets based on the image height for subsequent image correction; If the image height is rows, each row in the image is y i The i-th element is x i With an adjusted amplitude of A and a frequency of f, the wave model can be expressed as:
[0130] y i = i, i∈{0,1,2,…,rows-1}
[0131]
[0132] 2-7) Perform inverse transform correction. Calculate the horizontal offset using a wave model, and horizontally shift the pixels in each row to reverse the periodic distortion in the image, thus achieving image correction. Let the pixel value in the i-th row and j-th column of the corrected image be denoted as corrected_img(i,j), and the pixel value in the i-th row and j-th column of the original image be denoted as img(i,j). The displacement in the i-th row is denoted as x. i If the image width is C, then
[0133] corrected_img(i,j)=img(i,(j+x i )modC).
[0134] Step 3) Adaptive Image Preprocessing: To address issues such as cluttered backgrounds and uneven exposure in workpiece images that affect the accuracy of workpiece extraction, an adaptive binarization method based on minimum fusion spatial correlation is used to preprocess the corrected image to remove the background and reduce interference.
[0135] 3-1) Weighted grayscale conversion: Convert the distortion-corrected M×N resolution, RGB three-channel image I... clb The pixel value of the k-channel with coordinates (i,j) is denoted as... Then its weighted grayscale image I gray The pixel grayscale value at coordinate (i,j) is
[0136]
[0137] Where {α1, α2, α3} are weighting coefficients, and in this embodiment, α1 = 0.299, α2 = 0.578, and α3 = 0.114 are selected;
[0138] 3-2) The photographed workpiece is treated as a single, continuous, and complete object. The pixels in the workpiece image exhibit positional correlation. Based on this characteristic, an adaptive binarization method is proposed to maximize the continuity of foreground pixel coordinates. If the binarization threshold is G... k Obtain the binarized image I bin The pixel grayscale value at coordinate (i,j) The process is as follows:
[0139]
[0140] Using the current threshold G k n was obtained k Given a number of foreground pixels, calculate the geometric center c of the current foreground. k :
[0141]
[0142] Let ch(i,j) represent the area of the envelope of all points (i,j) in two-dimensional space. The fusion spatial correlation index fco of the current image foreground is defined by the quotient of the average second-order central moments of the foreground pixels and ch(i,j).
[0143]
[0144] When shooting against a cluttered background, if the segmentation threshold is small, many background points will be misclassified as foreground, resulting in a segmentation result containing numerous scattered background points, while the average second central moment of the foreground pixels remains essentially unchanged. In this case, the envelope area is large, thus the fco value remains low. If the segmentation threshold is moderate, foreground points will gradually be misclassified as background, and the average second central moment of the foreground pixels will remain essentially unchanged. As the threshold increases, the number of edge pixels far from the illumination center gradually decreases, the envelope area gradually decreases, and the fco value increases. If the segmentation threshold is large, a large number of foreground points will be misclassified as background, the remaining foreground points will be more concentrated, the average second central moment will decrease rapidly, while the envelope area will remain essentially unchanged, thus reducing the fusion spatial correlation index fco value.
[0145] Traversing G k The appropriate binarization threshold G can be obtained by minimizing the value of fco in the parameter space, i.e.:
[0146]
[0147] The obtained threshold G is used for binarization, and the binarized image is obtained as the final result of this step.
[0148] In this embodiment, the grayscale image of the workpiece, the fco variation with threshold, the comparison chart of the traditional OTSU method, the adaptive binarization method, and the method proposed in this invention are shown below. Figure 4 As shown, the proposed method can more accurately and effectively segment rectangular workpiece images under conditions of material reflection, uneven lighting, and complex backgrounds, achieving adaptive preprocessing that initially preserves geometric contour features.
[0149] Step 4) Adaptive initialization of geometric contour vertices based on ensemble clustering of directed potential energy field: Vertices of the workpiece geometry are obtained through edge detection and ensemble clustering based on directed potential energy field. Obtaining accurate vertices is a crucial step in image segmentation. However, step 3) adaptive preprocessing only provides a simple but imprecise image contour, so further image processing is still needed to obtain the precise positions of the vertices. Step 4) By constructing a directed potential energy field by adding a uniformly distributed set of particles to the preprocessed image, ensemble clustering is performed on the magnitude, direction, and potential energy value of the repulsive force at the edge points to adaptively obtain an initial point set containing the vertices of the image's outer contour, overcoming the influence of the uncertain pose of the workpiece on the production line on image segmentation.
[0150] 4-1) Obtain the image edge point set I by convolving the Canny operator with the grayscale image. edge To avoid edge effects caused by convolution operations in image boundary regions, the grayscale image is extended using periodic mirroring. Since the Canny edge detection technology is mature and stable, and is not the main focus of this invention, it will not be described in detail here.
[0151] 4-2) Add an initial particle set: Add an initial particle set P that is equally spaced and uniformly distributed in the space where the image is located. e If the initial particle's coordinates in the image coordinate system are... If the particle spacing is Δ, then the initial particle set P e It can be expressed as:
[0152]
[0153] 4-3) Obtain the internal particle set, and set the initial particle set P e The binary image I obtained by preprocessing in step (3) bin Perform convolution to obtain a binary image I. bin The particle set P inside the envelope:
[0154]
[0155] 4-4) Using the internal particle set P to compare the edge point set I edge Construct a directed potential energy field; solve for the magnitude, direction, and potential energy of the resultant force at all points in the edge point set;
[0156] Any point in particle set P (P x1 ,P y1 For edge point set I edge any point in the middle Generate repulsive force Its size is:
[0157]
[0158] In the above formula, k0 is the potential energy coefficient, and R0 is the limiting range of the repulsive force; obviously, when the particle (P x1 ,P y1 ) and point When the second moment exceeds the limit R0, there is no repulsive force between them. The repulsive force is a vector, its direction pointing from the point of origin to the point of application; the sum of the forces at a single point of application is the vector sum; any point at the edge point concentration... The vector of the resultant force of all particles is
[0159]
[0160] The angle between the resultant force vector and the unit vector (1,0) in the positive x-axis direction of the image coordinate system is:
[0161]
[0162] By particles (P) x1 ,P y1 In the potential field generated by ), the set of edge points I edgeany point in the middle The magnitude of the potential energy is used express:
[0163]
[0164] Potential energy is a scalar, therefore the sum of potential energy values is an algebraic sum; the potential field generated by all particles in the internal particle set P at point P... The sum of potential energies at each location is:
[0165]
[0166] 4-5) Use the Kmeans method to form three base clusters for each of these three indicators; since the Kmeans clustering technique is a mature algorithm, it will not be described in detail in this explanation.
[0167] Set the number of clusters K to an integer K that is second only to the square root of the number of edge point sets. ini ,Right now:
[0168]
[0169] 4-6) After constructing the three basic clusters, cluster ensembles are achieved through cluster merging and sample allocation. Cluster merging is implemented by calculating the inter-cluster similarity, which is applied to any point in the edge point set. Class L i With class L j The similarity is determined by the coefficient J. i,j Characterization:
[0170]
[0171] Merge the two classes with the highest similarity, and in the i-th iteration, make the number of clusters K i =K i-1 -1. If class L i With class L j During merging, any data point Appear and In this case, a voting method is used for sample allocation; data points New label L new satisfy:
[0172]
[0173] The clustering and sample allocation are performed recursively until the similarity between any two classes is no higher than the threshold γ; at this point, the number of clusters is denoted as K. end In this embodiment, the threshold γ is set to 0.9; four classes that satisfy the following conditions are selected sequentially. Initialize the workpiece geometry vertex class:
[0174]
[0175] In this embodiment, the grayscale image of the workpiece, the Canny edge point set, the initial particle set, the internal particle set, the magnitude of the repulsion force of the edge point set, the repulsion angle and potential energy clustering diagram, and the ensemble clustering diagram are shown in Figure 5. Clearly, the three evaluation metrics designed in this invention can efficiently distinguish between internal patterned edges, straight edges, and geometric contour vertices. Specifically, the class containing the geometric contour vertices has the largest average resultant force and the smallest average potential energy, while the direction of the resultant force is used to distinguish different contour vertices. In real image processing, using only these three metrics for discrimination is significantly affected by image noise. Therefore, this invention utilizes an ensemble clustering method to further improve the accuracy, stability, and robustness of geometric contour vertex type discrimination.
[0176] Step 5) Clustering to find vertices: Determine vertex coordinates by calculating the distance between each corner point in each class and the geometric center;
[0177] 5-1) Calculate the Euclidean distance from the initial cluster centers to all corner points. Based on the calculated distances, assign the corner points to the clusters belonging to the nearest cluster centers. If there are two corner points x = (x1, x2, ..., x...), ... n ) and y = (y1, y2, ..., y n If x, y, then their Euclidean distance d(x,y) can be expressed as:
[0178]
[0179] The aforementioned corner points must include the four vertices of the metal plate. In order to find the vertices in each corner point list, it is necessary to obtain the geometric center of the metal plate.
[0180] 5-2) Detect all contours in the image, binarize the image, identify and track edge points to determine the boundaries of objects in the image, extract the outer contours, ignore the nested inner contours, and remove redundant points; if the binary image is denoted as I(x,y), where the pixel values I(x,y)∈{0,255}, the binary image can be represented as:
[0181]
[0182] The pixel coordinates on the contour are (x i ,y i If the k-th contour C is... k Expressed as:
[0183] C k ={(x1,y1),(x2,y2),…,(x n ,y n )}
[0184] Use linked list encoding to record the points (x) on the contour. i+1 ,y i+1 Let the displacement corresponding to the direction code be denoted as Δ(x,y), then:
[0185] (x i+1 ,y i+1 )=(x i ,y i )+Δ(x,y)
[0186] Then the set of all contours C:
[0187] C = {C1,C2,…,C} k}
[0188] 5-3) Select the largest contour. The contour with the largest area among all contours is the contour of the metal plate. If (x i ,y i ) is the outline C i Let the j-th point on the contour be m, and let m be the total number of points on the contour C. i Area A(C) i This can be represented as:
[0189]
[0190] 5-4) Calculate the geometric moments of the profile to determine its geometric center (centroid). If the zeroth moment is denoted as M... 00 M 10 and M 01 Let be the first moment, used to calculate the x and y coordinates of the centroid. The formula for calculating the centroid is as follows:
[0191] x-coordinate of the centroid
[0192] y-coordinate of the centroid Select the corner point farthest from the geometric center from each list of corner points; that is, the vertex. In the corner point set P... i In the diagram, the corner point is denoted as (x... ij ,y ij ), then vertex (x) i,max ,y i,max ):
[0193]
[0194] Arranged in the order of top left, top right, bottom left, and bottom right, the four vertices of the metal plate are obtained.
[0195] Step 6) Calculate the rotation angle, create the rotation matrix, and perform rotation correction: Select two adjacent vertices from the vertices and connect them with a straight line. Calculate the rotation matrix using the arctangent function.
[0196] 6-1) Calculate the rotation angle. If there are two adjacent corner points p1 = (x1, y1) and p2 = (x2, y2), then the rotation angle θ can be expressed as:
[0197] θ = arctan(y2-y1, x2-x1)
[0198] Then convert the radian value to an angle value:
[0199]
[0200] 6-2) Create a rotation matrix. If a point (x, y) is rotated by an angle θ around a center point (cx, cy) to obtain new coordinates (x', y'), then the translation amounts tx and ty can be expressed as:
[0201] tx=(1-cosθ)·cx-sinθ·cy
[0202] ty = sinθ·cx + (1-cosθ)·cy
[0203] The final rotation matrix M can be expressed as:
[0204]
[0205] 6-3) Rotation Correction: Apply a rotation matrix to the original image around its center point to rotate it by a specific angle, bringing the image to the desired position while maintaining its original size. Then, perform a rotation transformation on the original image using the rotation matrix to generate a new rotated image. This is achieved by using an affine transformation function and applying an affine transformation matrix to move each pixel in the source image to its new position in the target image. Specifically, for each pixel (x, y) in the source image, the transformation matrix calculates its new position (x', y') in the target image.
[0206]
[0207] Among them, a 11 a 12 a 21 a 22 These are the elements in the rotation matrix, where b1 and b2 are the translation amounts.
[0208] Step 7) Set dynamic cropping boundaries: Determine the cropping area by recalculating the vertex coordinates of the rotated image, and dynamically crop the image;
[0209] 7-1) Set the boundary value. The boundary value is the minimum distance between the edge of the cropped area and the vertex of the rotated image. Set the boundary value to ensure that the cropped image can contain the target area while avoiding too much background interference.
[0210] border=α·min(w target ,h target )
[0211] Among them, w target h is the width of the metal plate in the image. target The height of the metal plate in the image is expressed in pixels. The scaling factor α is a small value that satisfies 0 < α < 1. In this embodiment, α = 0.01 is used.
[0212] 7-2) Determine the cropping region, recalculate the four vertices of the metal plate after image rotation and correction, and calculate the minimum and maximum values of the x and y coordinates of these vertices respectively. Let the image width be width, the height be height, and the boundary value be border. Then the boundary coordinates of the cropping region are represented as:
[0213] min x =max(0,min(x) i )+border)
[0214] max x =min(width,max(x) i )-border)
[0215] min y =max(0,min(y) i )+border)
[0216] max y =min(height,max(y) i )-border)
[0217] 7-3) Image cropping: Based on specified vertex coordinates, extract a sub-rectangular region from a two-dimensional array using array slicing, which can be represented as:
[0218] cropped_image=rotated_image[min y :max y ,min x :max x ];
[0219] In this embodiment, the workpiece rotation correction and trimming diagram are as follows: Figure 6 As shown; obviously, the rotation correction method designed in this invention can efficiently and accurately correct the target object in the image and segment it from the background. It can handle displacement and rotation in any direction and at any angle, and has high stability and strong robustness.
[0220] This invention provides a method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field. It preprocesses the corrected image using an adaptive binarization method based on minimum fusion spatial correlation to remove background and reduce interference. This method can more accurately and effectively segment rectangular workpiece images even under conditions of material reflection, uneven lighting, and complex backgrounds, achieving adaptive preprocessing that initially preserves geometric contour features. A directed potential energy field is constructed by adding a uniformly distributed particle set to the preprocessed image. The magnitude, direction, and potential energy value of the repulsive force at edge points are integrated and clustered to adaptively obtain an initial point set containing the vertices of the image's outer contour, overcoming the influence of uncertain workpiece poses on the production line on image segmentation. The geometric center and vertex positions of the metal plate are accurately calculated, and the skewed metal plate image is corrected using a rotation matrix. After correction, vertex detection and dynamic cropping are performed to achieve accurate correction and effective cropping of the metal plate image, making it particularly suitable for rectangular metal plates with large rotation angles, non-centered designs, and narrow shapes.
[0221] This invention is not limited to the above embodiments. Based on the technical solutions disclosed in this invention, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.
Claims
1. A method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field, characterized in that, Includes the following steps: Step 1) Obtain the original image of the target to be segmented: Capture the image of the target object using a line scan industrial camera and a heavy-duty conveyor device to provide the original image to be processed for subsequent distortion correction and image segmentation; Step 2) Analyze the periodic changes of the distorted image and establish a wave model: By reading and converting the image to the frequency domain, analyze the spectrum to detect the frequency of distortion, generate a wave model based on the detected frequency and amplitude, and obtain the corrected image through wave model processing; Step 3) Adaptive Image Preprocessing: The corrected image is preprocessed using an adaptive binarization method based on minimum fusion spatial correlation to remove background and reduce interference. The fusion spatial correlation is represented by ch(i,j) to depict the envelope area of all two-dimensional spatial points (i,j). The fusion spatial correlation index fco of the current image foreground is defined by the quotient of the average second-order central moment of the foreground pixels and ch(i,j). Among them, c k n represents the geometric center of the current foreground. k Indicates the number of foreground pixels. Represents the binarized image I bin The pixel grayscale value with coordinates (i,j); Step 4) Adaptive initialization of geometric contour vertices based on ensemble clustering of directed potential energy field: The vertices of the workpiece geometry are obtained through edge detection and ensemble clustering based on directed potential energy field; wherein, a directed potential energy field is constructed by adding a uniformly distributed set of particles to the preprocessed image, and the magnitude, direction and potential energy value of the repulsive force of the edge points are ensemble clustered to adaptively obtain the initial set of points containing the vertices of the outer contour of the image. Step 5) Clustering to find vertices: Determine vertex coordinates by calculating the distance between each corner point in each class and the geometric center; Step 6) Calculate the rotation angle, create the rotation matrix, and perform rotation correction: Select two adjacent vertices from the vertices and connect them with a straight line. Calculate the rotation matrix using the arctangent function. Step 7) Set boundary dynamic cropping: Determine the cropping area by recalculating the vertex coordinates of the rotated image, and dynamically crop the image.
2. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 1, characterized in that, Step 2) specifically includes: 2-1) By reading the image data, converting it into a grayscale image and storing it, and then performing a two-dimensional discrete Fourier transform, a spectrum containing the intensity and phase information of various frequency components in the image is obtained; its mathematical expression is as follows: Where f(x,y) is the pixel value of the input image, F(u,v) is the frequency domain representation of the output, and M and N are the width and height of the image; For sine and cosine waves represented by complex exponents; 2-2) Rearrange the frequency domain: Rearrange the frequency domain, moving low-frequency components to the center of the spectrum and high-frequency components to the edges; 2-3) Compress the spectral amplitude through logarithmic transformation. The range of the spectral amplitude is compressed by taking the natural logarithm of each element of the amplitude spectrum and adjusting it to a suitable visualization scale. 2-4) By analyzing the central region of the image spectrogram, detect the main frequency components and amplitudes of the image; analyze the main frequency components of the image to find the peak positions; calculate the coordinates of the frequency peaks in the entire spectrogram to determine the main frequency components in the image; 2-5) Calculate the frequency of periodic structures in the image and set an amplitude value for subsequent image correction; 2-6) Generate a wave model; simulate the periodic distortion in the image using a sine wave and generate a wave model based on a sine function to correct the image distortion; generate a set of horizontal offsets based on the image height for subsequent image correction; if the image height is rows, each row in the image is y i The i-th element is x i The adjusted amplitude is A, the frequency is f, and the wave model is expressed as: y i =i,i∈{0,1,2,…,rows-1} 2-7) Perform inverse transformation correction; calculate the horizontal offset using the fluctuation model, and horizontally shift the pixels of each row to reverse the periodic distortion in the image, thereby achieving image correction.
3. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 1, characterized in that, Step 3) specifically includes: 3-1) Weighted grayscale conversion: Convert the distortion-corrected M×N resolution, RGB three-channel image I... clb The pixel value of the k-channel with coordinates (i,j) is denoted as... Then its weighted grayscale image I gray The pixel grayscale value at coordinate (i,j) is Where {α1,α2,α3} are weighting coefficients; 3-2) The threshold is traversed using an adaptive binarization method that maximizes the continuity of foreground pixel coordinates. The binarization threshold is G. k Obtain the binarized image I bin The pixel grayscale value at coordinate (i,j) The process is as follows: Using the current binarization threshold G k n was obtained k Given a number of foreground pixels, calculate the geometric center c of the current foreground. k : Let ch(i,j) represent the area of the envelope of all points (i,j) in two-dimensional space. The fusion spatial correlation index fco of the current image foreground is defined by the quotient of the average second-order central moments of the foreground pixels and ch(i,j). Based on the binarization threshold, calculate the average second-order central moment of the foreground pixels, calculate the envelope area, and determine the minimum value of the fusion spatial correlation index fco. Traversing G k The appropriate binarization threshold G can be obtained by minimizing the value of the correlation index fco in the fusion space within the parameter space. The obtained threshold G is used for binarization to obtain the binarized image as the final result of step 3).
4. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 1, characterized in that, Step 4) specifically includes: 4-1) Obtain the image edge point set I by convolving the Canny operator with the grayscale image. edge Extending grayscale images using periodic mirroring; 4-2) Add an initial particle set: Add an initial particle set P that is equally spaced and uniformly distributed in the space where the image is located. e If the initial particle's coordinates in the image coordinate system are... And if the particle spacing is Δ, then the initial particle set P e Expressed as: 4-3) Obtain the internal particle set: Obtain the initial particle set P e The binarized image I obtained by preprocessing in step 3-2) bin Perform convolution to obtain the binarized image I. bin The particle set P inside the envelope; 4-4) Using the internal particle set P to compare the edge point set I edge Construct a directed potential energy field; solve for the magnitude, direction, and potential energy of the repulsive force at all points in the edge point set; 4-5) For the three indicators of repulsive force magnitude, repulsive force direction, and repulsive potential energy, three basic clusters are formed using the K-means method; the initial number of clusters K is set to an integer K second only to the square root of the number of edge point sets. ini ,Right now: in, Let be any point in the edge point set; 4-6) After constructing the three basic clusters, cluster ensembles are achieved through cluster merging and sample allocation; cluster merging is achieved by calculating the inter-class similarity for any point in the edge point set. Class L i With class L j The similarity is determined by the coefficient J. i,j Characterization: Merge the two classes with the highest similarity, and in the i-th iteration, make the number of clusters K i =K i-1 -1; if class L i With class L j During merging, any data point Appear and In this case, a voting method is used for sample allocation; data points New label L new satisfy: The clustering and sample allocation are performed recursively until the similarity between any two classes is no higher than the threshold γ; at this point, the number of clusters is denoted as K. end Select the four classes that meet the conditions in turn. Initialize the workpiece geometry vertex class: in, Represents any point in the edge point set The vector of repulsive forces from all particles. Point The sum of potential energy possessed at a given point.
5. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 4, characterized in that, Step 4-4) specifically includes solving for the magnitude, direction, and potential energy of the repulsive force at all points in the edge point set: Any point in particle set P (P x1 ,P y1 For edge point set I edge any point in the middle Generate repulsive force Its size is: Where k0 is the potential energy coefficient, and R0 is the limiting range of the repulsive force; when the particle (P x1 ,P y1 ) and point When the second moment exceeds the limit R0, there is no repulsive force between them; the repulsive force is a vector, its direction is from the point of origin to the point of application, and the sum of the forces at a single point of application is the vector sum; any point at the edge point concentration The vector of repulsive forces from all particles is The angle between the repulsive force vector and the unit vector (1,0) in the positive x-axis direction of the image coordinate system is: By particles (P) x1 ,P y1 In the potential field generated by ), the set of edge points I edge any point in the middle The magnitude of the potential energy is used express: The potential field generated by all particles in the internal particle set P at point The sum of potential energies at each location is:
6. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 1, characterized in that, Step 5) specifically includes: 5-1) Calculate the Euclidean distance from the initial cluster centers to all corner points. Based on the calculated distances, assign the corner points to the clusters belonging to the nearest cluster centers. If there are two corner points x = (x1, x2, ..., x...), ... n ) and y = (y1, y2, ..., y n If x, y, then their Euclidean distance d(x,y) can be expressed as: 5-2) Detect all contours in the image: Binarize the image, identify and track edge points to determine the boundaries of objects in the image, extract the outer contours, ignore the nested inner contours, and remove redundant points; if the binary image is denoted as I(x,y), where the pixel values I(x,y)∈{0,255}, the binary image is represented as: The pixel coordinates on the contour are (x i ,y i If the k-th contour C is... k Expressed as: C k ={(x1,y1),(x2,y2),…,(x n ,y n )} Use linked list encoding to record the points (x) on the contour. i+1 ,y i+1 Let the displacement corresponding to the direction code be denoted as Δ(x,y), then: (x i+1 ,and i+1 )=(x i ,and i )+Δ(x,y) Then the set of all contours C: C={C1,C2,…,C k }; 5-3) Select the largest contour. The contour with the largest area among all contours is the contour of the workpiece metal plate. If (x i ,y i ) is the outline C i Let the j-th point on the contour be m, and let m be the total number of points on the contour C. i Area A(C) i ) is represented as: 5-4) Calculate the geometric moments of the profile to determine its geometric center, i.e., its centroid; if the zeroth moment is denoted as M... 00 M 10 and M 01 Let be the first moment, used to calculate the x and y coordinates of the centroid. The formula for calculating the centroid is as follows: x-coordinate of the centroid y-coordinate of the centroid Select the corner point farthest from the geometric center from each list of corner points; that is, the vertex. In the corner point set P... i In the diagram, the corner point is denoted as (x... ij ,y ij ), then vertex (x) i,max ,y i,max ): Arrange them in the order of top left, top right, bottom left, and bottom right to obtain the four vertices of the workpiece metal plate.
7. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 1, characterized in that, Step 6) specifically includes: 6-1) Calculate the rotation angle. If there are two adjacent corner points p1 = (x1, y1) and p2 = (x2, y2), then the rotation angle θ is expressed as: θ = arctan(y2-y1, x2-x1), where θ is the value in radians; Then convert the radian value to an angle value: 6-2) Create a rotation matrix. If a point (x, y) is rotated by an angle θ around a center point (cx, cy) to obtain new coordinates (x', y'), then the translation amounts tx and ty are expressed as: tx=(1-cosθ)·cx-sinθ·cy ty = sinθ·cx + (1-cosθ)·cy The final rotation matrix M is expressed as: 6-3) Rotation Correction: Apply a rotation matrix to the original image around the center point to rotate it by a specific angle, rotating the image to the desired position while maintaining the original size of the image; perform a rotation transformation on the original image according to the rotation matrix to generate a new rotated image, that is, use the affine transformation function to move each pixel in the source image to a new position in the target image by applying the affine transformation matrix.
8. The method for workpiece image motion distortion correction and accurate segmentation based on a directed potential energy field according to claim 1, characterized in that, Step 7) specifically includes: 7-1) Set the boundary value. The boundary value is the minimum distance between the edge of the cropped area and the vertex of the rotated image. Set the boundary value border to ensure that the cropped image can contain the target area. border=α·min(w target ,h target ) Among them, w target h represents the width of the metal plate of the workpiece in the image. target The height of the workpiece metal plate in the image is expressed in pixels, and the scaling factor α is a small quantity that satisfies 0 < α < 1. 7-2) Determine the clipping region, recalculate the four vertices of the workpiece metal plate after image rotation and correction, and calculate the minimum and maximum values of the x and y coordinates of these vertices respectively. Let the image width be width, the height be height, and the boundary value be border. Then the boundary coordinates of the clipping region are represented as: my x =max(0,min(x i )+border) max x =min(width,max(x i )-border) my y =max(0,min(y i )+border) max y =min(height,max(y i )-border) 7-3) Cropping the image: Based on the specified vertex coordinates, extract a sub-rectangular region from the two-dimensional array using array slicing, represented as: cropped_image=rotated_image[min y :max y ,min x :max x ]。