Printing piece flaw detection method and system based on machine vision

Abstract

The invention provides a printing piece defect detection method and system based on machine vision, and relates to the technical field of machine vision, and the method comprises the steps: determining an optimal detection scale through image enhancement processing, multi-scale gradient pyramid construction and mutual information calculation, extracting key points through a curvature value, and carrying out the shape discrimination. And finally, generating a laser etching compensation path based on defect characteristics and performing layered compensation processing. According to the method, the automobile part surface printing flaws can be accurately identified, the compensation strategy is adaptively generated according to the defect types, and the defect repairing precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to machine vision technology, and particularly to a method and system for detecting defects on printed parts based on machine vision. Background Art

[0002] With the continuous improvement of the requirements for the surface quality of parts in the automotive manufacturing industry, the detection and repair of printing defects have become key process links. The existing manual detection methods have problems such as low efficiency and poor consistency. Although the automatic detection technology based on machine vision has been applied, the detection accuracy under complex curved surfaces and diverse defect types still needs to be improved.

[0003] At present, machine vision detection mainly uses image processing with a single scale and a defect segmentation method with a fixed threshold, which is difficult to adapt to printing defects of different sizes and shapes. At the same time, the traditional defect feature extraction method has poor recognition effect for defect regions with blurred edges and complex textures, and lacks an effective description of the spatial distribution characteristics of defects, affecting the accuracy of subsequent repair and processing.

[0004] Therefore, there is an urgent need for a method for detecting defects on printed parts that can adaptively process multi-scale image features, accurately extract defect edge contours, accurately identify defect types, and generate compensation processing trajectories. Summary of the Invention

[0005] The embodiments of the present invention provide a method and system for detecting defects on printed parts based on machine vision, which can solve the problems in the prior art.

[0006] In the first aspect of the embodiments of the present invention, a method for detecting defects on printed parts based on machine vision is provided, including: Collecting an original image of the printed area on the surface of the automotive part to be detected through an industrial camera and performing image enhancement processing to obtain an enhanced detection image; Constructing a multi-scale gradient pyramid for the enhanced detection image, extracting the histogram of gradient directions at each scale, calculating the mutual information value of the histograms of gradient directions between adjacent scales, determining the optimal detection scale, and performing defect region segmentation at the optimal detection scale to obtain candidate defect regions; Performing adaptive binarization processing on the candidate defect regions, extracting defect edge contour points, calculating the curvature values of the defect edge contour points, taking the contour points with curvature values greater than a preset curvature threshold as key points, discriminating the defect shape based on the spatial distribution characteristics of the key points to obtain a defect region, and extracting and classifying texture features of the defect region to obtain the defect type; Calculate the centroid coordinates of the defect based on the key point distribution in the defect area as the origin, establish a polar coordinate system, calculate the angle - radius distribution of the defect boundary curve, generate a sequence of laser engraving compensation path points according to the surface curvature characteristics of the automotive part, and set the laser engraving compensation depth parameter according to the defect type. Combine the sequence of laser engraving compensation path points and the laser engraving compensation depth parameter to generate three - dimensional compensation trajectory data for layer - by - layer compensation machining of the printing defects on the surface of the automotive part.

[0007] In an alternative embodiment, Construct a multi - scale gradient pyramid for the enhanced detection image, extract the histogram of gradient directions at each scale, calculate the mutual information value between the histograms of gradient directions at adjacent scales, determine the optimal detection scale, and perform defect area segmentation at the optimal detection scale to obtain candidate defect areas including: Perform multi - scale decomposition on the enhanced detection image, convolve the image with a variable - scale Gaussian kernel function, dynamically adjust the Gaussian kernel parameters based on the inter - layer gradient consistency of the convolution result, and construct a multi - scale gradient pyramid; Calculate the horizontal gradient and vertical gradient of pixel points at each scale layer of the multi - scale gradient pyramid, determine the gradient amplitude and direction, divide the gradient direction space into multiple intervals, and count the number of gradient points in each interval to construct a histogram of gradient directions; Construct a joint probability distribution matrix, normalize the joint probability distribution matrix, calculate the mutual information value between the histograms of gradient directions at adjacent scale layers based on the normalized joint probability, analyze the trend curve of the mutual information value changing with the scale, extract the position of the mutation point of the trend curve, and determine the scale layer with the largest mutual information mutation as the optimal detection scale; On the image layer corresponding to the optimal detection scale, calculate the local gradient amplitude of the image pixel points, use the pixel points with gradient amplitude greater than the preset gradient threshold as seed points, construct a local detection window centered on the seed points, calculate the gradient covariance matrix within the local detection window, determine the growth threshold, and perform region growing on the seed points according to the growth threshold to obtain candidate defect areas.

[0008] In an alternative embodiment, Analyze the trend curve of the mutual information value changing with the scale, extract the position of the mutation point of the trend curve, and determine the scale layer with the largest mutual information mutation as the optimal detection scale, including: Construct a trend curve of the mutual information value changing with the scale, construct a structure tensor for the local structure of the trend curve, adaptively determine the weight coefficient based on the eigenvalue distribution of the structure tensor, and perform non - linear local weighting on the trend curve to obtain an enhanced trend curve; Calculate the normalized difference of the enhanced trend curve at adjacent scale levels, construct an adaptive threshold by combining the local gradient density distribution, and extract the mutation features of the trend curve based on the adaptive threshold; at the same time, perform wavelet decomposition on the enhanced trend curve, extract the curvature coefficients at different scales, and construct a mutation point detection operator based on the multi-scale response of the curvature coefficients; Adaptively fuse the mutation features with the mutation point detection operator, construct a mutation point evaluation function, analyze the time-frequency characteristics of the mutation point evaluation function, extract the main frequency component of the feature response, determine the candidate mutation point set of the trend curve based on the main frequency component, construct a local phase consistency map at each scale level of the candidate mutation point set, and calculate the complementary texture features of the local phase consistency map and the gradient direction field; Verify the reliability of the candidate mutation points based on the complementary texture features, and select the scale level with the maximum mutual information mutation as the optimal detection scale.

[0009] In an alternative embodiment, Perform adaptive binarization processing on the candidate defect region, extract the defect edge contour points, calculate the curvature values of the defect edge contour points, use the contour points with curvature values greater than the preset curvature threshold as key points, and perform defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect region including: Construct a multi-directional filter bank for the candidate defect region, calculate the response intensity in each direction, extract the maximum response direction as the main direction from the response intensity, and perform adaptive enhancement on the neighborhood of the main direction. Based on the enhanced response intensity, construct a binarization threshold function, and perform adaptive binarization processing on the candidate defect region to obtain a binarized image; Construct a double-boundary tracking operator, use the double-boundary tracking operator to perform boundary tracking in the binarized image, extract the candidate contour points with double-boundary responses, and screen the candidate contour points based on the continuity of the main direction. Determine the candidate contour points whose main direction continuity meets the preset continuous condition as the defect edge contour points. Taking the defect edge contour points as the center, construct an annular neighborhood that changes with the main direction, and calculate the curvature values of the contour points within the annular neighborhood; Set a dynamic curvature threshold according to the continuity of the main direction, and mark the contour points with curvature values exceeding the dynamic curvature threshold as key points; Extract the spatial position sequence of the key points, calculate the polar coordinate parameters between adjacent key points based on the main direction, construct a shape feature vector including the distance ratio and the angle difference, and use the shape feature vector to perform defect shape discrimination to obtain the defect region.

[0010] In an alternative embodiment, Extract and classify the texture features of the defect region to obtain the defect types including: Extract the edge features of the gradient histogram in the defect area at multiple scales, weight them using the gradient amplitude, construct local texture features through non-uniform sampling and adaptive binarization, and obtain the edge texture feature vector; Calculate the gray-level co-occurrence matrix of the defect area based on multiple angles and distances, dynamically determine the window size according to the regional complexity to extract contrast and correlation information, and perform weighted processing in combination with the local variance distribution to obtain the statistical texture feature vector; Track and locate the boundary of the defect area and calculate the edge curvature distribution, construct the defect skeleton structure and extract branch connectivity information, and generate the shape feature vector in combination with the regional distance transformation; Normalize the edge texture feature vector, statistical texture feature vector and shape feature vector, calculate the ratio of the inter-class distance to the intra-class variance between features as the weight coefficient, and select and combine the features according to the weight coefficient to obtain the combined feature vector; Use the shape feature vector to divide the defect area into linear defect areas and block defect areas, identify the type of the divided defect area in combination with the combined feature vector, and at the same time construct the spatial association matrix of the defect area to optimize the defect type recognition result, and perform feature cross-validation on the recognition result with a recognition confidence less than the preset confidence threshold, and output the defect type.

[0011] In an alternative embodiment, Calculate the defect centroid coordinates as the origin based on the key point distribution of the defect area, establish a polar coordinate system, calculate the angle-radius distribution of the defect boundary curve, and generate the laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part, including: Calculate the local density contribution value of the key points in the defect area at multiple Gaussian kernel scales, perform non-linear mapping on the density contribution value to obtain the key point importance score, use the importance score as the weight coefficient, and calculate the defect centroid coordinates by the iterative weighted least squares method; Establish a polar coordinate system with the defect centroid coordinates as the origin, dynamically adjust the angle sampling interval based on the local curvature value of the defect boundary contour points, calculate the curvature variance of adjacent sampling points, and use the curvature variance as the weight coefficient to perform weighted calculation on the radial distance of the boundary points to generate the angle-radius distribution of the defect boundary curve; Use the moving least squares method to calculate the normal vector change rate of the surface of the automotive part around the defect area, divide the curvature feature area according to the normal vector change rate, and construct a compensation function in each curvature feature area to deform the angle-radius distribution to generate the laser engraving compensation path point sequence.

[0012] In an alternative embodiment, Combining the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data for hierarchical compensation machining of printing defects on the surface of automotive parts includes: Calculating the cutting force distribution of the laser engraving compensation path point sequence during the machining process, obtaining the heat accumulation value of the cutting process, dividing the laser engraving compensation path point sequence into multiple machining units, and determining the cooling time interval of each machining unit based on the heat accumulation value; Determining the total number of machining layers according to the material removal characteristics, distributing the laser engraving compensation depth parameter according to the number of machining layers, calculating the material removal amount of each layer, and generating a hierarchical machining sequence; Combining the cooling time interval, calculating the feed speed of each machining unit, converting the laser engraving compensation path point sequence into a motion trajectory with a feed speed, and inserting waiting nodes between adjacent reciprocating paths; Synchronously combining the hierarchical machining sequence and the motion trajectory with a feed speed to generate rough machining trajectory data containing timing information, adding tool lift and tool down positions to the rough machining trajectory data, planning the inter-layer transition path, and establishing a complete tool motion trajectory; Converting the complete tool motion trajectory into three-dimensional compensation trajectory data executable by the machine tool for hierarchical compensation machining of printing defects on the surface of automotive parts.

[0013] In the second aspect of the embodiments of the present invention, a printing defect detection system based on machine vision is provided, including: A first unit for collecting an original image of the printing area on the surface of the automotive part to be detected through an industrial camera and performing image enhancement processing to obtain an enhanced detection image; A second unit for constructing a multi-scale gradient pyramid for the enhanced detection image, extracting the gradient direction histogram at each scale, calculating the mutual information value of the gradient direction histograms between adjacent scales, determining the optimal detection scale, and performing defect area segmentation at the optimal detection scale to obtain candidate defect areas; A third unit for performing adaptive binarization processing on the candidate defect areas, extracting defect edge contour points, calculating the curvature values of the defect edge contour points, using the contour points with curvature values greater than a preset curvature threshold as key points, performing defect shape discrimination based on the spatial distribution characteristics of the key points to obtain defect areas, and extracting and classifying texture features of the defect areas to obtain defect types; The fourth unit is used to calculate the centroid coordinates of the defect as the origin based on the key point distribution in the defect area, establish a polar coordinate system, calculate the angle - radius distribution of the defect boundary curve, generate a laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part, set the laser engraving compensation depth parameter according to the defect type, combine the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three - dimensional compensation trajectory data, and perform layered compensation processing on the printing defects on the surface of the automotive part.

[0014] In the third aspect of the embodiments of the present invention, an electronic device is provided, including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0015] In the fourth aspect of the embodiments of the present invention, a computer - readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0016] In this embodiment, the method for detecting printing defects based on machine vision can adaptively process defect features at different scales, improve the accuracy and robustness of detection, and reduce the detection error rate by constructing a multi - scale gradient pyramid and calculating the mutual information value to determine the optimal detection scale. Key points are extracted using curvature features and the defect shape is discriminated, and the defect type is classified and recognized by combining texture features, realizing precise positioning and classification of printing defects on the surface of automotive parts, and different types of printing defects can be distinguished, providing accurate defect information for subsequent compensation. A polar coordinate system is established based on defect features and a laser engraving compensation path is generated, and the compensation depth parameter is adaptively set according to the defect type to form three - dimensional compensation trajectory data, which can perform precise layered compensation processing for the surface curvature characteristics of automotive parts, effectively improving the repair quality of printing defects and reducing the product defect rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic flow chart of the method for detecting printing defects based on machine vision according to the embodiments of the present invention; Figure 2 It is a schematic diagram of the performance comparison of different technical means in different scenarios; Figure 3 It is a schematic diagram of the defect centroid calculation iteration and the key point importance distribution; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0020] Figure 1 The following is a flowchart of the method for detecting defects on printed parts based on machine vision according to an embodiment of the present invention. As Figure 1 shown, the method includes: Collect the original image of the printed area on the surface of the automotive part to be detected through an industrial camera and perform image enhancement processing to obtain an enhanced detection image; Construct a multi-scale gradient pyramid for the enhanced detection image, extract the histogram of gradient directions at each scale, calculate the mutual information value of the histograms of gradient directions between adjacent scales, determine the optimal detection scale, and perform defect area segmentation at the optimal detection scale to obtain candidate defect areas; Perform adaptive binarization processing on the candidate defect areas, extract the defect edge contour points, calculate the curvature values of the defect edge contour points, use the contour points with curvature values greater than the preset curvature threshold as key points, and perform defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect area, extract and classify the texture features of the defect area to obtain the defect type; Calculate the centroid coordinates of the defect as the origin based on the distribution of the key points in the defect area, establish a polar coordinate system, calculate the angle - radius distribution of the defect boundary curve, generate a sequence of laser engraving compensation path points according to the curvature characteristics of the automotive part surface, and set the laser engraving compensation depth parameter according to the defect type. Combine the sequence of laser engraving compensation path points and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data, and perform layered compensation processing on the printed defects on the surface of the automotive part.

[0021] Exemplarily, in the process of defect detection and laser engraving compensation for the printed area on the surface of automotive parts, first, an industrial camera is used to collect the original image of the printed area on the surface of the automotive part to be detected. The industrial camera should have high resolution and a wide dynamic range to ensure stable acquisition of surface image information under different lighting conditions. During the acquisition process, the industrial camera is arranged perpendicular to the detected surface in a fixed installation manner, the workpiece is driven into the shooting area through an automated transmission line, and a flexible light supplement device is combined to achieve uniform illumination, avoiding the influence of reflected light on the printed surface on the image quality.

[0022] After the image acquisition is completed, image enhancement processing is performed on the collected original image to improve the accuracy and robustness of subsequent image analysis. Image enhancement mainly includes contrast improvement, noise suppression, and multi-directional filtering. Specifically, a multi-scale enhancement algorithm based on the Retinex principle is used to perform local brightness correction on the image, making the gray-scale contrast between the printed texture area and the background in the image more obvious. At the same time, the bilateral filtering method is combined for noise smoothing, removing high-frequency noise while retaining edge features. To further highlight the features of the printed defects in the texture direction, the image is convolved with a multi-directional Gabor filter, and the filtering response maps in four directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees are selected for fusion to obtain the enhanced detection image.

[0023] In an optional implementation manner, a multi-scale gradient pyramid is constructed for the enhanced detection image, the histogram of gradient directions at each scale is extracted, the mutual information value between the histograms of gradient directions at adjacent scales is calculated, the optimal detection scale is determined, and the defect area is segmented at the optimal detection scale to obtain candidate defect areas including: The enhanced detection image is subjected to multi-scale decomposition, the image is convolved with a Gaussian kernel function of variable scale, the Gaussian kernel parameters are dynamically adjusted based on the inter-layer gradient consistency of the convolution result, and a multi-scale gradient pyramid is constructed; At each scale layer of the multi-scale gradient pyramid, the horizontal gradient and vertical gradient of the pixel points are calculated to determine the gradient amplitude and direction, the gradient direction space is divided into multiple intervals, and the number of gradient points in each interval is statistically counted to construct a histogram of gradient directions; A joint probability distribution matrix is constructed, the joint probability distribution matrix is normalized, the mutual information value between the histograms of gradient directions at adjacent scale layers is calculated based on the normalized joint probability, the trend curve of the mutual information value changing with the scale is analyzed, the position of the mutation point of the trend curve is extracted, and the scale layer with the largest mutual information mutation is determined as the optimal detection scale; On the image layer corresponding to the optimal detection scale, calculate the local gradient magnitude of the image pixel points. Take the pixel points with gradient magnitude greater than the preset gradient threshold as seed points. Construct a local detection window centered on the seed points, calculate the gradient covariance matrix within the local detection window, determine the growth threshold, and perform region growing on the seed points according to the growth threshold to obtain candidate defect regions.

[0024] This embodiment discloses a defect detection method based on a multi-scale gradient pyramid. The method first constructs a multi-scale gradient pyramid for the enhanced detection image, then extracts the gradient direction histogram at each scale, calculates the mutual information value of the gradient direction histograms between adjacent scales, determines the optimal detection scale, and performs defect region segmentation at the optimal detection scale to obtain candidate defect regions.

[0025] In the stage of constructing the multi-scale gradient pyramid, perform a convolution operation on the enhanced detection image using a Gaussian kernel function with a variable scale. Specifically, the initial Gaussian kernel parameter is set to σ = 1.0, the scale factor k = 1.2, and 5 scale layers are constructed. After each convolution, calculate the included angle of the gradient vectors between adjacent layers. If the average included angle is greater than 30°, then reduce the scale factor k to 0.9 times of the original; if the average included angle is less than 10°, then increase the scale factor k to 1.1 times of the original. Through this dynamic adjustment mechanism, ensure that the multi-scale decomposition can adapt to the complexity characteristics of the detection image. For example, for an aluminum plate surface image of 512×512 pixels, after dynamic adjustment, the final scale sequence may be σ = {1.0, 1.16, 1.35, 1.57, 1.82}, and this adaptive adjustment can better capture defect features at different scales.

[0026] On each scale layer, calculate the horizontal gradient and vertical gradient of the pixel points. The horizontal gradient is obtained by performing a differential operation on the image in the horizontal direction, and the vertical gradient is obtained by performing a differential operation on the image in the vertical direction. For the pixel point at position (x, y), its gradient magnitude is calculated as the square root of the sum of the squares of the horizontal gradient and the vertical gradient, and the gradient direction is calculated as the arctangent of the vertical gradient to the horizontal gradient. Divide the gradient direction space into 12 intervals, each interval with a width of 30°, count the number of gradient points in each interval, and construct a gradient direction histogram. To improve the statistical stability, only consider the pixel points with gradient magnitude greater than 20% of the average gradient magnitude of the image. For example, the gradient direction histogram of the first scale layer may be [320, 560, 980, 1240, 890, 450, 230, 180, 290, 520, 730, 610], indicating the gradient distribution in different direction intervals.

[0027] For the histograms of gradient directions in adjacent scale levels, a joint probability distribution matrix P is constructed. The dimension of the matrix is 12×12, and each element P(i, j) represents the probability of simultaneous occurrence in direction interval i of the first scale level and direction interval j of the second scale level. Through the normalization of the histograms of gradient directions, the marginal probability distributions P1 and P2 of each scale level are obtained. Based on these probability distributions, the mutual information value is calculated. The larger the mutual information value, the more similar the information contained in the two scale levels. By analyzing the trend curve of the mutual information value changing with the scale, the position of the mutation point is extracted. For example, for 5 scale levels, 4 mutual information values [0.85, 0.78, 0.42, 0.39] may be obtained. The decrease in the mutual information value between the second and the third scale levels is the most obvious, with a decrease amplitude of 0.36, indicating that the third scale level (σ = 1.35) captures significantly different image structures. Therefore, the third scale level is determined as the optimal detection scale.

[0028] After determining the optimal detection scale, defect region segmentation is performed on this scale level. First, the local gradient magnitude of the image pixel points is calculated. The local gradient magnitude is defined as the average gradient magnitude within a 3×3 neighborhood. The preset gradient threshold is set to 2 times the average gradient magnitude of the image. The pixel points with gradient magnitude greater than this threshold are used as seed points. For each seed point, an 11×11 local detection window is constructed, and the covariance matrix of the gradient vectors of the pixel points within the window is calculated. The eigenvalue decomposition of the covariance matrix yields two eigenvalues λ1 and λ2 (λ1≥λ2), and the eigenvalue ratio r = λ1 / λ2 is used to determine the growth threshold. If r > 4, it indicates that there is a strong directional structure in the local area, and the growth threshold is set to 0.6 times the gradient magnitude of the seed point; if r ≤ 4, the growth threshold is set to 0.8 times the gradient magnitude of the seed point.

[0029] The region growing process starts from the seed points. Check its 8-neighborhood pixels. If the gradient magnitude of the neighborhood pixel is greater than the growth threshold and the difference between the gradient direction and the gradient direction of the seed point is less than 45°, then this pixel is added to the current region and serves as a new expansion point. Repeat this process until no new pixels can be added. To reduce the influence of noise, morphological processing is performed on the growth result, including opening operation to remove small regions and closing operation to fill holes, and the size of the structuring element is set to 3×3. The finally obtained connected region is the candidate defect region. For example, in an image of a metal surface with scratch defects, a scratch region about 60 pixels long and about 3 pixels wide can be successfully segmented by the above method, while traditional single-scale-based methods may miss detections or make false detections.

[0030] In this embodiment, by constructing a multi-scale gradient pyramid and dynamically adjusting the Gaussian kernel parameters, the effective expression of image features at different scales is realized, and the detection ability for tiny and blurred defects is enhanced. By calculating the histogram of gradient directions and their mutual information values at each scale, the optimal detection scale containing the richest structural information can be adaptively selected, thereby improving the detection accuracy and robustness. On this basis, combined with the local gradient magnitude screening and region growing algorithm, potential defect regions can be accurately located, the boundary consistency and integrity of candidate regions are enhanced, which helps the efficient execution of subsequent classification, recognition and compensation processing.

[0031] In an alternative embodiment, analyzing the trend curve of the mutual information value changing with the scale, extracting the position of the mutation point of the trend curve, and determining the scale layer with the largest mutual information mutation as the optimal detection scale includes: Construct a trend curve of the mutual information value changing with the scale, construct a structure tensor for the local structure of the trend curve, adaptively determine the weight coefficient based on the eigenvalue distribution of the structure tensor, and perform non-linear local weighting on the trend curve to obtain an enhanced trend curve; Calculate the normalized difference of the enhanced trend curve at adjacent scale layers, construct an adaptive threshold in combination with the local gradient density distribution, and extract the mutation features of the trend curve based on the adaptive threshold; at the same time, perform wavelet decomposition on the enhanced trend curve, extract the curvature coefficients at different scales, and construct a mutation point detection operator based on the multi-scale response of the curvature coefficients; Adaptively fuse the mutation features and the mutation point detection operator, construct a mutation point evaluation function, analyze the time-frequency characteristics of the mutation point evaluation function, extract the main frequency component of the feature response, determine the candidate mutation point set of the trend curve based on the main frequency component, construct a local phase consistency map at each scale layer of the candidate mutation point set, and calculate the complementary texture features of the local phase consistency map and the gradient direction field; Verify the reliability of the candidate mutation points based on the complementary texture features, and select the scale layer with the largest mutual information mutation as the optimal detection scale.

[0032] Exemplarily, first, it is necessary to analyze the trend curve of the mutual information value with the change of scale, and enhance the local structural features of the trend curve by constructing a structure tensor. Specifically, for a given sequence of multi-scale gradient direction histograms, calculate the mutual information value between adjacent scale levels. Taking the broken line defect on the surface of a printed matter as an example, after collecting the original image of 1024×1024 pixels, construct a 5-layer pyramid structure with a sampling interval of 2 for each layer of scale, and obtain a multi-scale image sequence of 512×512, 256×256, 128×128, and 64×64 pixels. Calculate the gradient direction histogram for each scale image, divide the gradient direction into 36 uniform intervals, and obtain a 36-dimensional feature vector. Calculate the mutual information value of the feature vectors between adjacent scales to form a trend curve of 4 data points.

[0033] To extract the local structural features of the trend curve, construct a structure tensor within the neighborhood of each data point. The structure tensor is a 2×2 real symmetric matrix, and its elements are calculated from the first-order partial derivatives of the trend curve at that point. Specifically, take a 5×5 neighborhood window around the data point, calculate the gradients in the horizontal and vertical directions, and sum the outer products of the gradients as the elements of the structure tensor. By performing eigenvalue decomposition on the structure tensor, the larger eigenvalue represents the main direction of the local structure at that point, and the smaller eigenvalue represents the secondary direction.

[0034] When determining the weight coefficient based on the eigenvalue distribution of the structure tensor, an adaptive weighting strategy is adopted. The calculation of the weight coefficient considers two factors: one is the ratio of eigenvalues, which reflects the degree of anisotropy of the local structure; the other is the sum of eigenvalues, which reflects the significance of the local structure. The weight coefficient increases with the increase of the ratio of eigenvalues and also increases with the increase of the sum of eigenvalues, and a sigmoid function is specifically used for mapping. For the example of the broken line defect, at the inflection point of the trend curve, due to the significant local structure, a larger weight coefficient is obtained, while a smaller weight coefficient is obtained in the flat region.

[0035] Perform non-linear local weighting on the trend curve using the determined weight coefficient. Within the neighborhood of each data point, multiply the original data value by the weight coefficient to enhance the local structure. This process can highlight the mutation features in the trend curve and suppress noise interference. In the example, after the weighting process, the trend curve shows more obvious jump features at the critical points of scale change.

[0036] Calculate the normalized difference of the enhanced trend curve between adjacent scale levels. First, normalize the curve to map the numerical range to the interval [0, 1]. Then calculate the differences between adjacent data points to obtain a difference sequence. At the same time, analyze the local gradient distribution of the curve and calculate the gradient density within the neighborhood of each data point. The greater the gradient density, the more drastic the change in that area. Construct an adaptive threshold based on the gradient density, where the threshold decreases as the gradient density increases, so that a smaller threshold is used for mutation feature extraction in areas with drastic changes.

[0037] Perform wavelet decomposition on the enhanced trend curve, using the Haar wavelet as the basis function and performing 3-layer decomposition. Extract the detail coefficients in each layer of decomposition, and these coefficients reflect the local change characteristics of the curve at different scales. By analyzing the distribution of the detail coefficients, calculate the curvature coefficient at each data point. The calculation of the curvature coefficient considers the degree of curvature of the curve segment formed by this point and its adjacent points before and after. Based on the curvature coefficients at different scales, construct a mutation point detection operator. This operator combines the responses of multiple scales and can effectively identify mutation points in the curve.

[0038] Adaptively fuse the obtained mutation features with the mutation point detection operator. The fusion process adopts a weighted average strategy, and the weight coefficients are dynamically adjusted according to the reliability of each feature. The reliability evaluation is based on the local consistency of the features, and features with better consistency obtain greater weights. After fusion, construct a mutation point evaluation function, which has a large response value at the potential mutation point positions.

[0039] Perform time-frequency analysis on the mutation point evaluation function, and use the short-time Fourier transform to extract local spectral features. Search for the frequency components with concentrated energy in the frequency domain, and these components correspond to the main change patterns in the evaluation function. Use the frequency component with the maximum energy as the main frequency component to guide the positioning of mutation points. In the example, the main frequency component corresponds to the most significant scale change position in the trend curve.

[0040] Determine the set of candidate mutation points at the local maximum points of the evaluation function. For each mutation point in the set, construct a local phase congruency map at its corresponding scale level. The phase congruency map characterizes the reliability of edge features by analyzing the degree of consistency of the local gradient directions in the image. At the same time, calculate the gradient direction field at this scale level and extract complementary texture features. These features can reflect the distinguishability between the defect area and the background.

[0041] Reliability verification of candidate mutation points is performed based on complementary texture features. The verification process considers the following aspects: first, the gradient consistency in the neighborhood of the mutation point; second, the significance of the edge response; and third, the degree of association with adjacent scales. By comprehensively evaluating these factors, the most reliable mutation points are screened out. For the broken line defect in the example, the most significant feature response is shown at the scale layer of 256×256 pixels, so this scale is selected as the optimal detection scale.

[0042] In the prior art, image feature extraction and mutation point analysis are often carried out at a fixed scale or an empirically set scale, which is difficult to adapt to the changes in different image structures and information complexities, resulting in the sensitivity of the detection results to scale selection and insufficient accuracy and stability. To solve the above problems, in this application, a trend curve of the mutual information value changing with the scale is constructed and its mutation behavior is analyzed. A structure tensor is introduced to model the local change pattern of the trend curve, and weighted coefficients are adaptively assigned based on the eigenvalues, so as to enhance the mutation features while maintaining the original change trend. In addition, wavelet decomposition is used to extract multi-scale curvature characteristics, a detection operator responsive to local mutations is constructed, and the mutation features are fused to form an evaluation function. The main frequency response position is extracted through frequency domain analysis to more accurately locate potential mutation regions. Furthermore, a local phase consistency map is introduced on the candidate scale layer, and complementary texture features are constructed in combination with the gradient direction field to further screen and verify the mutation points. This scheme has obvious improvements compared with the existing methods in terms of adaptive scale optimization, mutation detection sensitivity, and anti-interference ability, significantly enhancing the accuracy and robustness of extracting key structure information in complex image backgrounds.

[0043] Figure 2Schematic diagram for performance comparison of different technical means in different scenarios. Among them, Scenario Alpha: Detection of incomplete fine printing of dashboard warning icons; Scenario Beta: Detection of tiny broken lines of engine compartment label characters; Scenario Gamma: Detection of printing distortion and consistency of functional symbols on curved interior trim parts; Scenario Delta: Detection of multi-scale defects of complex pattern decorative strips. In all four test scenarios, the height of the bar chart corresponding to "the method of the present invention" (i.e., the performance value, such as 92.3% in Scenario Beta) is generally better than or at least equal to that of the "baseline method" (such as 88.0% in Scenario Beta) and "comparative method A" (such as 78.5% in Scenario Beta). This fully demonstrates the significant advantage of the method of the present invention in improving the defect detection performance. This advantage is mainly attributed to the unique technical path adopted by the present invention: by constructing a trend curve of the mutual information value varying with the scale, and combining a series of precise steps such as structure tensor enhancement, wavelet decomposition analysis, a mutation point evaluation function of multi-feature fusion, and reliability verification of candidate mutation points based on local phase consistency and complementary texture features, the adaptive and accurate determination of the optimal detection scale for image analysis is achieved. This enables the present invention to more accurately and robustly extract key structure information under complex image backgrounds and variable defect morphologies, thereby improving the overall detection efficiency and reliability.

[0044] In an optional implementation manner, perform adaptive binarization processing on the candidate defect region, extract the defect edge contour points, calculate the curvature values of the defect edge contour points, use the contour points with curvature values greater than the preset curvature threshold as key points, and perform defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect region including: Construct a multi-directional filter bank for the candidate defect region, calculate the response intensity in each direction, extract the maximum response direction from the response intensities as the main direction, and perform adaptive enhancement on the neighborhood of the main direction. Based on the enhanced response intensity, construct a binarization threshold function, and perform adaptive binarization processing on the candidate defect region to obtain a binarized image; Construct a double-boundary tracking operator, use the double-boundary tracking operator to perform boundary tracking in the binarized image, extract the candidate contour points with double-boundary responses, and screen the candidate contour points based on the continuity of the main direction. Determine the candidate contour points with the continuity of the main direction meeting the preset continuous condition as the defect edge contour points. With the defect edge contour points as the center, construct an annular neighborhood varying with the main direction, and calculate the curvature values of the contour points within the annular neighborhood; Set a dynamic curvature threshold according to the continuity of the main direction, and mark the contour points with curvature values exceeding the dynamic curvature threshold as key points; Extract the spatial position sequence of the key points, calculate the polar coordinate parameters between adjacent key points based on the main direction, construct a shape feature vector containing the distance ratio and the angle difference, and use the shape feature vector to discriminate the defect shape to obtain the defect area.

[0045] Exemplarily, for the input candidate defect area image, construct a multi-direction filter bank to enhance the defect edge features. Specifically, design a set of direction filters covering the range from 0° to 180°, with an interval of 15°, for a total of 12 directions. Each direction filter uses a Gaussian first derivative kernel function with a size of 9×9 pixels and a standard deviation of 1.5. Apply these direction filters to the candidate defect area and calculate the response intensity of each pixel point in each direction. For example, for a candidate defect area with a size of 256×256 pixels, a corresponding response map can be obtained in each direction. By comparing the response values of each pixel point in the response maps of each direction, determine the maximum response direction as the main direction of the pixel point. For the detected main direction, further perform adaptive enhancement processing on its neighborhood. Taking the main direction as the center, select the direction response maps within 30° on both the left and right sides of it for weighted average, and the weight is proportional to the cosine value of the direction difference. This enhancement processing ensures the continuity of the edge direction and enables weak edges to be effectively extracted.

[0046] Based on the enhanced response intensity, construct a binarization threshold function to achieve adaptive binarization processing. The threshold function is designed as a function of the response intensity, specifically, the mean value of the response intensity within the 7×7 neighborhood around the current pixel point plus 0.5 times the standard deviation. Pixel points with a response intensity higher than the threshold are marked as 1, otherwise marked as 0, thereby obtaining a binarized image. For example, for a defect area with a weak edge response, the average response intensity of the local area may be 0.35 and the standard deviation is 0.12, then the binarization threshold of this area is 0.41.

[0047] After obtaining the binarized image, construct a double-boundary tracking operator to extract the defect edge contour points. The double-boundary tracking operator is defined as a two-way detector in the direction perpendicular to the main direction, with a detection distance of 5 pixels. Starting from the edge starting point in the binarized image, detect the change in pixel values on both sides in the direction perpendicular to the main direction. When jumps from 0 to 1 or from 1 to 0 are detected on both sides simultaneously, mark the current position as a candidate contour point. To improve the accuracy of the contour points, screen the candidate contour points based on the continuity of the main direction. Specifically, calculate the change amount of the main direction of adjacent pixel points. When the change amount is less than 20°, it is considered to meet the continuity condition, and this candidate point is determined as the defect edge contour point. For a typical crack defect, about 200 - 300 contour points can be extracted through this step.

[0048] To calculate the curvature value of the contour points, a circular neighborhood that varies with the principal direction is constructed centered on each defect edge contour point. The inner radius of the circular neighborhood is 3 pixels, the outer radius is 7 pixels, and the angular range is 60° on each side of the principal direction. Within this circular neighborhood, the curvature value of the contour point is calculated by analyzing the pixel intensity distribution. The specific calculation method is the second-order rate of change of the pixel intensity along the principal direction, which numerically represents the rate of change of the pixel intensity difference on both sides of the principal direction. For example, for a contour point located at the crack corner, its curvature value may reach 0.85, while the curvature value of a contour point located on a straight line segment is usually less than 0.3.

[0049] A dynamic curvature threshold is set according to the continuity of the principal direction to adapt to the characteristics of different defect shapes. When the principal direction changes greatly, the curvature threshold is lowered; when the principal direction changes little, the curvature threshold is raised. Specifically, when the change amount of the principal direction between adjacent points exceeds 15°, the curvature threshold is set to 0.5; when the change amount is between 5° and 15°, the curvature threshold is 0.6; when the change amount is less than 5°, the curvature threshold is 0.7. The contour points with curvature values exceeding the dynamic curvature threshold are marked as key points. In this way, the key feature points of the defect shape, such as corners and sharp points, can be accurately captured.

[0050] Extract the spatial position sequence of the marked key points and arrange them in the order along the contour. Based on the principal direction, the polar coordinate parameters between adjacent key points are calculated, including distance and angle. The distance is represented as the Euclidean distance between two points, and the angle is represented as the angle between the connecting line and the horizontal direction. Further, calculate the distance ratio and angle difference to form a shape feature vector. The distance ratio is defined as the ratio of the lengths of two line segments formed by three adjacent key points, and the angle difference is defined as the included angle between these two line segments. For example, for a typical L-shaped crack, the number of key points is about 4 - 6, the distance ratio between adjacent key points is close to 1.0, and the angle difference is close to 90° at a certain position. For different types of defects, a shape feature library is established, which contains the feature vectors of typical shapes such as rectangular defects, triangular defects, and crack defects. By calculating the similarity between the shape feature vector of the defect to be detected and each type in the feature library, the defect shape is discriminated, and the type information of the defect area is obtained. When the similarity exceeds 0.85, it is determined as the corresponding type of defect; when the similarity is lower than 0.65, it is marked as an unknown type; when it is between the two, the type with the highest similarity is selected, and low confidence is marked at the same time.

[0051] In the prior art, global fixed thresholds or single-direction filtering are often used for defect area recognition and edge extraction, which are easily affected by background noise or insensitive to complex deformation boundaries, resulting in low defect recognition accuracy and inaccurate boundary positioning. In this application, a multi-direction filter bank is constructed to extract the main direction information of the candidate area, and adaptive response enhancement is performed within the main direction neighborhood, realizing sensitive perception of directional defects. The adaptive threshold function constructed based on the enhanced response can adapt to different backgrounds and texture intensities, significantly improving the robustness of the binarization effect. At the same time, by introducing a double-boundary tracking operator to extract the contour points with continuous main directions and constructing a dynamic curvature calculation neighborhood in combination with the main direction, the accurate capture of the real boundary structure is achieved. On this basis, key structure points are identified according to the curvature change of the contour points, and by constructing a polar coordinate shape feature containing the distance ratio and angle difference, the accurate discrimination of the defect shape is realized. This solution breaks through the limitation of the insufficient response of traditional methods to the complexity of edges through the directional modeling of the boundary structure and the recognition of curvature key points, effectively improving the defect recognition accuracy and shape classification ability under weak boundaries and complex backgrounds, and enhancing the adaptability and practicability of the system.

[0052] In an optional implementation manner, texture features of the defect area are extracted and classified, and the obtained defect types include: Extract the edge features of the direction gradient histogram of the defect area at multiple scales, weight them using the gradient amplitude, construct local texture features through non-uniform sampling and adaptive binarization, and obtain the edge texture feature vector; Calculate the gray-level co-occurrence matrix of the defect area based on multiple angles and multiple distances, dynamically determine the window size according to the regional complexity to extract the contrast and correlation information, and perform weighted processing in combination with the local variance distribution to obtain the statistical texture feature vector; Track and locate the boundary of the defect area and calculate the edge curvature distribution, construct the defect skeleton structure and extract the branch connectivity information, and generate the shape feature vector in combination with the regional distance transformation; Normalize the edge texture feature vector, statistical texture feature vector, and shape feature vector, calculate the ratio of the between-class distance to the within-class variance between the features as the weight coefficient, and select and combine the features according to the weight coefficient to obtain the combined feature vector; Use the shape feature vector to divide the defect area into a linear defect area and a block defect area, identify the type of the divided defect area in combination with the combined feature vector, and at the same time construct a spatial association matrix of the defect area to optimize the defect type recognition result, and perform feature cross-validation on the recognition result with a recognition confidence less than the preset confidence threshold, and output the defect type.

[0053] Exemplarily, after obtaining the image of the workpiece to be detected, the edge features of the detected defect area are extracted by multi-scale histogram of oriented gradients. The system selects windows of 3 different scales (4×4 pixels, 8×8 pixels, 16×16 pixels) to scan the defect area, and calculates the pixel gradient direction and amplitude within each window. The direction angle is evenly divided into 9 intervals (0°-20°, 20°-40°, ..., 160°-180°), and each pixel is classified into the corresponding interval according to its gradient direction, and an orientation histogram is constructed with the gradient amplitude as the weight. To enhance the local texture features, the system adopts a non-uniform sampling strategy, and the sampling density in the defect edge area is 2 times that of the central area. During the adaptive binarization process, the threshold is set to 0.85 times the average gray value of the local window to generate a binary texture map, thus forming a 72-dimensional edge texture feature vector.

[0054] For the extraction of statistical texture features, the system calculates the gray-level co-occurrence matrix at multiple angles and distances. In the specific implementation, the combination of 4 angles (0°, 45°, 90°, 135°) and 3 distance values (1 pixel, 3 pixel, 5 pixel) is selected to calculate the gray-level co-occurrence matrix. According to the complexity of the defect area, the window size is dynamically determined. The complexity is calculated by the pixel variance within the area. When the variance is greater than 100, a 32×32 window is used, otherwise a 16×16 window is used. The contrast feature is extracted from the co-occurrence matrix to calculate the degree of difference between gray value pairs, and the correlation feature is calculated to measure the linear correlation of adjacent pixel gray values. At the same time, the local variance distribution is introduced for weighting. A larger weight (weight coefficient is 1.5) is given to the area with a high variance value, and the weight coefficient of the low variance area is 0.8, forming a 48-dimensional statistical texture feature vector.

[0055] For the extraction of shape features, the system uses a boundary tracking algorithm to locate the defect boundary. The system starts from any point on the boundary and performs an 8-neighborhood search in the clockwise direction to find the next boundary point until it returns to the starting point to complete the closed boundary. A curvature value is calculated every 5 pixels along the boundary, and the curvature is represented by the angle change rate of adjacent boundary points. The defect skeleton structure is generated by a morphological thinning algorithm, which retains the topological structure of the object. Endpoints and branch points are identified on the skeleton, and the number and length of branches are recorded. The branch connectivity information includes the angle distribution and density between branches. The system calculates the regional distance transform to obtain the shortest distance from each point inside the defect to the boundary, and extracts features such as the radius of the largest inscribed circle and the aspect ratio based on this, finally forming a 36-dimensional shape feature vector.

[0056] Feature normalization is processed using the min-max method, mapping each feature value to the interval [0, 1]. Feature weight assignment is based on the ratio of between-class distance to within-class variance. Calculate the mean and variance of each feature in samples of different defect categories. The between-class distance is the absolute value of the difference in feature means between different defect categories, and the within-class variance is the degree of dispersion of feature values of samples in the same category. Features with a ratio greater than 1.5 are assigned a high weight (1.2), features with a ratio between 0.8 and 1.5 have a weight of 1.0, and features with a ratio less than 0.8 have a weight of 0.6. Through weight screening, the top 80% of important features are retained to form the combined feature vector after dimensionality reduction.

[0057] Defect type classification first uses the shape feature vector to preliminarily divide the defects. The system calculates the aspect ratio (the ratio of the major axis to the minor axis) of the defect. When the aspect ratio is greater than 3.5 and the maximum width is less than 1.8 times the average width of the defect area, it is classified as a linear defect; otherwise, it is classified as a block defect. For linear defects, focus on the directionality and continuity features; for block defects, focus on the texture distribution and boundary regularity features. The system uses a support vector machine classifier for defect type recognition, with a radial basis kernel function, the kernel parameter γ set to 0.05, and the penalty coefficient C to 10. To optimize the recognition result, a spatial association matrix of the defect area is constructed to record the distance and relative position relationship between defects. When adjacent defects (distance less than 50 pixels) are classified into different types and the probability difference is less than 0.15, the defect with a lower probability is adjusted to the same type as the defect with a higher probability. For recognition results with a confidence level lower than 0.75, the system performs feature cross-validation, voting with classifiers constructed using different feature subsets, and finally determines the output of the defect type.

[0058] In this embodiment, by fusing multi-scale, multi-angle texture, statistical, and shape features, the complex structure and detailed changes of the defect area can be comprehensively characterized, improving the accuracy and robustness of defect type recognition. Compared with the existing technology that relies on a single feature or a fixed threshold judgment method, this solution introduces the histogram of oriented gradients, gray-level co-occurrence matrix, and boundary skeleton information, dynamically constructs a feature vector and performs normalized fusion, significantly enhancing the separability of different types of defects. At the same time, the recognition result is optimized through the spatial association matrix, and the low-confidence output is processed by combining the feature cross-validation mechanism, effectively reducing the misjudgment rate. Overall, accurate recognition and classification of defect types are achieved, improving the stability and adaptive ability of the system, and being applicable to defect detection tasks under various complex backgrounds.

[0059] Table 1: Performance comparison table of defect type recognition methods:

[0060] As shown in Table 1, in terms of the overall recognition accuracy, the method of the present invention (94.6%) is significantly higher than the existing method A (78.5%) and method B (85.2%), indicating that it has stronger comprehensive recognition ability. For defects with different macro shapes, the method of the present invention shows higher accuracy, which benefits from its preliminary classification strategy and targeted feature utilization. For tiny defects that are difficult to detect (such as 0.1mm broken wires), the missed detection rate of the method of the present invention (3.2%) is much lower than that of other methods, reflecting its ability to capture detailed features.

[0061] In an alternative embodiment, based on the key point distribution in the defect area, the defect center-of-gravity coordinates are calculated as the origin, a polar coordinate system is established, the angle-radius distribution of the defect boundary curve is calculated, and generating a laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part includes: Calculating the local density contribution values of the key points in the defect area at multiple Gaussian kernel scales, performing a non-linear mapping on the density contribution values to obtain key point importance scores, using the importance scores as weight coefficients, and calculating the defect center-of-gravity coordinates by iterative weighted least squares method; Taking the defect center-of-gravity coordinates as the origin to establish a polar coordinate system, dynamically adjusting the angle sampling interval based on the local curvature values of the defect boundary contour points, calculating the curvature variance of adjacent sampling points, and using the curvature variance as a weight coefficient to perform weighted calculation on the radial distance of the boundary points to generate the angle-radius distribution of the defect boundary curve; Using the moving least squares method to calculate the normal vector change rate of the surface of the automotive part around the defect area, dividing the curvature feature areas according to the normal vector change rate, and constructing a compensation function in each curvature feature area to deform the angle-radius distribution to generate a laser engraving compensation path point sequence.

[0062] Exemplarily, when calculating the defect center-of-gravity coordinates, first preprocess the defect area, including image enhancement and segmentation, to obtain a clear defect contour. After image segmentation, extract the set of key points P={p1, p2,..., p n} in the defect area, where each key point p i is represented as two-dimensional coordinates (x i , y i ). To calculate the local density contribution value of each key point, set multiple Gaussian kernel scales σ={5, 10, 15, 20} pixels. For each scale σ j and each key point p i , calculate its density contribution to other points in the defect area. For example, the density contribution value D i of point p j to other points at scale σ ijIt can be obtained by calculating the Gaussian kernel function value. In practical applications, when the distance between two points exceeds 30 pixels, the density contribution value can be approximated to zero to improve the calculation efficiency.

[0063] Perform a non-linear mapping on the calculated density contribution value to obtain the key point importance score. . The non-linear mapping is implemented using a variant of the sigmoid function, so that the importance scores are distributed between 0 and 1. In practical applications, for a defect area of approximately 1200 square pixels, the number of key points is usually between 100 and 150, and approximately 15% of the key points have an importance score higher than 0.8, indicating that these points have a significant impact on determining the defect centroid. Use these importance scores as weight coefficients and calculate the defect centroid coordinates by the iteratively reweighted least squares method. During the iteration process, the weight coefficients will be dynamically adjusted according to the current estimated centroid position. Usually, after 5 to 7 iterations, when the change in the centroid position is less than 0.1 pixel, the calculation is considered to have converged.

[0064] Establish a polar coordinate system with the calculated defect centroid coordinates (x0, y0) as the origin. In practical applications, an irregular depression defect found on the surface of a car front fender, after calculation, its centroid coordinates are (354.28, 216.93) pixels. Based on this centroid point, extract the defect boundary contour point set B = {b1, b2,..., b m}, and convert it to polar coordinate representation (r i , θ i ), where r i represents the distance from the boundary point to the centroid, represents the corresponding polar angle.

[0065] To accurately describe the defect boundary shape, it is necessary to dynamically adjust the angle sampling interval. The local curvature value κ i of the boundary point is calculated by the reciprocal of the radius of the circle formed by three adjacent points. In the area with a large curvature (κ i > 0.05), the angle sampling interval is set to 2 degrees; in the area with a moderate curvature (0.02 ≤ κ i ≤ 0.05), it is set to 5 degrees; in the area with a small curvature (κ i < 0.02), it is set to 8 degrees. This dynamic adjustment strategy ensures a higher sampling density at complex boundary shapes and a lower sampling density in smooth areas, improving the calculation efficiency and accuracy.

[0066] Calculate the curvature variance Vκ of adjacent sampling points as an index reflecting the local complexity of the boundary. For the above-mentioned dent defect on the automotive fender, the boundary perimeter is approximately 320 pixels, and a total of 62 boundary points are sampled. Among them, the maximum value of the curvature variance is 0.0038, and the minimum value is 0.0002. Use the curvature variance as the weight coefficient to perform weighted calculation on the radial distance of the boundary points to generate a more accurate angular - radius distribution of the defect boundary curve.

[0067] To adapt to the curvature characteristics of the automotive part surface, the moving least squares method is used to calculate the change rate of the normal vector on the surface of the automotive part around the defect area. Take the point cloud data within 20 mm of the outer periphery of the defect boundary, fit the local surface within a radius of 10 mm around each point, and calculate the normal vector. By comparing the included angles of the normal vectors of adjacent points, the change rate of the normal vector is obtained. In practical applications, for the above-mentioned automotive fender surface, the change rate of the normal vector is distributed between 0.002 and 0.058. According to this distribution, the surface is divided into three types of curvature characteristic regions: flat region (change rate < 0.01), transition region (0.01 - 0.03), and high - curvature region (> 0.03).

[0068] Construct compensation functions in different curvature characteristic regions to deform the angular - radius distribution. In the flat region, the compensation coefficient is 1.05; in the transition region, the compensation coefficient is 1.12; in the high - curvature region, the compensation coefficient is 1.20. Adjust the angular - radius distribution of the original defect boundary through these compensation coefficients to finally generate the point sequence of the laser engraving compensation path. Practical tests show that the laser engraving repair effect processed by this compensation method is significantly better than the traditional method, the edge transition is more natural, and the color difference between the repaired surface and the surrounding original surface is controlled within △E ≤ 1.8, meeting the high - end automotive appearance quality requirements.

[0069] In this embodiment, the defect centroid is accurately calculated by the method of key - point density weighting, and the angular and radial changes of the defect boundary are dynamically modeled in the polar coordinate system, making the representation of the defect contour more in line with its true morphological characteristics. Compared with the traditional method of constructing the compensation path only based on the geometric center or fixed sampling method, this scheme introduces the key - point importance scoring and curvature variance weighting mechanism, effectively improving the ability to depict the complex boundary structure. At the same time, combined with the moving least squares method to analyze the change of the normal vector on the part surface, construct the local curvature response region, and perform compensation function deformation in different regions, making the laser engraving path more conform to the part surface morphology, realizing refined and adaptive compensation planning, and significantly improving the accuracy and surface consistency of the laser engraving process.

[0070] Figure 3 It is a schematic diagram of the defect centroid calculation iteration and the key - point importance distribution. The scattered dots in the figure represent the set of key points extracted from the segmented defect area. These key points are classified into two types according to their importance scores: Light blue small circles: represent "ordinary key points". After calculating the contribution value of the multi-scale Gaussian kernel density and non-linear mapping (such as a variant of the sigmoid function), the importance scores of these key points are relatively low (for example, the score is less than 0.8). Slightly larger golden circles: represent "high-importance key points". These key points have relatively high importance scores (for example, the score is not less than 0.8). They usually correspond to regions with higher density or significant morphological influence in the defect structure and are given greater weights in the centroid calculation.

[0071] The coordinates of the "final defect centroid" obtained by the convergence of the above iterative calculation are marked by green large dots. This centroid point will be used as the reference origin for subsequent establishment of the polar coordinate system, analysis of the angle-radius distribution of the defect boundary, and generation of the laser engraving compensation path point sequence.

[0072] The centroid calculation method proposed by the present invention does not simply take the geometric center, but fully considers the heterogeneity of the key point distribution in the defect region. By assigning higher importance scores to the key points in the high-density region or structure and adopting an iterative weighting method, it can effectively resist the influence of noise key points, making the calculated centroid more conform to the actual mass distribution center or structural core of the defect, laying a solid foundation for subsequent accurate defect morphology description and compensation path generation.

[0073] In an alternative embodiment, combining the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data, and performing layer-by-layer compensation processing on the printed defects on the surface of the automotive part includes: Calculating the cutting force distribution during the processing of the laser engraving compensation path point sequence, obtaining the heat accumulation value during the cutting process, dividing the laser engraving compensation path point sequence into multiple processing units, and determining the cooling time interval for each processing unit based on the heat accumulation value; Determining the total number of processing layers according to the material removal characteristics, distributing the laser engraving compensation depth parameter according to the number of processing layers, calculating the material removal amount for each layer, and generating a layer-by-layer processing sequence; Combining the cooling time interval, calculating the feed rate for each processing unit, converting the laser engraving compensation path point sequence into a motion trajectory with a feed rate, and inserting waiting nodes between adjacent reciprocating paths; Synchronously combining the layer-by-layer processing sequence with the motion trajectory with a feed rate to generate rough machining trajectory data containing timing information, adding tool lift and tool entry positions to the rough machining trajectory data, planning the inter-layer transition path, and establishing a complete tool motion trajectory; Converting the complete tool motion trajectory into three-dimensional compensation trajectory data executable by the machine tool for layer-by-layer compensation processing of the printed defects on the surface of the automotive part.

[0074] This embodiment provides a method for generating three-dimensional compensation trajectory data by combining a laser engraving compensation path point sequence and laser engraving compensation depth parameters, and performing hierarchical compensation machining on the printing defects on the surface of automotive parts. The specific implementation process of this method is as follows: Before performing compensation machining, the system first calculates the cutting force distribution of the laser engraving compensation path point sequence during the machining process. The system establishes a finite element analysis model of the laser engraving machining area, converts the compensation path point sequence into discrete machining units, and the cutting force of each unit is jointly determined by the tool diameter, cutting depth, and feed rate. For example, for a cylindrical tool with a diameter of 0.2 mm, a cutting depth of 0.05 mm, and a feed rate of 100 mm / min, the system can calculate that the cutting force per unit time is 4.8 N.

[0075] Obtaining the heat accumulation value during the cutting process involves calculating the heat generation and diffusion at each machining point. The system uses a heat conduction model, considering the thermal conductivity, specific heat capacity, and density of the material, to calculate the heat propagation law in the material. Taking aluminum alloy as an example, with a thermal conductivity of 237 W / (m·K) and a specific heat capacity of 897 J / (kg·K), it is calculated that under standard cutting parameters, about 3.2 J of heat is generated per second, forming a heat accumulation area with a diameter of about 0.8 mm.

[0076] Based on the heat analysis, the system divides the laser engraving compensation path point sequence into multiple machining units. The division principle is to ensure that the heat accumulation in each machining unit does not exceed the heat resistance threshold of the material. Taking the surface machining of aluminum alloy as an example, when the surface temperature exceeds 120 °C, it will cause a decline in the surface quality of the workpiece. The system divides the path point sequence into a machining unit every 5 mm. Through thermal simulation analysis, aluminum alloy requires a cooling interval of 0.6 seconds under this condition to ensure that the temperature drops to a safe range.

[0077] Determine the total number of machining layers according to the material removal characteristics. The system determines the optimal layering strategy by analyzing the relationship between the material removal rate and cutting parameters. For a coated material with a hardness of 80 HRC, the maximum single-layer removal depth is 0.08 mm. If the total compensation depth is 0.4 mm, it needs to be divided into 5 layers for machining. The system distributes the laser engraving compensation depth parameters according to the number of machining layers, adopting a decreasing strategy: the first layer removes 0.12 mm, the second layer removes 0.1 mm, the third layer removes 0.08 mm, the fourth layer removes 0.06 mm, and the fifth layer removes 0.04 mm. This distribution method can ensure that the machining accuracy is improved layer by layer.

[0078] Calculate the feed rate of each processing unit in combination with the cooling time interval. The calculation method is based on a heat control model to ensure that the heat accumulation does not exceed the threshold. The specific implementation is as follows: the feed rate of the first layer is 80 mm / min, and the feed rates of the subsequent layers are increased to 90 mm / min, 100 mm / min, 110 mm / min, and 120 mm / min in sequence. This strategy controls heat accumulation in the rough machining stage and improves efficiency in the finish machining stage.

[0079] Convert the laser engraving compensation path point sequence into a motion trajectory with a feed rate. For a rectangular defect area of 20 mm × 15 mm, the system generates a reciprocating parallel scanning path with an adjacent path spacing of 0.15 mm, totaling 100 parallel paths. Insert a waiting node of 0.6 seconds between adjacent reciprocating paths. The specific implementation is to add the G04 P0.6 instruction at the path turning points to ensure sufficient heat dissipation.

[0080] Synchronously combine the layer machining sequence with the motion trajectory with a feed rate to generate rough machining trajectory data containing timing information. For 5-layer machining, the system inserts an interlayer cooling time of 5 seconds after each layer is completed, and records the timestamp at the start of each layer and the corresponding Z-axis position. For example, the Z-axis position of the first layer is -0.12 mm, and the start time is 0 seconds; the Z-axis position of the second layer is -0.22 mm, and the start time is 126 seconds; and so on.

[0081] Add the tool lift and cutting positions to the rough machining trajectory data. The system lifts the tool to a safe height of Z = 2 mm before the start of each layer of machining, then quickly positions to the starting point of that layer, and then cuts to the machining depth at a speed of 1 mm / s. The interlayer transition path adopts a diagonal transition strategy with an angle of 45 degrees to ensure smooth tool transition to the starting position of the next layer.

[0082] The system finally converts the complete tool motion trajectory into three-dimensional compensation trajectory data executable by the machine tool. For a numerical control machine tool, the system generates G-codes compliant with the ISO standard; for an industrial robot, it generates motion instructions compliant with the controller of a specific brand. The generated trajectory contains XYZ coordinates, feed rate, dwell time, and cooling instructions.

[0083] In this embodiment, by fusing the laser engraving compensation path point sequence with the compensation depth parameter, three-dimensional compensation trajectory data is constructed, enabling the compensation machining process to have coordinated consistency in space and depth. Compared with the prior art where the path and depth are separated and there is a lack of a thermodynamic feedback mechanism, this solution introduces an analysis of cutting force distribution and heat accumulation to achieve dynamic cooling time control of the machining unit, effectively preventing thermal damage and contour deformation. Layered planning is carried out in combination with the material removal characteristics, and the feed speed and insertion waiting nodes are synchronously adjusted, improving the continuity of the path and machining stability. In addition, through the generation of a complete tool trajectory and the integration of timing information, the machine tool can accurately perform layered compensation operations, significantly improving the machining accuracy of printing defects and the consistency of finished products, and is applicable to high-demand automotive part surface repair scenarios.

[0084] In the second aspect of the embodiments of the present invention, a printing defect detection system based on machine vision is provided. The system includes: A first unit for collecting an original image of the printed area on the surface of the automotive part to be detected through an industrial camera and performing image enhancement processing to obtain an enhanced detection image; A second unit for constructing a multi-scale gradient pyramid for the enhanced detection image, extracting the histogram of gradient directions at each scale, calculating the mutual information value of the histograms of gradient directions between adjacent scales, determining the optimal detection scale, and performing defect area segmentation at the optimal detection scale to obtain candidate defect areas; A third unit for performing adaptive binarization processing on the candidate defect areas, extracting defect edge contour points, calculating the curvature values of the defect edge contour points, using the contour points with curvature values greater than a preset curvature threshold as key points, performing defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect area, and extracting and classifying texture features of the defect area to obtain the defect type; A fourth unit for calculating the centroid coordinates of the defect as the origin based on the distribution of key points in the defect area, establishing a polar coordinate system, calculating the angle-radius distribution of the defect boundary curve, generating a laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part, setting the laser engraving compensation depth parameter according to the defect type, and combining the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data for performing layered compensation machining on the printing defects on the surface of the automotive part.

[0085] In the third aspect of the embodiments of the present invention, an electronic device is provided, including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0086] In a fourth aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.

[0087] The present invention may be a method, an apparatus, a system, and / or a computer program product. The computer program product may include a computer-readable storage medium, on which computer-readable program instructions for implementing various aspects of the present invention are loaded.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects of printed matter based on machine vision, characterized in that Including: Collecting the original image of the printed area on the surface of the automotive part to be detected through an industrial camera and performing image enhancement processing to obtain an enhanced detection image; Constructing a multi-scale gradient pyramid for the enhanced detection image, extracting the histogram of oriented gradients (HOG) at each scale, calculating the mutual information value of the HOG between adjacent scales, determining the optimal detection scale, and performing defect region segmentation at the optimal detection scale to obtain candidate defect regions; Performing adaptive binarization processing on the candidate defect regions, extracting the defect edge contour points, calculating the curvature values of the defect edge contour points, taking the contour points with curvature values greater than the preset curvature threshold as key points, and performing defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect regions, and extracting and classifying the texture features of the defect regions to obtain the defect types; Calculating the defect centroid coordinates as the origin based on the distribution of key points in the defect region, establishing a polar coordinate system, calculating the angle-radius distribution of the defect boundary curve, generating a laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part, setting the laser engraving compensation depth parameter according to the defect type, combining the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data, and performing layered compensation processing on the printing defects on the surface of the automotive part.

2. The method according to claim 1, wherein Constructing a multi-scale gradient pyramid for the enhanced detection image, extracting the histogram of oriented gradients (HOG) at each scale, calculating the mutual information value of the HOG between adjacent scales, determining the optimal detection scale, and performing defect region segmentation at the optimal detection scale to obtain candidate defect regions including: Performing multi-scale decomposition on the enhanced detection image, convolving the image with a variable-scale Gaussian kernel function, dynamically adjusting the Gaussian kernel parameters based on the inter-layer gradient consistency of the convolution result, and constructing a multi-scale gradient pyramid; Calculating the horizontal gradient and vertical gradient of the pixel points at each scale layer of the multi-scale gradient pyramid, determining the gradient amplitude and direction, dividing the gradient direction space into multiple intervals, and statistically counting the number of gradient points in each interval to construct a histogram of oriented gradients; Constructing a joint probability distribution matrix, normalizing the joint probability distribution matrix, calculating the mutual information value of the HOG between adjacent scale layers based on the normalized joint probability, analyzing the trend curve of the mutual information value changing with the scale, extracting the position of the mutation point of the trend curve, and determining the scale layer with the largest mutual information mutation as the optimal detection scale; On the image layer corresponding to the optimal detection scale, calculating the local gradient amplitude of the image pixel points, taking the pixel points with gradient amplitudes greater than the preset gradient threshold as seed points, constructing a local detection window centered on the seed points, calculating the gradient covariance matrix within the local detection window, determining the growth threshold, and performing region growth on the seed points according to the growth threshold to obtain candidate defect regions.

3. The method according to claim 2, wherein Analyzing the trend curve of the mutual information value changing with the scale, extracting the position of the mutation point of the trend curve, and determining the scale layer with the largest mutual information mutation as the optimal detection scale including: Construct a trend curve of the mutual information value varying with the scale, construct a structure tensor for the local structure of the trend curve, adaptively determine the weight coefficient based on the eigenvalue distribution of the structure tensor, perform non-linear local weighting on the trend curve to obtain an enhanced trend curve; Calculate the normalized difference of the enhanced trend curve at adjacent scale levels, construct an adaptive threshold in combination with the local gradient density distribution, and extract the mutation features of the trend curve based on the adaptive threshold; at the same time, perform wavelet decomposition on the enhanced trend curve, extract the curvature coefficients at different scales, and construct a mutation point detection operator based on the multi-scale response of the curvature coefficients; Adaptively fuse the mutation features with the mutation point detection operator, construct a mutation point evaluation function, analyze the time-frequency characteristics of the mutation point evaluation function, extract the main frequency component of the feature response, determine the candidate mutation point set of the trend curve based on the main frequency component, construct a local phase consistency map at each scale level of the candidate mutation point set, and calculate the complementary texture features of the local phase consistency map and the gradient direction field; Verify the reliability of the candidate mutation points based on the complementary texture features, and select the scale level with the maximum mutual information mutation as the optimal detection scale.

4. The method according to claim 1, wherein Perform adaptive binarization processing on the candidate defect region, extract the defect edge contour points, calculate the curvature values of the defect edge contour points, and use the contour points with curvature values greater than the preset curvature threshold as key points, and perform defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect region including: Construct a multi-directional filter bank for the candidate defect region, calculate the response intensity in each direction, extract the maximum response direction as the main direction from the response intensity, and perform adaptive enhancement on the neighborhood of the main direction. Based on the enhanced response intensity, construct a binarization threshold function to perform adaptive binarization processing on the candidate defect region to obtain a binarized image; Construct a double-boundary tracking operator, use the double-boundary tracking operator to perform boundary tracking in the binarized image, extract the candidate contour points with double-boundary responses, and screen the candidate contour points based on the continuity of the main direction. Determine the candidate contour points whose main direction continuity meets the preset continuous condition as the defect edge contour points. With the defect edge contour points as the center, construct an annular neighborhood that changes with the main direction, and calculate the curvature values of the contour points in the annular neighborhood; Set a dynamic curvature threshold according to the continuity of the main direction, and mark the contour points with curvature values exceeding the dynamic curvature threshold as key points; Extract the spatial position sequence of the key points, calculate the polar coordinate parameters between adjacent key points based on the main direction, construct a shape feature vector including the distance ratio and the angle difference, and use the shape feature vector to perform defect shape discrimination to obtain the defect region.

5. The method according to claim 1, wherein Extract and classify the texture features of the defect region to obtain the defect types including: Extract the edge features of the direction gradient histogram of the defect region at multiple scales, weight them using the gradient amplitude, and construct local texture features through non-uniform sampling and adaptive binarization to obtain the edge texture feature vector; Calculate the gray-level co-occurrence matrix of the defect area based on multiple angles and distances, dynamically determine the window size according to the regional complexity to extract contrast and correlation information, and perform weighted processing in combination with the local variance distribution to obtain a statistical texture feature vector; Track and locate the boundary of the defect area and calculate the edge curvature distribution, construct a defect skeleton structure and extract branch connectivity information, and generate a shape feature vector in combination with the regional distance transform; Normalize the edge texture feature vector, statistical texture feature vector and shape feature vector, calculate the ratio of the inter-class distance to the intra-class variance between features as the weight coefficient, and select and combine the features according to the weight coefficient to obtain a combined feature vector; Use the shape feature vector to divide the defect area into linear defect areas and block defect areas, combine the combined feature vector to identify the types of the divided defect areas, and at the same time construct a spatial association matrix of the defect area to optimize the defect type recognition result, and perform feature cross-validation on the recognition results with a recognition confidence less than the preset confidence threshold, and output the defect type.

6. The method according to claim 1, wherein Calculate the defect centroid coordinates based on the key point distribution of the defect area as the origin, establish a polar coordinate system, calculate the angle-radius distribution of the defect boundary curve, and generate a laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part, including: Calculate the local density contribution value of the key points in the defect area at multiple Gaussian kernel scales, perform non-linear mapping on the density contribution value to obtain the key point importance score, use the importance score as the weight coefficient, and calculate the defect centroid coordinates by the iterative weighted least squares method; Establish a polar coordinate system with the defect centroid coordinates as the origin, dynamically adjust the angle sampling interval based on the local curvature value of the defect boundary contour points, calculate the curvature variance of adjacent sampling points, and use the curvature variance as the weight coefficient to perform weighted calculation on the radial distance of the boundary points to generate the angle-radius distribution of the defect boundary curve; Use the moving least squares method to calculate the normal vector change rate of the automotive part surface around the defect area, divide the curvature feature area according to the normal vector change rate, and construct a compensation function in each curvature feature area to deform the angle-radius distribution to generate a laser engraving compensation path point sequence.

7. The method according to claim 1, wherein Combine the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data, and perform layered compensation processing on the printing defects on the automotive part surface, including: Calculate the cutting force distribution during the processing of the laser engraving compensation path point sequence, obtain the heat accumulation value during the cutting process, divide the laser engraving compensation path point sequence into multiple processing units, and determine the cooling time interval of each processing unit based on the heat accumulation value; Determine the total number of processing layers according to the material removal characteristics, distribute the laser engraving compensation depth parameter according to the number of processing layers, calculate the material removal amount of each layer, and generate a layered processing sequence; Combine the cooling time interval, calculate the feed speed of each processing unit, convert the laser engraving compensation path point sequence into a motion trajectory with a feed speed, and insert waiting nodes between adjacent reciprocating paths; Synchronously combine the said layered machining sequence with the motion trajectory with a feed speed to generate rough machining trajectory data containing timing information, add tool lifting and tool lowering positions to the rough machining trajectory data, plan the inter-layer transition path, and establish a complete tool motion trajectory; Convert the complete tool motion trajectory into three-dimensional compensation trajectory data executable by the machine tool for performing layered compensation machining on the printing defects on the surface of the automotive part.

8. A printing defect detection system based on machine vision for implementing the method according to any one of the preceding claims 1-7, characterized in that, Comprising: A first unit for collecting an original image of the printing area on the surface of the automotive part to be detected through an industrial camera and performing image enhancement processing to obtain an enhanced detection image; A second unit for constructing a multi-scale gradient pyramid for the enhanced detection image, extracting the histogram of gradient directions at each scale, calculating the mutual information value of the histograms of gradient directions between adjacent scales, determining the optimal detection scale, and performing defect area segmentation at the optimal detection scale to obtain candidate defect areas; A third unit for performing adaptive binarization processing on the candidate defect areas, extracting the defect edge contour points, calculating the curvature values of the defect edge contour points, taking the contour points with curvature values greater than a preset curvature threshold as key points, performing defect shape discrimination based on the spatial distribution characteristics of the key points to obtain the defect area, and extracting and classifying the texture features of the defect area to obtain the defect type; A fourth unit for calculating the centroid coordinates of the defect as the origin based on the distribution of the key points in the defect area, establishing a polar coordinate system, calculating the angle - radius distribution of the defect boundary curve, generating a laser engraving compensation path point sequence according to the surface curvature characteristics of the automotive part, setting the laser engraving compensation depth parameter according to the defect type, combining the laser engraving compensation path point sequence and the laser engraving compensation depth parameter to generate three-dimensional compensation trajectory data for performing layered compensation machining on the printing defects on the surface of the automotive part.

9. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images