Image data enhancement processing method for industrial visual inspection

Abstract

The invention relates to the technical field of image data processing, and discloses an image data enhancement processing method for industrial visual inspection, which comprises the following steps of: performing hue correction on a defect image block, calculating an original gradient field of the defect image block, multiplying the original gradient field by an intensity attenuation factor which is greater than 0 and less than 1 to obtain an attenuated gradient field; and finally, solving a gradient domain fusion equation, taking the attenuated gradient field as internal guidance, and enabling the boundary of the gradient field to be seamlessly fused with the target background image. By introducing the intensity attenuation factor, a regulation and control mechanism of defect sensing intensity is established, so that defect samples with weak signals can be generated as required, and the defect sensing intensity is improved. The method is used for training a model to cope with a low-signal-to-noise-ratio detection scene, and the detection sensitivity of the model is improved.

Description

Technical Field

[0001] This invention relates to an image data enhancement processing method for industrial visual inspection, belonging to the field of image data processing technology. Background Technology

[0002] In current applications of image data processing and generation, especially in industrial vision inspection systems that rely on deep learning, the quality and diversity of the training dataset are among the factors that determine system performance. However, in many industrial scenarios, particularly when dealing with materials with complex textured surfaces, defect samples are scarce. Therefore, artificially synthesizing high-quality training samples through image data augmentation has become a routine and necessary technique in this field. One common image augmentation method is to overlay a known defective image patch onto a defect-free target background image to expand the dataset. However, this synthesis method has a long-standing inherent technical limitation in image processing mechanisms: simple pixel value overlay, such as transparency blending, cannot handle the differences in lighting, tone, and texture between the defect and the background. This inevitably produces visual artifacts such as hard edges or blurry halos at the fusion seams. These artifacts, as a high-frequency statistically false feature, are preferentially captured by deep learning networks during model training and mistakenly used as the core criterion for defects.

[0003] This cognitive bias introduced by the data generation method causes the model to, on the one hand, miss subtle defects that do not exist in the real world during actual detection; on the other hand, it is very easy to judge any normal but drastic changes in texture or lighting in a defect-free background (because they are statistically similar to artifacts) as defects, leading to serious false alarms. Existing multimodal data processing and fusion systems also have fundamental shortcomings in the real-time performance, adaptability, and continuous optimization capabilities of the system-level architecture. For example, Chinese invention patent CN119444686A discloses an industrial vision inspection system based on multimodal fusion. Although this system proposes to include data acquisition and calibration, scene image capture, visual information synthesis, algorithm optimization and training, etc., The system architecture incorporates deep learning image analysis algorithms within the visual information synthesis module. However, in its core data processing stage, it primarily relies on traditional image alignment methods such as feature point detection, descriptive extraction, and transformation matrices to ensure spatial consistency. This alignment method based on fixed features lacks robustness for dynamic, non-rigid, or unevenly textured industrial product scenarios. Once feature point extraction or matching fails, the accuracy of the fusion result will drop sharply, severely impacting the reliability of the final detection report. Furthermore, the system only mentions defining the cost function and adjusting weights in terms of training optimization, lacking a deep feedback and iterative optimization mechanism based on objective quality evaluation metrics (such as PSNR or SSIM) for the entire process of multimodal synthesis model from preprocessing and alignment to fusion operations.

[0004] Therefore, the technical problem to be solved by this invention is how to provide an image data enhancement processing method that is no longer a simple application of a single tool, but can systematically and sequentially solve the complex problems of context incompatibility, input source contamination, global brightness difference and local boundary discontinuity in image synthesis in a complete processing flow, and achieve controllable generation of defect intensity. Summary of the Invention

[0005] This invention provides an image data enhancement processing method for industrial visual inspection. Its main purpose is to solve the complex problems of context incompatibility, input source contamination, global brightness difference and local boundary discontinuity in image synthesis, and to achieve the problem of controllable generation of defect intensity.

[0006] To achieve the above objectives, the present invention provides an image data enhancement processing method for industrial visual inspection, comprising the following steps:

[0007] Obtain a defect image patch from a source image and a defect mask used to identify the defect image patch; obtain a target background image and the target location used for fusion on the target background image.

[0008] Based on the image features of the defect image patch and the target background image at the target location, the defect image patch is tone-corrected to obtain a tone-corrected defect image patch; the original gradient field of the tone-corrected defect image patch in the region defined by the defect mask is calculated; an intensity attenuation factor greater than 0 and less than 1 is set; the original gradient field and the intensity attenuation factor are multiplied point by point to obtain an attenuated gradient field.

[0009] Solve a gradient domain fusion equation that satisfies the following two objective constraints: Condition a, within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to the decayed gradient field; Condition b, at the boundary of the fusion region, the pixel value of the solved image is constrained to be equal to the pixel value of the target background image at the boundary; and output the solution of the gradient domain fusion equation as an enhanced image containing the fusion defect.

[0010] Preferably, the step of performing tone correction on the defective image block based on the image features of the defective image block and the target background image at the target location specifically includes: extracting defective region pixels of the defective image block within the area defined by the defect mask, extracting background region pixels of the target background image within the same area as the defect mask at the target location, calculating the color histogram of the defective region pixels and the color histogram of the background region pixels, and applying a histogram matching algorithm to match the color histogram of the defective region pixels to the color histogram of the background region pixels to obtain the tone-corrected defective image block.

[0011] Preferably, the gradient domain fusion equation is the Poisson equation. The steps to solve a gradient domain fusion equation are as follows: determine the final pixel value inside the fusion region by solving the Poisson equation.

[0012] Preferably, before performing tone correction on the defective image block, the method further includes: analyzing the image statistical features of all pixels within the area covered by the defect mask; automatically separating all pixels into a true defect pixel group and a background contamination pixel group based on a clustering algorithm; generating a purified defect mask based on the true defect pixel group; and using the purified defect mask to perform subsequent tone correction steps and solve a gradient domain fusion equation.

[0013] Preferably, the clustering algorithm is the K-Means clustering algorithm, and the number of clusters K in the K-Means clustering algorithm is set to 2.

[0014] Preferably, before the step of tone correction of the defective image patch, the method further includes a context compatibility pre-matching step, which includes: using a texture description algorithm to extract the original context texture features of the defective image patch in the neighborhood outside its defect mask; scanning multiple candidate target locations on the target background image and using the texture description algorithm to extract the candidate background texture features of each candidate target location; calculating the texture similarity distance between the original context texture features and each of the multiple candidate background texture features; and determining the candidate target location as the target location for performing subsequent steps only when the texture similarity distance is less than a preset context compatibility threshold.

[0015] Preferably, the texture description algorithm is a local binary mode algorithm, and the texture similarity distance is the chi-square distance.

[0016] Preferably, the steps of setting an intensity attenuation factor and performing point-by-point scalar multiplication of the original gradient field with the intensity attenuation factor are modified as follows: Based on the internal image features of the defect image block after tone correction, a spatial attenuation map is generated. The spatial attenuation map is a matrix of the same size as the defect mask, and each element value in the matrix of the spatial attenuation map is greater than 0 and less than 1. Then, the original gradient field and the spatial attenuation map are multiplied point-by-point to obtain the attenuated gradient field.

[0017] Preferred spatial attenuation spectrum The generation rules include: first, calculating the value of each point within the fusion region. Minimum distance to the boundary of the defect mask Then, based on the minimum distance calculate The numerical value is calculated according to the following rules: ,in, This is a preset minimum attenuation value greater than 0. This is a preset distance attenuation coefficient.

[0018] Preferably, the gradient domain fusion equation is set to satisfy two objective constraints simultaneously, and condition a is modified to: within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to a mixed gradient field, and the mixed gradient field selects the attenuated gradient field and the gradient of the target background image that has a larger magnitude at the corresponding point.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] 1. This method establishes a domain-specific processing flow for image data. First, in the global statistical feature domain, histogram matching is used to correct the overall illumination and tone differences between the defective image patch and the target background, providing a unified brightness basis for subsequent fusion. Then, in the local differential feature domain, the gradient domain fusion equation is solved. This process focuses on preserving the internal gradient structure of the defect while allowing its boundary pixels to naturally transition to the background. This order of assimilating global features first and then solving local features avoids the problem of gradient solver failure due to sudden changes in illumination, and also avoids texture blurring caused by separate tone correction. This ensures that the generated image maintains both the brightness continuity at the boundary and the authenticity of the internal texture.

[0021] 2. By eliminating boundary artifacts, the generated enhanced image has a visual realism. During training, the deep learning model no longer incorrectly learns the false features at the fusion seam as defect criteria. The model's attention is guided to the internal structure and texture features of the defect sample. This makes it less likely for the model to produce false alarms for normal drastic texture changes such as wrinkles or color differences in a defect-free background during actual detection. At the same time, it is also less likely to miss real defects that blend into the background.

[0022] 3. This method also establishes a multi-dimensional image data generation and verification process. Before fusion, context compatibility pre-matching based on texture descriptors is introduced to pre-exclude fusion locations where defective textures are logically incompatible with background textures, thus avoiding logical artifacts. On this basis, a clustering algorithm is used to statistically separate pixels within the mask, automatically purifying defective input sources and avoiding ghost artifacts caused by background contamination. Finally, this input, which has undergone dual purification of context and content, is then processed by the core tone correction and gradient domain fusion steps. This progressive processing approach, from logical rationality and content purity to boundary realism, ensures that the generated enhanced image closely resembles the real physical world in terms of semantic content and optical dimensions. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the multi-stage image data enhancement processing method of the present invention;

[0024] Figure 2 DSNR and the defect signal strength modulation effect of the present invention Relationship curve diagram;

[0025] Figure 3 This is a diagram illustrating the application architecture of the data augmentation method of this invention in an industrial detection closed-loop system. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be noted that the specific embodiments and their descriptions herein are merely for explaining the invention and are not intended to limit the scope of protection of the invention.

[0027] This invention provides an image data enhancement processing method for industrial visual inspection. The method constructs a multi-stage image data processing flow, aiming to generate visually seamlessly fused image data with controllable defect intensity, thereby improving the performance of deep learning models in industrial inspection applications. This image data processing flow can be executed by a computing device configured with a general-purpose processor and standard image processing libraries. Its core stages may include: an optional context-compatible pre-matching stage, an optional defect mask purification stage, a global brightness correction stage, a defect signal intensity modulation stage, and a local gradient domain fusion stage. In one specific embodiment, the execution of this image data enhancement processing method begins with data acquisition and preprocessing. The system first acquires defect image patches from a source image. And a binarized defect mask for accurately identifying defect regions in defective image patches. Simultaneously, the system acquires one or more defect-free target background images. Given that in industrial applications, blindly blending a defect in one texture, such as a broken thread in fabric, into another completely unrelated texture, such as a brushed metal surface, can produce logical artifacts that mislead subsequent model training, this method preferably initiates a context-compatible pre-matching step before performing any pixel processing. This step is used to match the target background image... The system first selects compatible target locations based on texture compatibility. The specific procedure is as follows: First, the system employs a well-known texture description algorithm, specifically Local Binary Pattern (LBP) algorithm, to analyze defective image patches. Its defect mask The external neighborhood, that is, the original context in which the defect is located, is analyzed to extract and calculate an original contextual texture feature vector. Next, the system analyzes the target background image. Up scan multiple candidate target locations And using the exact same LBP algorithm, the location of each candidate target is extracted. Candidate background texture features of the corresponding region Subsequently, the system calculates... With each The texture similarity distance between the two textures can be quantized using the chi-square distance, a standard technique in this field. Finally, the system compares this distance with a preset context compatibility threshold. Compare this threshold This can be determined through an offline calibration process. For example, by calculating the chi-square distance between a set of known compatible and incompatible texture pairs, and selecting a numerical point that best distinguishes between the two classes. The system ultimately only considers those textures with a similarity distance of less than [a certain value]. Candidate target location This is determined as the target location for performing subsequent fusion steps, thus ensuring the texture logic consistency of the synthesized image at the source of processing.

[0028] After determining the context-compatible target location, considering the defect mask provided by the operator during actual data acquisition... Defects often contain background contamination around their surroundings, which can interfere with subsequent brightness correction and produce ghost artifacts. This method preferably performs a defect mask purification step, which employs a clustering algorithm, such as K-Means clustering, and counts the number of clusters. Determined as The procedure is as follows: The system first extracts the defect mask. The system analyzes all pixels within the covered area and uses their image statistical features, such as the pixel values ​​or grayscale values ​​of the R, G, and B channels, as input to the K-Means algorithm. After execution, the algorithm automatically separates all pixels into a true defect pixel group and a background contamination pixel group. The system automatically identifies the background contamination pixel group using a deterministic rule, such as determining which group has more pixels or which group's average pixel value is closer to the background pixel value of the mask's outer neighborhood. Finally, based on the true defect pixel group, the system generates a purified defect mask containing only the real defect areas. and use this The system then executes all subsequent processing steps. Next, it enters the global brightness correction stage to address the illumination and tone differences between the defective image patch and the target background. This step is the luminance foundation for achieving seamless fusion, and its procedure is as follows: Based on the image features of the defective image patch and the target background image at the determined target location, the system performs tone correction on the defective image patch. Specifically, this step employs a histogram matching algorithm well-known in the field. The system first extracts the defective image patch from the purified defect mask. Calculate the color histogram of the defective pixels within the defined region. Simultaneously, the system extracts the background image of the target at the target location and... Calculate the color histogram of background pixels within a large region. The histogram matching algorithm then calculates its respective cumulative distribution function (CDF), such as... and Establish a pixel value mapping relationship, and Match to Applying this mapping relationship, the system obtains a color-corrected defective image patch. The image patch is consistent with the target background area in terms of global illumination and tone.

[0029] Subsequently, in order to generate samples on demand for training the model to cope with low signal-to-noise ratio detection scenarios, this method introduces a signal intensity modulation mechanism. This mechanism is executed in the differential domain, i.e., the gradient domain, rather than the pixel domain, and its procedure is as follows: First, the system calculates the defect image patch after tone correction. In the purified defect mask The original gradient field within the defined region This gradient field It is a vector field that represents the texture and structural information inside the defect, which can be obtained by analyzing... The gradient is obtained by applying standard gradient operators, such as the Sobel operator; then, the system sets an intensity decay factor. This factor It is determined to be a scalar value greater than 0 and less than 1, for example, it can be specified by the user as 0.2, or... Random sampling is performed within the interval to increase data diversity; then, the system applies the original gradient field. With intensity attenuation factor Perform point-by-point scalar multiplication. This yields a decayed gradient field. ;this The gradient direction of the original defect is preserved, but its gradient magnitude is reduced proportionally. Finally, the system performs a gradient domain seamless fusion step to solve the boundary artifact problem caused by direct pixel domain superposition. The purpose of this step is to achieve a continuous transition between the defect boundary and the target background in a differential sense while maintaining the attenuated internal structure of the defect. The procedure is as follows: The system solves a gradient domain fusion equation, which can be specifically a Poisson equation in the relevant field, and determines the final pixel value inside the fusion region by solving the Poisson equation. The equation is set to simultaneously satisfy the following two objective constraints: Step a, in the purified defect mask Within the defined fusion region, the solved image The gradient field is constrained to be equal to the decayed gradient field. ,Right now This constraint guides the fusion processor to reconstruct the internal texture of defects with controlled intensity; in step B, at the boundary of the fusion region, the solved image... The pixel values ​​are constrained to be equal to the target background image. The pixel value at the boundary, i.e. This constraint ensures seamless continuity of pixel values ​​at the boundary. The system solves the Poisson equation using standard numerical methods, such as the finite difference method combined with an iterative solver, and finally outputs the solution. As an enhanced image with seamless boundaries and controllable intensity.

[0030] Where there is no conflict, the present invention also provides alternative implementations of the above-described processing flow. For example, the step for controlling the defect intensity can be replaced by a spatially variable method: the system can generate a spatial attenuation map of the same size as the defect mask. Each element in the graph has a value greater than 0 and less than 1; The generation rules can rely on geometric features; for example, first calculate the value of each point within the fusion region. Minimum distance to the mask boundary Then based on that distance calculate The value of follows a deterministic rule, for example... ,in This is a preset minimum attenuation value greater than 0. A preset distance attenuation coefficient is used; at this time, the attenuated gradient field is then... This is obtained through point-by-point multiplication, a method that allows for the generation of defect signals with different intensities at the center and edges; in one specific implementation, when a spatial attenuation map is used... When performing signal modulation, its internal parameters and The engineering calibration procedure includes: First, based on the signal-to-noise ratio requirements of the weakest defect for the specific testing task, a minimum attenuation value is set. For example, it can be set to 0.05 or 0.1; then, the system analyzes the purified defect mask. Given the geometric shape of the mask, calculate the distance from each pixel within the mask to its nearest boundary, and determine the maximum distance value. Finally, the distance attenuation coefficient The value is obtained through The calculation formula determines this, ensuring that the gradient decay intensity of the defect core region furthest from the mask boundary converges to the set value. This provides a deterministic parameter setting basis for the spatially variable attenuation method. Furthermore, the internal constraints of the gradient domain fusion equation, i.e., step a mentioned above, can also be modified: within the fusion region, the gradient field of the solved image can be constrained to be equal to a mixed gradient field; this mixed gradient field is a gradient selection mechanism, which at each point within the fusion region... At each location, calculate the decayed gradient field. The gradient magnitude and the target background image gradient magnitude at the corresponding point The gradient vector with the larger magnitude is preferentially selected as the guiding gradient at that point. This hybrid approach allows for the fusion of defective textures while preserving the original texture features of the target background, resulting in a transparent overlay visual effect. In another specific implementation, when an iterative solver (e.g., the Gauss-Seidel method) is used to solve the gradient domain fusion equation, the convergence condition for terminating the iteration is set as follows: the entire fused image generated in the next iteration... Compared with the previous iteration result Between these points, the sum of squared differences (SSD) of all corresponding pixel values ​​is less than a preset convergence threshold. (For example Desirable or The iteration terminates and outputs the current result when the total number of iterations reaches a preset upper limit (e.g., 5000); and when the internal constraints are modified to use a hybrid gradient field, the hybrid gradient... At every point within the integration area The vector value selection procedure at each point is as follows: calculate the decayed gradient vector respectively. The L2 norm (i.e., gradient magnitude) and the gradient vector of the target background image at the corresponding point. The L2 norm of the point is calculated, and the vector with the larger L2 norm value is selected as the point. The final gradient vector used to solve the constraints. .

[0031] Example 1: In a specific industrial visual inspection application scenario, the goal is to train a deep learning model to detect subtle scratches on the surface of high-value brushed metal, which are on the edge of detection. Such samples are scarce in real production lines, and their signal characteristics are statistically similar to the brushed texture of the background material itself. Directly collected sample libraries cannot meet the model training requirements for low signal-to-noise ratio scenarios. To generate such data, the system first acquires a source image containing typical but relatively high-contrast scratches. And a defect mask whose area is larger than the actual defect, drawn by the operator. Simultaneously, the system acquires a batch of defect-free target background images with different lighting conditions, but all with acceptable textures. The method first performs a context-compatible pre-matching step, using the Local Binary Pattern (LBP) algorithm to extract defective image patches. Its defect mask Original contextual texture features of the outer neighborhood This feature characterizes the texture orientation and density of the original brushed background; the system then applies this feature to the target background image. Up scan multiple candidate target locations Extract its candidate background texture features By calculating the chi-square distance, the system only filters out those... and Distance less than the preset context compatibility threshold Location This procedure ensures that the target location and the defect source have a consistent brushed texture context, helping to avoid logical artifacts caused by texture mismatch; then, the system applies the input defect mask... Perform the defect mask purification step, because The data may contain background contamination pixels from the original background. Without purification, the statistical characteristics of these pixels may interfere with subsequent tone correction, and their gradient information may also produce ghost artifacts during fusion. Therefore, the system employs the K-Means clustering algorithm to... All pixels under the coverage are separated into two clusters: a group of true defective pixels and a group of background contamination pixels, and the number of clusters is calculated. Set as A purified defect mask is generated based on the true defect pixel group. This purified defect mask This was then used in the global brightness correction stage, where the system applied a histogram matching algorithm to identify defective image blocks. exist The color histogram of defective pixels within the region is matched to the target background image. At the selected target location The color histogram of the background pixels in the corresponding area is used to output the defect image patch after tone correction. This brightness processing provides a unified brightness basis for subsequent gradient domain fusion, allowing the fusion solver to focus on gradient continuity and reduce the need to handle drastic lighting changes.

[0032] Before fusion, in order to achieve the goal of generating low signal strength defect samples, the system activated a defect signal strength modulation mechanism. The system first calculated... exist Original gradient field within the region Subsequently, the system sets an intensity attenuation factor. ,For example Then, perform pointwise scalar multiplication to obtain a decayed gradient field. ;this The structural information of the scratches was preserved, but their gradient magnitude was reduced; finally, the system solved a gradient domain fusion equation based on the Poisson equation, which was set to satisfy two objective constraints simultaneously: firstly, in Within the region, the image to be solved The gradient field is constrained to be equal to (Right now Secondly, in At the boundary, the solved image The pixel values ​​are constrained to be equal to the target background image. The pixel value at that boundary (i.e. The enhanced image output by solving this equation has its scratch defect signal intensity controlled at a low level, while the defect boundary and the brushed background image achieve a continuous gradient transition, avoiding the boundary abruptness that may be caused by direct superposition of pixel values. Through the application of this series of image data processing steps, this method transforms an original sample that may have context incompatibility risks, mask containing background pollution and signal intensity that does not meet the requirements into an enhanced image data with consistent context logic, purified content, unified lighting, smooth boundaries and signal intensity that meets the training objectives. It is suitable for model training in low signal-to-noise ratio scenarios.

[0033] Example 2: To objectively verify the practical effect of an image data enhancement processing method for industrial visual inspection, particularly its ability to eliminate boundary artifacts and control defect intensity, the following comparative experiment was designed and executed. The purpose of the experiment was to quantitatively evaluate the differences between the enhanced image generated by the method of this invention and the image generated by the traditional pixel overlay method in terms of boundary realism and defect signal intensity control. The experimental platform was configured as a general-purpose computer equipped with standard image processing libraries such as OpenCV 4.5 and numerical computing capabilities, with an Intel Core i7-10700 processor, 16GB of memory, and a Windows 10 operating system. The data sources used in the experiment included two types: one type was the source images, which included representative defect images collected from industrial production lines. The image patches and their corresponding pixel-level binary defect masks were selected, specifically two typical defects: fabric yarn breakage defects (source image resolution 1024x1024 pixels, 8-bit grayscale) and metal surface scratch defects (source image resolution 512x512 pixels, 8-bit grayscale). Another type was the target background image, which selected fabric background images (resolution 2048x2048 pixels) and metal wire drawing background images (resolution 2048x2048 pixels) that matched the texture of the defect source but were defect-free. All images were acquired under uniform industrial line scan light source conditions to reduce initial interference caused by inconsistent lighting. To simulate the noise effect in engineering reality, Gaussian white noise with a mean of 0 and a standard deviation of 5 grayscale values ​​was superimposed on the target background images of some experimental groups.

[0034] The experiment consisted of two main groups: a control group and the present invention sample group. The control group used a commonly used transparency blending method to generate enhanced images. The blending process involved directly superimposing defective image patches (without tone correction) onto designated positions in the target background image, with a blending transparency factor set to 0.8. This method simulated simple pixel value superposition. The present invention sample group strictly followed the complete process disclosed in the aforementioned specific embodiments to generate enhanced images, including optional context-compatible pre-matching (in this experiment, it was assumed that texture-compatible positions had been selected after screening), defect mask purification (using K-Means clustering, K=2), global brightness correction based on histogram matching, and gradient domain fusion steps based on solving the Poisson equation. To verify the defect intensity control mechanism, the present invention sample group was further subdivided into multiple subgroups, each with different intensity attenuation factors. The specific value is (No attenuation) , and To quantify the quality of the generated images, two core metrics were defined: the first is the Boundary Artifact Metric (BAM), which measures the smoothness of the boundary transition by calculating the root mean square value of the gradient magnitude difference between the image patches inside and outside the fusion region boundary (with a width of 5 pixels). A smaller BAM value indicates a smoother boundary and less noticeable artifacts. The second is the Defect Signal-to-Noise Ratio (DSNR), defined as the ratio of the average gradient magnitude inside the defect region to the average gradient magnitude of the background region, used to quantify the discernibility or intensity of the defect signal. In the experiment, for each defect type and background combination, 10 enhanced images were generated in each experimental group (control group and each subgroup of this invention), and the average values ​​of each metric were calculated. The experimental process and data results are recorded below, see Table 1. The table shows some key data examples of the enhanced images generated on the fabric background for fabric yarn breakage defects. The original input represents the state before direct superposition, and the intermediate feature is the difference between the average gray value of the defect region and the average gray value of the background region after tone correction. This is used to demonstrate the brightness correction effect, and the final output is the calculated BAM and DSNR indices.

[0035] Table 1: Example table of enhanced image quality assessment data for fabric yarn breakage defects.

[0036]

[0037] As can be seen from Table 1 and the analysis of similar test data for metal scratch defects, the control group did not undergo color correction ( The image generated by directly stacking pixels (with high BAM values) exhibits abrupt gradient changes at the boundaries, resulting in generally high BAM values ​​(38.5). Even after adding noise, the BAM value further increases (42.1), indicating severe boundary artifacts. In contrast, the sample group of this invention reduces the initial grayscale difference between defects and the background through a global tone correction step. The BAM value was reduced to around 4.1. Subsequently, a gradient domain fusion step was used to maintain the gradient continuity of the fusion boundary. The BAM values ​​of all subgroups were significantly reduced to single digits (e.g., 5.3), and remained at a low level (e.g., 6.8) even in noisy backgrounds. This objectively confirms the effectiveness of the method in eliminating boundary artifacts. Simultaneously, by adjusting the intensity attenuation factor... The defect signal-to-noise ratio (DSNR) of the sample group in this invention exhibits the same characteristics as... The value shows an approximately linear and controllable decreasing trend, from The DSNR was 6.5 at that time, and gradually decreased to The DSNR was 0.7, which clearly demonstrates that the signal strength modulation mechanism introduced in this invention can generate defect samples with different visibility as needed, including low signal-to-noise ratio samples (DSNR < 1.0) close to the detection limit. Even after adding background noise, the absolute value of DSNR decreases, but it increases with... The controlled trend of change remains. Experimental results show that, compared with the traditional pixel overlay method, the image data enhancement processing method of the present invention for industrial visual inspection can generate enhanced images with smooth boundary transitions and no obvious visual artifacts. Furthermore, the signal intensity of the generated defects can be quantitatively and controllably adjusted by setting an intensity attenuation factor.

[0038] Example 3: This example combines Figures 1 to 3 This describes an image data enhancement processing method for industrial visual inspection, such as... Figure 1 As shown, it receives defect image blocks, defect masks, and target background images as input, and sequentially performs the following steps: an optional context compatibility pre-matching step to use a texture description algorithm to filter texture-compatible target locations and avoid logical artifacts; an optional defect mask purification step to automatically separate true defect pixel groups based on a clustering algorithm such as K-Means with K=2 to avoid ghost artifacts; global brightness correction to solve global illumination and color differences based on a histogram matching algorithm; and the calculation of the original gradient field based on the corrected image. Then, the original gradient field is multiplied by an intensity attenuation factor greater than 0 and less than 1 through a defect signal intensity modulation step to obtain an attenuated gradient field. Finally, in the local gradient domain fusion step, the gradient domain fusion equation such as the Poisson equation is solved to constrain the internal gradient to be equal to the attenuated gradient field and the boundary to be equal to the target background. The final output is an enhanced image containing fused defects with controllable signals and seamless boundaries.

[0039] like Figure 2 As shown in the figure, the horizontal axis represents the intensity attenuation factor. The vertical axis represents the defect signal-to-noise ratio (DSNR). The figure includes two curves: DSNR without noise (solid line) and DSNR with noise (dashed line). As... As the value decreases from 1.0 to 0.1, the DSNR value also decreases steadily, confirming the controllability of this method for defect strength; Figure 3 As shown, in this architecture, the image acquisition device transmits the raw image to the image database, which is responsible for storing the raw defect image, the target background image, and the generated enhanced image. It also provides training data to the model training server containing the deep learning framework to generate a trained model. This model is deployed in an online detection system containing the trained defect detection model. At the same time, the database also provides the raw data to the data augmentation server. Under the control of the configuration parameters issued by the operator terminal, which includes system management and parameter configuration, the latter generates enhanced images and stores them back in the database through its internal image data augmentation processing modules, such as the context compatibility pre-matching module, the defect mask purification module, and the gradient domain fusion module. The detection results of the online detection system are then fed back to the operator terminal, forming a complete deployment and application closed loop.

[0040] Example 4: To further illustrate the technical advantages of the gradient domain fusion step in the method of the present invention compared to the conventional pixel blending method after only performing global brightness correction, the following comparative experiment was conducted. This comparative experiment simulates a method that only partially adopts the technical path of the present invention. This method includes the same initial steps as the sample group of the present invention in Example 2, including acquiring the same source image. Defect mask (and similarly perform defect mask purification to obtain) ), and target background image Furthermore, a global brightness correction step based on histogram matching is also performed to obtain the defective image patch after tone correction. The key difference in this comparative example is that it omits the gradient domain fusion step in the method of this invention, and instead uses a pixel blending technique, namely alpha transparency blending, to blend the defective image patches after tone correction. Blend into target background image The above setup is used to evaluate the effect achievable by relying solely on global brightness correction without subsequent gradient domain processing; the experimental platform, data source, evaluation metrics (BAM and DSNR), and noise conditions are consistent with Example 2, and the alpha blending transparency factor is set to 0.8 and applied to the tone-corrected image; the data obtained from this comparative experiment is named Comparative Sample Group 1, and compared with the present invention sample group in Example 2 (using... For example, a comparison was made, and the results are shown in Table 2; the table also shows some key data examples of the enhanced images generated on the fabric background (including noise conditions) for fabric yarn breakage defects;

[0041] Table 2: Example table of comparative sample group 1 and sample group of the present invention (including noise).

[0042]

[0043] As can be seen from the data in Table 2, due to the implementation of a global tone correction step, the average grayscale difference between the defect area and the background area in comparative sample group 1 is significant. Similarly, the value was reduced to a lower level (4.3), consistent with the sample group of the present invention, indicating that the global brightness difference problem was addressed; however, in the final output boundary artifact metric BAM index, the value of the comparative sample group 1 was 35.8, which was much higher than the 6.8 of the sample group of the present invention, and was in a similar range to the BAM value (42.1) obtained by the control group in Example 2 that directly performed alpha mixing without tone correction; during the processing, it was observed that although the tone tended to be consistent, the image generated by the comparative sample group 1 still had visible hard edges or blurred transition bands at the defect boundaries, especially in areas where there was a structural difference between the defect texture and the background texture; at the same time, the DSNR value was 5.8, reflecting the impact of alpha mixing on signal strength. While the intensity of the signal was considered, the problem of smooth boundary transitions remained unresolved. Examination of the generated images revealed that hard edges also existed in the comparative experiment of metal scratch defects, and due to the high contrast of the metal surface texture, artifacts were more easily observed. Analysis suggests that the results of comparative sample group 1 indicate that relying solely on global brightness correction, such as histogram matching, even with good correction results, failed to resolve the boundary artifact problem caused by the discontinuity in local structure and texture between the defective image block and the background image. Conventional pixel blending methods, such as alpha blending, as a weighted averaging process of pixel values, do not process local gradient information of the image, thus easily causing gradient abrupt changes at the fusion boundary, forming visual artifacts.

[0044] Example 5: This example illustrates the method for determining the context compatibility threshold. And a standardized engineering calibration procedure for identifying background contamination pixel clusters in the defect mask purification step; for calibrating context compatibility thresholds. First, a texture image dataset for calibration and verification is constructed. This dataset contains two types of image pairs: positive sample pairs, consisting of multiple pairs of image patches considered to be texture context compatible in the target industrial application scenario, such as fabrics of the same type and processing batch; and negative sample pairs, consisting of multiple pairs of image patches considered to be texture context incompatible, such as fabric and metal, wood and leather. The construction of the dataset should cover the main texture types and ranges of variation that may be encountered in practical applications. For each image pair in this dataset... Calculate their LBP texture feature histograms respectively. and And calculate the chi-square distance between the two. Collect the set of chi-square distances for all positive sample pairs. The set of chi-square distances to all negative sample pairs Subsequently, the threshold can be determined using receiver operating characteristic curve analysis. By evaluating the true positive rate (TPR) and false positive rate (FPR) that distinguish between positive and negative samples at a series of candidate thresholds, ROC curves were plotted. The chi-square distance value corresponding to the point on the curve that results in a high TPR and a low FPR (either the point closest to the top left corner or the point that meets the false positive rate requirements of a specific application) was selected as the final context compatibility threshold. In a specific calibration experiment targeting textile defect detection, a dataset containing 500 pairs of positive samples (different areas of the same fabric) and 500 pairs of negative samples (fabric and metal / wood) was used. Through the aforementioned ROC analysis method, a [database name missing] was determined. With a value of 0.65, this threshold can achieve a positive and negative sample differentiation accuracy of over 95% on this test set.

[0045] In the defect mask purification step, after performing K-Means clustering (K=2) to divide the pixels within the mask into two clusters, cluster A and cluster B, a procedure for identifying background contamination pixel groups can be established. This procedure combines intra-cluster statistical features with prior defect information. This procedure is based on phenomena observed in most industrial defect images, namely, that defect areas typically have different grayscale or color distribution characteristics relative to their surrounding background. For example, scratches and stains are usually darker or raised, bright spots are usually brighter, and the standard deviation of their pixel value distribution may also differ. Therefore, the procedure is as follows: First, calculate the average grayscale values ​​of the two pixel clusters, A and B, respectively. and standard deviation of gray values Secondly, acquire or set prior information about the type of defect to be detected, i.e., whether the defect is expected to be dark or bright. When processing fabric yarn breaks or metal scratches, it is usually expected that their grayscale value is lower than the background average. Then, make a judgment based on this prior information. If the defect is expected to be dark, then compare... and The cluster with the higher average gray value is identified as the background contamination pixel group; if the expected defect is brighter, the cluster with the lower average gray value is identified as the background contamination pixel group. When prior information is unclear or the defect types are diverse, standard deviation can be introduced as an auxiliary criterion. Generally, the texture consistency of the background area is relatively high, and its pixel value standard deviation is relatively high. The clusters with smaller standard deviations tend to be smaller than the more structurally complex defect regions; therefore, the cluster with the smaller standard deviation can be initially identified as a background contamination pixel group. In an application that processes scratches on a metal surface where a darker color is expected, the average gray level of cluster A is obtained after K-Means clustering. Standard deviation The average gray level of cluster B Standard deviation Due to the expected defects being darker, and ,at the same time Based on this procedure, the system identifies cluster B as the background contaminated pixel group and generates a purified defect mask based on cluster A. .

[0046] Example 6: This example further illustrates a supplementary procedure for automatically identifying background contamination pixel groups when prior information about defect types is insufficient or inapplicable during the defect mask purification step; in this case, the system uses the K-Means algorithm to refine the mask... After separating the inner pixels into clusters A and B, the system does not rely on preset defect brightness information, but instead uses pixels in the immediate vicinity of the mask as a reference. Specifically, the system extracts pixels immediately adjacent to the initial defect mask. For background pixels within a ring-shaped region outside the boundary, calculate the grayscale or color channel statistical characteristics of these background pixels, such as their grayscale histogram. Subsequently, the system calculates the grayscale histograms of pixels within clusters A and B, respectively. and ; through calculation and Between, and and The statistical distance between them can be measured using metrics such as Bach distance or KL divergence, and compared with the external background histogram. The cluster with the closer pixel distance is identified as the background contamination pixel group; if calculated... and If so, the system determines that cluster B is a group of background contaminated pixels and generates a purified defect mask based on cluster A. This procedure provides a data-driven identification method.

[0047] To enable the intensity attenuation factor The generated defect samples can correspond to specific sensing intensities or detection difficulty levels, and an offline calibration process can be performed to establish... The process involves establishing a relationship between the intensity attenuation value and objective image metrics. First, a set of typical defect samples with different original contrast or visibility is selected. These samples are then categorized into several levels, such as high visibility, medium visibility, low visibility, or extreme visibility, either through manual evaluation or based on the performance of existing detection models. For representative defect samples at each level, the image data enhancement processing method of this invention is applied, systematically changing the intensity attenuation factor while keeping other steps unchanged. The value can be incremented from 0.05 to 1.0, with a step size of 0.05, and the defect signal-to-noise ratio (DSNR) or other relevant gradient strength metrics are calculated for each generated enhanced image; by recording different... The corresponding DSNR or other index values ​​can be used to construct a lookup table or fit a function curve. When performing batch data augmentation later, the training requirements can be met, such as generating a batch of samples with DSNR between 0.5 and 1.0. This can be done by looking up the lookup table or solving the problem. To determine the required intensity attenuation factor The range of values ​​is determined to achieve targeted control over the strength of the generated defective samples.

[0048] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image data enhancement processing method for industrial vision inspection, characterized by, The method comprises the following steps: obtaining a defect image block in a source image and a defect mask used for identifying the defect image block, obtaining a target background image and a target position used for fusion on the target background image; based on the image features of the defect image block and the target background image at the target position, performing tone correction on the defect image block to obtain a tone-corrected defect image block; calculating the original gradient field of the tone-corrected defect image block within the region defined by the defect mask; setting an intensity attenuation factor greater than 0 and less than 1; performing point-by-point scalar multiplication operation on the original gradient field and the intensity attenuation factor to obtain an attenuated gradient field; solving a gradient domain fusion equation, which is set to satisfy the following two objective constraints simultaneously: condition a, within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to the attenuated gradient field; condition b, at the boundary of the fusion region, the pixel value of the solved image is constrained to be equal to the pixel value of the target background image at the boundary; and output the solution of the gradient domain fusion equation as an enhanced image containing fused defects.

2. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, The step of performing tone correction on the defect image block based on the image features of the defect image block and the target background image at the target position specifically comprises: extracting defect region pixels within the region defined by the defect mask in the defect image block, extracting background region pixels within the region equal to the defect mask in the target position of the target background image, calculating the color histogram of the defect region pixels and the color histogram of the background region pixels, and applying a histogram matching algorithm to match the color histogram of the defect region pixels to the color histogram of the background region pixels to obtain the tone-corrected defect image block.

3. The image data enhancement processing method for industrial vision inspection according to claim 1, characterized in that, The gradient domain fusion equation is a Poisson equation, and the step of solving a gradient domain fusion equation specifically determines the final pixel value within the fusion region by solving the Poisson equation.

4. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, Before the step of performing tone correction on the defect image block, the method further comprises: analyzing the image statistical features of all pixel points within the region covered by the defect mask; based on a clustering algorithm, automatically separating all pixel points into a true defect pixel group and a background pollution pixel group; based on the true defect pixel group, generating a purified defect mask; and using the purified defect mask to perform the subsequent tone correction step and the step of solving a gradient domain fusion equation.

5. The image data enhancement processing method for industrial vision inspection according to claim 4, characterized in that, The clustering algorithm is a K-Means clustering algorithm, and the number of clustering clusters K of the K-Means clustering algorithm is set to 2.

6. The image data enhancement processing method for industrial vision inspection according to claim 1, characterized in that, Before the step of performing tone correction on the defect image block, the method further comprises a context compatibility pre-matching step, which comprises: using a texture description algorithm to extract original context texture features of a neighborhood outside the defect mask of the defect image block; scanning a plurality of candidate target positions on the target background image, and using the texture description algorithm to extract candidate background texture features of each candidate target position; calculating a texture similarity distance between the original context texture features and each candidate background texture feature of the plurality of candidate background texture features; and only when the texture similarity distance is less than a preset context compatibility threshold, the candidate target position is determined as the target position for performing subsequent steps.

7. The image data enhancement processing method for industrial vision inspection according to claim 6, characterized in that, The texture description algorithm is a local binary pattern algorithm, and the texture similarity distance is a chi-square distance.

8. The image data enhancement processing method for industrial vision inspection according to claim 1, characterized in that, The step of setting an intensity attenuation factor and the step of performing point-by-point scalar multiplication of the original gradient field and the intensity attenuation factor are modified as follows: based on the internal image features of the defect image block after tone correction, a spatial attenuation map is generated, the spatial attenuation map is a matrix equal in size to the defect mask, and each element value in the matrix of the spatial attenuation map is greater than 0 and less than 1; and the original gradient field is multiplied by the spatial attenuation map point by point to obtain an attenuated gradient field.

9. The image data enhancement processing method for industrial visual inspection according to claim 8, wherein, Spatial attenuation map The generation rule comprises: firstly calculating the minimum distance from the defect mask boundary of each point in the fusion region ; then, calculating the numerical value of according to the minimum distance , and the calculation follows the following rules: , wherein, is a preset minimum attenuation value greater than 0, is a preset distance attenuation coefficient.

10. The image data enhancement processing method for industrial vision inspection according to claim 1, characterized in that, The condition a in the two objective constraint conditions that the gradient domain fusion equation is required to satisfy is modified as follows: within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to a hybrid gradient field, and the hybrid gradient field selects the attenuated gradient field and the gradient of the target background image at the corresponding point with a larger amplitude.

Images