An image data enhancement processing method for industrial visual inspection

Abstract

The application relates to the technical field of image data processing, and discloses an image data enhancement processing method for industrial visual detection, which comprises the following steps: performing tone correction on a defect image block, calculating an original gradient field, multiplying the original gradient field by an intensity attenuation factor greater than 0 and less than 1 to obtain an attenuated gradient field, and finally solving a gradient domain fusion equation, taking the attenuated gradient field as internal guidance, and making the boundary of the attenuated gradient field seamlessly fused with a target background image. By introducing the intensity attenuation factor, a defect perception intensity regulation mechanism is established, so that a defect sample with weak signal can be generated as needed to train a model to cope with a low signal-to-noise ratio detection scene, and the detection sensitivity of the model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to an image data enhancement processing method for industrial visual inspection, belonging to the technical field of image data processing. BACKGROUND

[0002] In the current application of image data processing and generation, especially in the industrial visual inspection system relying on deep learning, the quality and diversity of the training data set are one of the factors determining the performance of the system. However, in many industrial scenarios, especially when dealing with materials with complex texture surfaces, defect samples are scarce. Therefore, artificially synthesizing high-quality training samples through image data enhancement has become a conventional and necessary technical means in the field. A common image enhancement method is to superimpose a known defect image block onto a defect-free target background image to expand the data set. However, this synthetic method has an inherent technical limitation in image processing mechanism: simple pixel value superposition, such as transparency mixing, cannot handle the differences in lighting, color tone, and texture structure between the defect and the background. Therefore, it will inevitably produce visually unrealistic hard edges or blurred halos at the junction of the two, which are high-frequency statistical false features. These features will be captured by the deep learning network during the model training process and incorrectly identified as the core criterion for defects.

[0003] The cognitive bias introduced by the data generation mode makes the model miss the subtle defects that do not exist in the real world during actual detection, and easily misreport any normal but dramatic texture or light change in the defect-free background as a defect because it is statistically similar to the artifact, resulting in serious false positives. The existing multi-modal data processing and fusion system also has fundamental deficiencies in real-time performance, adaptability and continuous optimization capability at the system level architecture. For example, the Chinese invention patent with publication number CN119444686A discloses an industrial visual detection system based on multi-modal fusion. Although the system proposes an architecture including data acquisition and calibration, scene image capture, visual information synthesis, algorithm optimization and training modules, and embeds a deep learning image analysis algorithm in the visual information synthesis module, in the core data processing link, it mainly relies on traditional image alignment means such as feature point detection, description extraction and transformation matrix to ensure spatial consistency. This alignment method based on fixed features lacks robustness for dynamic, non-rigid or uneven textured industrial product scenes. Once the feature point extraction or matching fails, the precision of the fusion result will decrease sharply, seriously affecting the reliability of the final detection report. In addition, the system only mentions defining the cost function and adjusting the weight in the training optimization, and lacks a deep feedback and iterative optimization mechanism based on objective quality evaluation indicators (such as PSNR or SSIM) for the whole process from preprocessing, alignment to fusion operation of the multi-modal synthesis model.

[0004] Therefore, how to provide an image data enhancement processing method that is no longer a simple application of a single tool, but can systematically and sequentially solve the complex problems of context incompatibility, input source pollution, global brightness difference and local boundary discontinuity in image synthesis, and realize controllable generation of defect intensity, has become a technical problem to be solved by the present application. SUMMARY

[0005] The present application provides an image data enhancement processing method for industrial visual detection, which mainly aims to solve the complex problems of context incompatibility, input source pollution, global brightness difference and local boundary discontinuity in image synthesis, and realize controllable generation of defect intensity.

[0006] To achieve the above-mentioned purpose, the present application provides an image data enhancement processing method for industrial visual detection, comprising the following steps:

[0007] Obtaining a defect image block in a source image and a defect mask for identifying the defect image block, obtaining a target background image and a target position for fusion on the target background image;

[0008] based on the image features of the defect image block and the target background image at the target position, tone correction is performed on the defect image block to obtain a tone-corrected defect image block; an original gradient field of the tone-corrected defect image block in the region defined by the defect mask is calculated; an intensity decay factor greater than 0 and less than 1 is set; and a point-by-point scalar multiplication operation is performed on the original gradient field and the intensity decay factor to obtain an attenuated gradient field;

[0009] a gradient domain fusion equation is solved, the equation is set to simultaneously satisfy the following two objective constraints: condition a, the gradient field of the solved image is constrained to be equal to the attenuated gradient field inside the fusion region defined by the defect mask; condition b, the pixel value of the solved image is constrained to be equal to the pixel value of the target background image at the boundary of the fusion region; and the solving result of the gradient domain fusion equation is output as an enhanced image containing the fused defect.

[0010] Preferably, the step of performing tone correction on the defect image block based on the image features of the defect image block and the target background image at the target position specifically includes: extracting defect region pixels of the defect image block in the region defined by the defect mask, extracting background region pixels of the target background image in the region equal to the defect mask at the target position, calculating the color histogram of the defect region pixels and the color histogram of the background region pixels, and applying a histogram matching algorithm to match the color histogram of the defect region pixels to the color histogram of the background region pixels to obtain the tone-corrected defect image block.

[0011] Preferably, the gradient domain fusion equation is a Poisson equation, and the step of solving a gradient domain fusion equation specifically includes determining the final pixel value inside the fusion region by solving the Poisson equation.

[0012] Preferably, before the step of performing tone correction on the defect image block, the method further includes: analyzing the image statistical features of all pixel points in the region covered by the defect mask; based on a clustering algorithm, automatically separating all pixel points into a true defect pixel group and a background pollution pixel group; based on the true defect pixel group, generating a purified defect mask; and using the purified defect mask to perform the subsequent tone correction step and the step of solving a gradient domain fusion equation.

[0013] Preferably, the clustering algorithm is a K-Means clustering algorithm, and the number of clustering clusters K of the K-Means clustering algorithm is set to 2.

[0014] Preferably, before the step of performing tone correction on the defect image block, the method further comprises a context compatibility pre-matching step, which comprises: using a texture description algorithm to extract original context texture features of a neighborhood outside the defect mask of the defect image block; scanning a plurality of candidate target positions on the target background image, and using the texture description algorithm to extract candidate background texture features of each candidate target position; calculating a texture similarity distance between the original context texture features and each candidate background texture feature of the plurality of candidate background texture features; and only when the texture similarity distance is less than a preset context compatibility threshold, the candidate target position is determined as the target position for performing the subsequent steps.

[0015] Preferably, the texture description algorithm is a local binary pattern algorithm, and the texture similarity distance is a chi-square distance.

[0016] Preferably, the step of setting an intensity decay factor and the step of performing point-by-point scalar multiplication of the original gradient field and the intensity decay factor are modified as follows: based on the internal image features of the defect image block after tone correction, a spatial decay map is generated, the spatial decay map is a matrix equal in size to the defect mask, and each element value in the matrix of the spatial decay map is greater than 0 and less than 1; and the original gradient field is multiplied by the spatial decay map point by point to obtain the decayed gradient field.

[0017] Preferably, the spatial decay map is generated according to the following rules: first, the minimum distance from each point in the fusion region to the boundary of the defect mask is calculated; then, the value of is calculated according to the minimum distance , and the calculation follows the following rules: , wherein is a preset minimum decay value greater than 0, is a preset distance decay coefficient.

[0018] Preferably, the condition a in the two objective constraint conditions that the gradient domain fusion equation is required to satisfy is modified as follows: within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to a hybrid gradient field, and the hybrid gradient field selects the decayed gradient field and the gradient of the target background image at the corresponding point with a larger amplitude.

[0019] Compared with the prior art, the present application has the following advantages:

[0020] 1、The method establishes a sub-domain processing flow of image data, which first corrects the overall illumination and color difference between the defective image block and the target background in the global statistical feature domain through histogram matching, providing a unified brightness basis for subsequent fusion; then, in the local differential feature domain, gradient domain fusion equation is used for solving, this process focuses on maintaining the internal gradient structure of the defect, while making the boundary pixels of the defect naturally transition to the background. This order of first assimilating global features and then solving local features avoids the problem of gradient solver failure caused by sudden changes in illumination, and also avoids the texture blur caused by separate color correction, so that the generated image maintains the brightness continuity at the boundary and the internal texture authenticity.

[0021] 2、By eliminating boundary artifacts, the generated enhanced image has visual realism. In the training process, the deep learning model no longer learns false features at the fusion joint as defect criteria, and the model's attention is directed to the internal structure and texture features of the defect sample, which makes the model less likely to produce false positives for normal severe texture changes such as wrinkles or color differences in the defect-free background, and less likely to produce missed detections for real defects that have been fused into the background.

[0022] 3、The method also establishes a multi-dimensional image data generation and verification process. Before performing fusion, context compatibility pre-matching based on texture descriptors is introduced to preclude fusion positions where the defect texture and background texture are not logically compatible, avoiding the generation of logical artifacts. On this basis, clustering algorithm is further used to statistically separate pixels within the mask, automatically purifying the defect input source and avoiding ghost artifacts caused by background pollution. Finally, this input, which has been purified in terms of context and content, is processed by the core color correction and gradient domain fusion steps. This layer-by-layer processing from logical rationality, content purity to boundary authenticity ensures that the generated enhanced image is close to the real physical world in terms of semantic content and optical dimensions. BRIEF DESCRIPTION OF DRAWINGS

[0023] Fig. 1 Fig. 1 is a flowchart of the multi-stage image data enhancement processing method of the present application;

[0024] Fig. 2 Fig. 4 is a DSNR and DSNR relationship curve diagram of the defect signal intensity regulation effect of the present application;

[0025] Fig. 3 Fig. 6 is an application architecture diagram of the data enhancement method of the present application in an industrial detection closed-loop system. DETAILED DESCRIPTION

[0026] ​To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be noted that the specific embodiments and their descriptions herein are merely for explaining the invention and are not intended to limit the scope of protection of the invention.

[0027] This invention provides an image data enhancement processing method for industrial visual inspection. The method constructs a multi-stage image data processing flow, aiming to generate visually seamlessly fused image data with controllable defect intensity, thereby improving the performance of deep learning models in industrial inspection applications. This image data processing flow can be executed by a computing device configured with a general-purpose processor and standard image processing libraries. Its core stages may include: an optional context-compatible pre-matching stage, an optional defect mask purification stage, a global brightness correction stage, a defect signal intensity modulation stage, and a local gradient domain fusion stage. In one specific embodiment, the execution of this image data enhancement processing method begins with data acquisition and preprocessing. The system first acquires defect image patches from a source image. And a binarized defect mask for accurately identifying defect regions in defective image patches. Simultaneously, the system acquires one or more defect-free target background images. Given that in industrial applications, blindly blending a defect in one texture, such as a broken thread in fabric, into another completely unrelated texture, such as a brushed metal surface, can produce logical artifacts that mislead subsequent model training, this method preferably initiates a context-compatible pre-matching step before performing any pixel processing. This step is used to match the target background image... The system first selects compatible target locations based on texture compatibility. The specific procedure is as follows: First, the system employs a well-known texture description algorithm, specifically Local Binary Pattern (LBP) algorithm, to analyze defective image patches. Its defect mask The external neighborhood, that is, the original context in which the defect is located, is analyzed to extract and calculate an original contextual texture feature vector. Next, the system analyzes the target background image. Up scan multiple candidate target locations And using the exact same LBP algorithm, the location of each candidate target is extracted. Candidate background texture features of the corresponding region Subsequently, the system calculates... With each The texture similarity distance between the two textures can be quantized using the chi-square distance, a standard technique in this field. Finally, the system compares this distance with a preset context compatibility threshold. To compare, this threshold may be determined by an offline calibration procedure, for example, by computing the chi-square distance of a set of known compatible and incompatible texture pairs, and choosing a numerical point that can best distinguish the two categories as ; the system finally only selects those candidate target positions with texture similarity distance less than as the target positions for performing the subsequent fusion step, thus ensuring the logical consistency of the texture of the synthesized image at the source of the processing.

[0028] After the context-compatible target positions are determined, considering that the defect mask provided by the operator in the actual acquisition often contains background pollution around the defects, which will interfere with the subsequent brightness correction and produce ghost artifacts, the method preferably performs a defect mask purification step, which uses a clustering algorithm, such as the K-Means clustering algorithm, and determines the number of clusters as , whose procedure is as follows: the system first extracts all the pixel points in the area covered by the defect mask , and uses the image statistical features of these pixel points, such as the pixel values of the R, G, and B channels or the gray values, as the input of the K-Means algorithm; after the algorithm is executed, all the pixel points are automatically separated into a true defect pixel group and a background pollution pixel group; the system automatically identifies the background pollution pixel group through a deterministic rule, such as judging which group has more pixel quantity or which group has an average pixel value closer to the background pixel value outside the mask neighborhood; finally, the system generates a purified defect mask containing only the true defect area based on the true defect pixel group, and uses this to perform all subsequent processing steps; next, the system enters the global brightness correction phase to solve the illumination and tone differences between the defect image block and the target background, which is the brightness basis for achieving seamless fusion, whose procedure is as follows: the system performs tone correction on the defect image block based on the image features of the defect image block and the target background image at the determined target position; specifically, this step uses the histogram matching algorithm known in the art; the system first extracts the defect area pixels within the area defined by the purified defect mask , and calculates its color histogram ; at the same time, the system extracts the background area pixels within the target position and , and calculates its color histogram ; the histogram matching algorithm then establishes a pixel value mapping relationship by calculating the cumulative distribution functions, i.e., CDFs, of each other, such as and ; the system then uses the mapping relationship to correct the pixel values of the defect image block , and uses the corrected defect image block to perform the subsequent fusion step. Match to Applying this mapping relationship, the system obtains a color-corrected defective image patch. The image patch is consistent with the target background area in terms of global illumination and tone.

[0029] Subsequently, in order to generate samples on demand for training the model to cope with low signal-to-noise ratio detection scenarios, this method introduces a signal intensity modulation mechanism. This mechanism is executed in the differential domain, i.e., the gradient domain, rather than the pixel domain, and its procedure is as follows: First, the system calculates the defect image patch after tone correction. In the purified defect mask The original gradient field within the defined region This gradient field It is a vector field that represents the texture and structural information inside the defect, which can be obtained by analyzing... The gradient is obtained by applying standard gradient operators, such as the Sobel operator; then, the system sets an intensity decay factor. This factor It is determined to be a scalar value greater than 0 and less than 1, for example, it can be specified by the user as 0.2, or... Random sampling is performed within the interval to increase data diversity; then, the system applies the original gradient field. With intensity attenuation factor Perform point-by-point scalar multiplication. This yields a decayed gradient field. ;this The gradient direction of the original defect is preserved, but its gradient magnitude is reduced proportionally. Finally, the system performs a gradient domain seamless fusion step to solve the boundary artifact problem caused by direct pixel domain superposition. The purpose of this step is to achieve a continuous transition between the defect boundary and the target background in a differential sense while maintaining the attenuated internal structure of the defect. The procedure is as follows: The system solves a gradient domain fusion equation, which can be specifically a Poisson equation in the relevant field, and determines the final pixel value inside the fusion region by solving the Poisson equation. The equation is set to simultaneously satisfy the following two objective constraints: Step a, in the purified defect mask Within the defined fusion region, the solved image The gradient field is constrained to be equal to the decayed gradient field. ,Right now This constraint guides the fusion processor to reconstruct the internal texture of defects with controlled intensity; in step B, at the boundary of the fusion region, the solved image... The pixel values ​​are constrained to be equal to the target background image. The pixel value at the boundary, i.e. This constraint is used to ensure the seamless continuity of the boundary pixel values; the system solves the Poisson equation by standard numerical methods, such as finite difference method combined with iterative solvers, and finally outputs the solution , as a seamless, intensity-controllable enhanced image.

[0030] In the case of no conflict, the present application also provides alternative embodiments of the above processing flow, for example, the step of regulating the intensity of defects can be replaced by a spatially variable manner: the system can generate a spatial decay map of the same size as the defect mask , where each element value is greater than 0 and less than 1; the generation rule of this map may depend on the geometric features, for example, first calculate the minimum distance from each point in the fusion region to the mask boundary , then calculate the value of according to the distance , which follows a deterministic rule, for example , where is a preset minimum decay value greater than 0, is a preset distance decay coefficient; at this time, the decayed gradient field is obtained by point-by-point multiplication operation of , this way allows to generate defect signals with different center and edge intensities; in a specific embodiment, when using the spatial decay map to regulate the signal, the engineering calibration procedure of its internal parameters and includes: first, according to the signal-to-noise ratio requirement of the weakest defect for specific detection tasks, set a minimum decay value , for example, set it to 0.05 or 0.1; then, the system analyzes the geometric shape of the purified defect mask , calculates the distance from all pixel points in the mask to its nearest boundary, and determines the maximum distance value ; finally, the value of the distance decay coefficient is determined by the calculation formula of , this way makes the gradient decay intensity of the defect core region farthest from the mask boundary converge to the set , thus providing a deterministic parameter setting basis for this spatially variable decay method; in addition, the internal constraint condition of the gradient domain fusion equation, i.e. the aforementioned step a, can also be modified: within the fusion region, the gradient field of the image to be solved can be constrained to be equal to a hybrid gradient field; this hybrid gradient field is a gradient selection mechanism, which at each point within the fusion region, respectively calculates the decayed gradient field The gradient magnitude and the target background image gradient magnitude at the corresponding point The gradient vector with the larger magnitude is preferentially selected as the guiding gradient at that point. This hybrid approach allows for the fusion of defective textures while preserving the original texture features of the target background, resulting in a transparent overlay visual effect. In another specific implementation, when an iterative solver (e.g., the Gauss-Seidel method) is used to solve the gradient domain fusion equation, the convergence condition for terminating the iteration is set as follows: the entire fused image generated in the next iteration... Compared with the previous iteration result Between these points, the sum of squared differences (SSD) of all corresponding pixel values ​​is less than a preset convergence threshold. (For example Desirable or The iteration terminates and outputs the current result when the total number of iterations reaches a preset upper limit (e.g., 5000); and when the internal constraints are modified to use a hybrid gradient field, the hybrid gradient... At every point within the integration area The vector value selection procedure at each point is as follows: calculate the decayed gradient vector respectively. The L2 norm (i.e., gradient magnitude) and the gradient vector of the target background image at the corresponding point. The L2 norm of the point is calculated, and the vector with the larger L2 norm value is selected as the point. The final gradient vector used to solve the constraints. .

[0031] Example 1: In a specific industrial visual inspection application scenario, the goal is to train a deep learning model to detect subtle scratches on the surface of high-value brushed metal, which are on the edge of detection. Such samples are scarce in real production lines, and their signal characteristics are statistically similar to the brushed texture of the background material itself. Directly collected sample libraries cannot meet the model training requirements for low signal-to-noise ratio scenarios. To generate such data, the system first acquires a source image containing typical but relatively high-contrast scratches. And a defect mask whose area is larger than the actual defect, drawn by the operator. Simultaneously, the system acquires a batch of defect-free target background images with different lighting conditions, but all with acceptable textures. The method first performs a context-compatible pre-matching step, using the Local Binary Pattern (LBP) algorithm to extract defective image patches. Its defect mask Original contextual texture features of the outer neighborhood This feature characterizes the texture orientation and density of the original brushed background; the system then applies this feature to the target background image. Up scan multiple candidate target locations Extract its candidate background texture features By calculating the chi-square distance, the system only filters out those... and Distance less than the preset context compatibility threshold Location This procedure ensures that the target location and the defect source have a consistent brushed texture context for subsequent fusion, helping to avoid logical artifacts caused by texture mismatch; then, the system applies the input defect mask... Perform the defect mask purification step, because The data may contain background contamination pixels from the original background. Without purification, the statistical characteristics of these pixels may interfere with subsequent tone correction, and their gradient information may also produce ghost artifacts during fusion. Therefore, the system employs the K-Means clustering algorithm to... All pixels under the coverage are separated into two clusters: a group of true defective pixels and a group of background contamination pixels, and the number of clusters is calculated. Set as A purified defect mask is generated based on the true defect pixel group. This purified defect mask This was then used in the global brightness correction stage, where the system applied a histogram matching algorithm to identify defective image blocks. exist The color histogram of defective pixels within the region is matched to the target background image. At the selected target location The color histogram of the background pixels in the corresponding area is used to output the defect image patch after tone correction. This brightness processing provides a unified brightness basis for subsequent gradient domain fusion, allowing the fusion solver to focus on gradient continuity and reduce the need to handle drastic lighting changes.

[0032] Before fusion, in order to achieve the goal of generating low signal strength defect samples, the system activated a defect signal strength modulation mechanism. The system first calculated... exist Original gradient field within the region Subsequently, the system sets an intensity attenuation factor. ,For example Then, perform pointwise scalar multiplication to obtain a decayed gradient field. ;this The structural information of the scratches was preserved, but their gradient magnitude was reduced; finally, the system solved a gradient domain fusion equation based on the Poisson equation, the solution of which was set to simultaneously satisfy two objective constraints: first, in Within the region, the image to be solved The gradient field is constrained to be equal to (Right now Secondly, in At the boundary, the solved image The pixel values ​​are constrained to be equal to the target background image. The pixel value at that boundary (i.e. The enhanced image output by solving this equation has its scratch defect signal intensity controlled at a low level, while the defect boundary and the brushed background image achieve a continuous gradient transition, avoiding the boundary abruptness that may be caused by direct superposition of pixel values. Through the application of this series of image data processing steps, this method transforms an original sample that may have context incompatibility risks, mask containing background pollution and signal intensity that does not meet the requirements into an enhanced image data with consistent context logic, purified content, unified lighting, smooth boundaries and signal intensity that meets the training objectives. It is suitable for model training in low signal-to-noise ratio scenarios.

[0033] Example 2: To objectively verify the practical effect of an image data enhancement processing method for industrial visual inspection, particularly its ability to eliminate boundary artifacts and control defect intensity, the following comparative experiment was designed and executed. The purpose of the experiment was to quantitatively evaluate the differences between the enhanced image generated by the method of this invention and the image generated by the traditional pixel overlay method in terms of boundary realism and defect signal intensity control. The experimental platform was configured as a general-purpose computer equipped with standard image processing libraries such as OpenCV 4.5 and numerical computing capabilities, with an Intel Core i7-10700 processor, 16GB of memory, and a Windows 10 operating system. The data sources used in the experiment included two types: one type was the source images, which included representative defect images collected from industrial production lines. The image patches and their corresponding pixel-level binary defect masks were selected, specifically two typical defects: fabric yarn breakage defects (source image resolution 1024x1024 pixels, 8-bit grayscale) and metal surface scratch defects (source image resolution 512x512 pixels, 8-bit grayscale). Another type was the target background image, which selected fabric background images (resolution 2048x2048 pixels) and metal wire drawing background images (resolution 2048x2048 pixels) that matched the texture of the defect source but were defect-free. All images were acquired under uniform industrial line scan light source conditions to reduce initial interference caused by inconsistent lighting. To simulate the noise effect in engineering reality, Gaussian white noise with a mean of 0 and a standard deviation of 5 grayscale values ​​was superimposed on the target background images of some experimental groups.

[0034] The test sets two main groups: a control group and an inventive group. The control group generates an enhanced image using the transparency blending method commonly used in the art, which sets the blending process to directly superimpose the defect image block (without tone correction) onto the specified position of the target background image, and sets the blending transparency factor to 0.8, which simulates simple pixel value superposition. The inventive group generates an enhanced image strictly according to the complete process disclosed in the foregoing embodiments, including optional context compatibility pre-matching (it is assumed that the test has been filtered and a texture-compatible position has been selected), defect mask purification (K-Means clustering is used, K = 2), global brightness correction based on histogram matching, and gradient domain fusion based on the solution of the Poisson equation. To verify the defect intensity regulation mechanism, the inventive group is further subdivided into multiple subgroups, each of which sets a different intensity attenuation factor , the specific values are (no attenuation), , and ; To quantitatively evaluate the quality of the generated image, two core indicators are defined: one is the boundary artifact measure BAM, which measures the smooth transition degree of the boundary by calculating the root mean square value of the gradient amplitude difference between the image blocks in the neighborhood (width of 5 pixels) inside and outside the boundary of the fusion region. The smaller the BAM value, the smoother the boundary and the less obvious the artifacts. The second is the defect signal-to-noise ratio DSNR, which is defined as the ratio of the average gradient amplitude in the defect region to the average gradient amplitude in the background region, and is used to quantify the recognizability or intensity of the defect signal. In the test, for each defect type and background combination, 10 enhanced images are generated under each test group (control group and each subgroup of the invention), and the average values of each indicator are calculated. The test process and data results are as follows, see Table 1, which shows some key data examples of enhanced images generated for fabric broken yarn defects on fabric backgrounds. The original input represents the state before direct superposition, and the intermediate feature selects the difference between the average gray value of the defect region after tone correction and the average gray value of the background region to reflect the brightness correction effect, and the final output is the calculated BAM and DSNR indicators.

[0035] Table 1: Example of fabric broken yarn defect enhanced image quality evaluation data table.

[0036]

[0037] As can be seen from Table 1 and similar test data for metal scratch defects, the control group does not perform tone correction The image generated by directly stacking pixels (with high BAM values) exhibits abrupt gradient changes at the boundaries, resulting in generally high BAM values ​​(38.5). Even after adding noise, the BAM value further increases (42.1), indicating severe boundary artifacts. In contrast, the sample group of this invention reduces the initial grayscale difference between defects and the background through a global tone correction step. The BAM value was reduced to around 4.1. Subsequently, a gradient domain fusion step was used to maintain the gradient continuity of the fusion boundary. The BAM values ​​of all subgroups were significantly reduced to single digits (e.g., 5.3), and remained at a low level (e.g., 6.8) even in noisy backgrounds. This objectively confirms the effectiveness of the method in eliminating boundary artifacts. Simultaneously, by adjusting the intensity attenuation factor... The defect signal-to-noise ratio (DSNR) of the sample group in this invention exhibits the same characteristics as... The value shows an approximately linear and controllable decreasing trend, from The DSNR was 6.5 at that time, and gradually decreased to The DSNR was 0.7, which clearly demonstrates that the signal strength modulation mechanism introduced in this invention can generate defect samples with different visibility as needed, including low signal-to-noise ratio samples (DSNR < 1.0) close to the detection limit. Even after adding background noise, the absolute value of DSNR decreases, but it increases with... The controlled trend of change remains. Experimental results show that, compared with the traditional pixel overlay method, the image data enhancement processing method of the present invention for industrial visual inspection can generate enhanced images with smooth boundary transitions and no obvious visual artifacts. Furthermore, the signal intensity of the generated defects can be quantitatively and controllably adjusted by setting an intensity attenuation factor.

[0038] Example 3: This example combines Figs. 1 to 3 This describes an image data enhancement processing method for industrial visual inspection, such as... Fig. 1 As shown, it receives defect image blocks, defect masks, and target background images as input, and sequentially performs the following steps: an optional context compatibility pre-matching step to use a texture description algorithm to filter texture-compatible target locations and avoid logical artifacts; an optional defect mask purification step to automatically separate true defect pixel groups based on a clustering algorithm such as K-Means with K=2 to avoid ghost artifacts; global brightness correction to solve global illumination and color differences based on a histogram matching algorithm; and the calculation of the original gradient field based on the corrected image. Then, the original gradient field is multiplied by an intensity attenuation factor greater than 0 and less than 1 through a defect signal intensity modulation step to obtain an attenuated gradient field. Finally, in the local gradient domain fusion step, the gradient domain fusion equation such as the Poisson equation is solved to constrain the internal gradient to be equal to the attenuated gradient field and the boundary to be equal to the target background. The final output is an enhanced image containing fused defects with controllable signals and seamless boundaries.

[0039] like Fig. 2 As shown in the figure, the horizontal axis represents the intensity attenuation factor. The vertical axis represents the defect signal-to-noise ratio (DSNR). The figure includes two curves: DSNR without noise (solid line) and DSNR with noise (dashed line). As... As the value decreases from 1.0 to 0.1, the DSNR value also decreases steadily, confirming the controllability of this method for defect strength; Fig. 3 As shown, in this architecture, the image acquisition device transmits the raw image to the image database, which is responsible for storing the raw defect image, the target background image, and the generated enhanced image. It also provides training data to the model training server containing the deep learning framework to generate a trained model. This model is deployed in an online detection system containing the trained defect detection model. At the same time, the database also provides the raw data to the data augmentation server. Under the control of the configuration parameters issued by the operator terminal, which includes system management and parameter configuration, the latter generates enhanced images and stores them back in the database through its internal image data augmentation processing modules, such as the context compatibility pre-matching module, the defect mask purification module, and the gradient domain fusion module. The detection results of the online detection system are then fed back to the operator terminal, forming a complete deployment and application closed loop.

[0040] Example 4: To further illustrate the technical advantages of the gradient domain fusion step in the method of the present invention compared to the conventional pixel blending method after only performing global brightness correction, the following comparative experiment was conducted. This comparative experiment simulates a method that only partially adopts the technical path of the present invention. This method includes the same initial steps as the sample group of the present invention in Example 2, including acquiring the same source image. Defect mask (and similarly perform defect mask purification to obtain) ), and target background image Furthermore, a global brightness correction step based on histogram matching is also performed to obtain the defective image patch after tone correction. The key difference in this comparative example is that it omits the gradient domain fusion step in the method of this invention, and instead uses a pixel blending technique, namely alpha transparency blending, to blend the defective image patches after tone correction. Blend into target background image The above setup is used to evaluate the effect achievable by relying solely on global brightness correction without subsequent gradient domain processing; the experimental platform, data source, evaluation metrics (BAM and DSNR), and noise conditions are consistent with Example 2, and the alpha blending transparency factor is set to 0.8 and applied to the tone-corrected image; the data obtained from this comparative experiment is named Comparative Sample Group 1, and compared with the present invention sample group in Example 2 (using... For example) are compared, and the results are shown in Table 2; Table 2 also shows some key data examples of the enhanced images generated for fabric broken yarn defects on the fabric background (with noise);

[0041] Table 2: Example of effect comparison data table of the comparative sample group 1 and the inventive sample group (with noise).

[0042]

[0043] As can be seen from the data in Table 2, the average gray level difference between the defect area and the background area of the comparative sample group 1 is reduced to a lower level (4.3) due to the execution of the global tone correction step, which is consistent with the inventive sample group, indicating that the global brightness difference problem has been handled; however, the value of the boundary artifact metric BAM of the final output of the comparative sample group 1 is 35.8, which is much higher than the 6.8 of the inventive sample group, and is in the same range as the BAM value (42.1) obtained by the control group in Example 2 without tone correction and directly performing Alpha blending; it is observed during the processing that, although the tone tends to be consistent, the image generated by the comparative sample group 1 still has visible hard edges or fuzzy transition zones at the defect boundaries, especially in areas where the defect texture and the background texture have structural differences; at the same time, the DSNR value is 5.8, reflecting the influence of Alpha blending on signal intensity, but failing to solve the problem of boundary smooth transition; it can be seen from the generated image that the hard edge phenomenon also exists in the comparative test of the metal scratch defect, and since the metal surface texture has high contrast, the artifact is more easily observed; it is analyzed that the results of the comparative sample group 1 show that, relying only on global brightness correction, such as histogram matching, even if the correction effect is good, it cannot solve the boundary artifact problem caused by the discontinuity of the local structure and texture of the defect image block and the background image; a conventional pixel blending method, such as Alpha blending, as a weighted average process of pixel values, does not process the local gradient information of the image itself, and thus is prone to gradient mutation at the fusion boundary, forming visual artifacts.

[0044] Example 5: This example describes a standardization engineering calibration procedure for determining the context compatibility threshold and for identifying the background pollution pixel group in the defect mask purification step; for calibrating the context compatibility threshold , a texture image dataset for calibration and validation is first constructed, which contains two categories of image pairs: one category is positive sample pairs, which are composed of multiple pairs of image blocks that are considered texture context compatible, in the target industrial application scenario, such as fabric of the same type and processing batch; the other category is negative sample pairs, which are composed of multiple pairs of image blocks that are considered texture context incompatible, such as fabric and metal, wood and leather; the dataset construction should cover the main texture types and variation ranges that may be encountered in actual applications; for each image pair in the dataset , its LBP texture feature histogram and are calculated respectively, and the chi-square distance between them is calculated; after collecting the chi-square distance set of all positive sample pairs and the chi-square distance set of all negative sample pairs , the receiver operating characteristic curve analysis method can be used to determine the threshold , by evaluating the true positive rate TPR and false positive rate FPR that distinguish positive and negative samples at a series of candidate thresholds, draw the ROC curve, and select the point on the curve that makes TPR higher and FPR lower, which can be the point closest to the upper left corner, or the point that meets the specific application's misjudgment rate requirement, the corresponding chi-square distance value, as the final context compatibility threshold ; in a specific calibration experiment for a textile defect detection scenario, using a dataset containing 500 positive samples, i.e. different regions of the same fabric, and 500 negative samples, i.e. fabric and metal, wood, through the aforementioned ROC analysis method, a value of 0.65 is determined, this threshold can achieve more than 95% accuracy in distinguishing positive and negative samples on this test set.

[0045] For the defect mask purification step, after performing K-Means clustering with K=2, the mask pixels are divided into two clusters, cluster A and cluster B, to establish a procedure for identifying background pollution pixel groups, a method based on combining intra-cluster statistical features and defect prior information can be used; this procedure is based on the phenomenon observed in most industrial defect images, that is, the defect area usually has different gray or color distribution characteristics relative to its surrounding background, such as scratches, stains are usually dark or raised, bright spots are usually bright, and their pixel value distribution standard deviation may also be different; according to this procedure: first, calculate the average gray value and gray value standard deviation of the two pixel clusters A and B respectively; second, obtain or set a prior information about the target detection defect type, that is, whether the expected defect is dark or bright, when processing fabric broken ends or metal scratches, the gray value is usually lower than the background mean; then, based on this prior information, if the expected defect is dark, compare and The cluster with the higher average gray value is identified as the background contamination pixel group; if the expected defect is brighter, the cluster with the lower average gray value is identified as the background contamination pixel group. When prior information is unclear or the defect types are diverse, standard deviation can be introduced as an auxiliary criterion. Generally, the texture consistency of the background area is relatively high, and its pixel value standard deviation is relatively high. The clusters with smaller standard deviations tend to be smaller than the more structurally complex defect regions; therefore, the cluster with the smaller standard deviation can be initially identified as a background contamination pixel group. In an application that processes scratches on a metal surface where a darker color is expected, the average gray level of cluster A is obtained after K-Means clustering. Standard deviation The average gray level of cluster B Standard deviation Due to the expected defects being darker, and ,at the same time Based on this procedure, the system identifies cluster B as the background contaminated pixel group and generates a purified defect mask based on cluster A. .

[0046] Example 6: This example further illustrates a supplementary procedure for automatically identifying background contamination pixel groups when prior information about defect types is insufficient or inapplicable during the defect mask purification step; in this case, the system uses the K-Means algorithm to refine the mask... After separating the inner pixels into clusters A and B, the system does not rely on preset defect brightness information, but instead uses pixels in the immediate vicinity of the mask as a reference. Specifically, the system extracts pixels immediately adjacent to the initial defect mask. For background pixels within a ring-shaped region outside the boundary, calculate the grayscale or color channel statistical characteristics of these background pixels, such as their grayscale histogram. Subsequently, the system calculates the grayscale histograms of pixels within clusters A and B, respectively. and ; through calculation and Between, and and The statistical distance between them can be measured using metrics such as Bach distance or KL divergence, and compared with the external background histogram. The cluster with the closer pixel distance is identified as the background contamination pixel group; if calculated... and If so, the system determines that cluster B is a group of background contaminated pixels and generates a purified defect mask based on cluster A. This procedure provides a data-driven identification method.

[0047] To enable the intensity attenuation factor The generated defect samples can correspond to specific sensing intensities or detection difficulty levels, and an offline calibration process can be performed to establish... The process involves establishing a relationship between the intensity attenuation value and objective image metrics. First, a set of typical defect samples with different original contrast or visibility is selected. These samples are then categorized into several levels, such as high visibility, medium visibility, low visibility, or extreme visibility, either through manual evaluation or based on the performance of existing detection models. For representative defect samples at each level, the image data enhancement processing method of this invention is applied, systematically changing the intensity attenuation factor while keeping other steps unchanged. The value can be increased from 0.05 to 1.0, with a step size of 0.05, and the defect signal-to-noise ratio (DSNR) or other relevant gradient strength metrics are calculated for each generated enhanced image; by recording different... The corresponding DSNR or other index values ​​can be used to construct a lookup table or fit a function curve. When performing batch data augmentation later, the training requirements can be met, such as generating a batch of samples with DSNR between 0.5 and 1.0. This can be done by looking up the lookup table or solving the problem. To determine the required intensity attenuation factor The range of values ​​is determined to achieve targeted control over the intensity of generated defective samples.

[0048] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image data enhancement processing method for industrial visual inspection, characterized in that, Includes the following steps: Obtain a defect image patch from a source image and a defect mask used to identify the defect image patch; obtain a target background image and the target location used for fusion on the target background image. Based on the image features of the defective image patch and the target background image at the target location, the defective image patch is tone-corrected to obtain the tone-corrected defective image patch. Calculate the original gradient field of the defect image patch after tone correction within the region defined by the defect mask; based on the internal image features of the defect image patch after tone correction, generate a spatial attenuation map, which is a matrix of the same size as the defect mask, and each element in the matrix of the spatial attenuation map is greater than 0 and less than 1; and perform point-by-point multiplication operation between the original gradient field and the spatial attenuation map to obtain the attenuated gradient field. Solve a gradient domain fusion equation that satisfies the following two objective constraints: Condition a, within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to the decayed gradient field; Condition b, at the boundary of the fusion region, the pixel value of the solved image is constrained to be equal to the pixel value of the target background image at the boundary. It also outputs the solution results of the gradient domain fusion equation as an enhanced image containing fusion defects; Before performing tone correction on the defective image patch, the method further includes: analyzing the image statistical features of all pixels within the area covered by the defect mask; automatically separating all pixels into a true defect pixel group and a background contamination pixel group based on a clustering algorithm; generating a purified defect mask based on the true defect pixel group; and using the purified defect mask to perform subsequent tone correction steps and solve a gradient domain fusion equation. Before the step of tone correction of the defective image patch, the method also includes a context compatibility pre-matching step, which includes: using a texture description algorithm to extract the original context texture features of the defective image patch in the neighborhood outside its defect mask; scanning multiple candidate target locations on the target background image and using the texture description algorithm to extract the candidate background texture features of each candidate target location; calculating the texture similarity distance between the original context texture features and each of the multiple candidate background texture features; and only when the texture similarity distance is less than a preset context compatibility threshold is the candidate target location determined as the target location for performing subsequent steps.

2. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, The steps for tone correction of the defective image block based on the image features of the defective image block and the target background image at the target location include: extracting defective region pixels of the defective image block within the area defined by the defect mask; extracting background region pixels of the target background image within the same area as the defect mask at the target location; calculating the color histograms of the defective region pixels and the background region pixels; and applying a histogram matching algorithm to match the color histograms of the defective region pixels to the color histograms of the background region pixels to obtain the tone-corrected defective image block.

3. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, The gradient domain fusion equation is the Poisson equation. The steps to solve a gradient domain fusion equation are to determine the final pixel value within the fusion region by solving the Poisson equation.

4. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, The clustering algorithm is the K-Means clustering algorithm, and the number of clusters K in the K-Means clustering algorithm is set to 2.

5. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, The texture description algorithm is a local binary mode algorithm, and the texture similarity distance is the chi-square distance.

6. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, Spatial attenuation map The generation rules include: first, calculating the value of each point within the fusion region. Minimum distance to the boundary of the defect mask Then, based on the minimum distance calculate The numerical value is calculated according to the following rules: ,in, This is a preset minimum attenuation value greater than 0. This is a preset distance attenuation coefficient.

7. The image data enhancement processing method for industrial visual inspection according to claim 1, characterized in that, The gradient domain fusion equation is set to satisfy two objective constraints simultaneously. Condition 'a' is replaced with: within the fusion region defined by the defect mask, the gradient field of the solved image is constrained to be equal to a mixed gradient field. This mixed gradient field is a gradient selection mechanism, which applies to every point within the fusion region. At each location, calculate the decayed gradient field. The gradient magnitude and the target background image gradient magnitude at the corresponding point And select the gradient vector with the larger magnitude of the two as the guiding gradient at that point.

Images