Image-based defect generation method and system
By performing four-point mapping and edge fusion on industrial product images, high-fidelity defect samples are generated, solving the problems of data scarcity and mismatch between synthesized defects, and improving the generalization ability and detection accuracy of the defect detection model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI TECHSENSE CO LTD
- Filing Date
- 2026-04-15
- Publication Date
- 2026-07-14
AI Technical Summary
In the visual inspection of precision industrial products such as chips, the training of defect detection models relies on a large amount of diverse and accurately labeled training data. Data is scarce and costly. Traditional data augmentation methods cannot generate new defects with real physical characteristics. Generative adversarial networks have poor controllability, and the synthesized defects do not match the product outline, resulting in insufficient generalization ability of the detection model.
An image-based defect generation method is adopted. By acquiring the target image and source image of the defect to be synthesized, the geometric transformation matrix is calculated using four pairs of corresponding vertices of the product region to map the defect region to the target image, and pixel fusion operation is performed to ensure a smooth transition between the defect region and the target image. An edge-weighted fusion algorithm is used to improve the geometric fidelity and visual realism of the synthesized sample.
The generated synthetic samples have high fidelity in geometry and visual appearance, which significantly improves the generalization ability and detection accuracy of the defect detection model, solves the problem of data scarcity, and the generated defect samples are closer to real physical defects.
Smart Images
Figure CN122391391A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence image processing technology, and more specifically, to an image-based defect generation method and system. Background Technology
[0002] In visual inspection applications for precision industrial products such as chips and semiconductors, deep learning-based defect detection models are the mainstream technology. However, the training performance of these models is highly dependent on a large amount of diverse and accurately labeled training data. In actual production, real defect samples are often very scarce and unevenly distributed in type, and the cost of obtaining and labeling them is high, which severely restricts the development and performance improvement of detection models.
[0003] To address the problem of data scarcity, the industry has proposed various data augmentation and synthesis techniques. While traditional geometric transformations and noise addition methods can expand the amount of data, they cannot generate new defects with realistic physical characteristics. Methods based on deep learning models such as generative adversarial networks can generate new images, but they are usually poorly controllable, resulting in blurry or unnatural edges on the generated defects, and are difficult to precisely adapt to the contours and perspectives of specific products.
[0004] Other image synthesis techniques, such as those used for panoramic image stitching, calculate transformation matrices by matching common local texture feature points between images and then fusing overlapping areas. However, these methods have significant drawbacks when applied to industrial products such as chips: First, the surfaces of industrial products typically have weak textures or numerous repetitive structures, leading to a severe decrease in the stability and accuracy of general feature point matching algorithms. This can easily result in incorrect geometric alignment, causing positional shifts or morphological distortions in the synthesized defects. Second, simple image overlay or fusion methods that do not adequately consider edge characteristics can create harsh stitching marks between the edges of the synthesized defects and the product background, deviating significantly from the visual characteristics of real defects and thus affecting the generalization ability of the trained model. Summary of the Invention
[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide an image-based defect generation method and system.
[0006] A defect generation method based on an image according to the present invention includes the following steps: Obtain the target image of the defect to be synthesized, the source image containing the source defect, and the defect region in the source image; Product regions are determined on the target image and the source image respectively, and four pairs of corresponding vertices of the product regions are obtained; Based on the four pairs of corresponding vertices, calculate the geometric transformation matrix used to map the source image coordinate system to the target image coordinate system; Using the geometric transformation matrix, a geometric transformation is performed on the defective region to map the defective region onto the product region of the target image; On the target image, a pixel fusion operation is performed on the edges of the mapped defect region to achieve a smooth transition between the defect region and the region in the target image corresponding to the defect region.
[0007] Preferably, the geometric transformation matrix is a homography matrix, and the geometric transformation is a perspective transformation.
[0008] Preferably, the pixel fusion operation is edge-weighted fusion, which includes: A fusion band is defined at the edge of the mapped defect region, and a fusion formula is applied. Calculate the final pixel values of the pixels within the fusion band, where, For the final pixel value, This represents the pixel value of the mapped defect region at that location. The original pixel value of the target image at that location. The fusion weight is determined based on the distance from the pixel to the edge of the mapped defect region.
[0009] Preferably, the step of acquiring the source image includes: The image features of the target image are extracted and similarity calculation is performed with the features of multiple images in a defect library to output one or more candidate source images containing the source defect; One of the one or more candidate source images is selected as the source image.
[0010] Preferably, the method further includes: An interactive editing function is provided, which allows users to manually fine-tune the positions of the four pairs of corresponding vertices and / or to translate, rotate, scale, adjust grayscale or contrast of the mapped defect area.
[0011] An image-based defect generation system according to the present invention includes: The image acquisition module is used to acquire the target image of the defect to be synthesized, the source image containing the source defect, and the defect region in the source image; The product region vertex acquisition module is used to determine the product region on the target image and the source image respectively, and acquire four pairs of corresponding vertices of the product region; The geometric transformation mapping module is used to calculate the geometric transformation matrix based on the four pairs of corresponding vertices, and to perform a geometric transformation on the defect region using the geometric transformation matrix, so as to map the defect region to the product region of the target image; An edge blending module is used to perform pixel blending operations on the edges of the mapped defective region on the target image to achieve a smooth transition between the defective region and the region in the target image corresponding to the defective region.
[0012] Preferably, the geometric transformation mapping module is configured to calculate the homography matrix and perform perspective transformation.
[0013] Preferably, the edge blending module is configured to perform edge-weighted blending, which defines a blending band at the edge of the mapped defect region and follows a blending formula. Calculate the final pixel values of the pixels within the fusion band, where, For the final pixel value, This represents the pixel value of the mapped defect region at that location. The original pixel value of the target image at that location. The fusion weight is determined based on the distance from the pixel to the edge of the mapped defect region.
[0014] Preferably, the system further includes: The similarity matching module is used to obtain the target image from the image acquisition module, extract the image features of the target image, and perform similarity calculation with the features of multiple images in a defect library to output one or more candidate source images containing source defects; Furthermore, the image acquisition module is configured to select one of the one or more candidate source images as the source image.
[0015] Preferably, the system further includes: An interactive editing module is provided to offer an interactive interface that allows users to manually fine-tune the positions of the four pairs of corresponding vertices and / or to translate, rotate, scale, adjust grayscale or contrast of the mapped defect area.
[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This application uses the four vertices of the product outline itself as reference points for geometric transformation, which fundamentally solves the problem of alignment failure or insufficient accuracy caused by weak texture or repetitive structure when relying on general texture feature point matching in the prior art. It ensures that defects can be accurately aligned according to the actual outline and viewpoint of the product, avoids position offset and shape distortion, and thus significantly improves the geometric fidelity of the synthesized sample.
[0017] 2. This application employs edge pixel fusion operations, such as distance-based edge-weighted fusion algorithms, enabling a smooth and natural transition between the edges of the synthesized defects and the background. This eliminates the harsh stitching marks produced by traditional overlay methods, resulting in samples that visually more closely resemble real physical defects. Ultimately, because the generated synthetic samples possess high fidelity in both geometry and visual appearance, they can provide high-quality training data for deep learning models that more closely approximates real-world distributions, thereby effectively improving the generalization ability and detection accuracy of defect detection models. Attached Figure Description
[0018] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the structure of a defect generation system provided in an embodiment of this application; Figure 2 A flowchart illustrating a defect generation method provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the acquisition of product region vertices as provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the edge-weighted fusion principle provided in an embodiment of this application. Detailed Implementation
[0019] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.
[0020] Example 1 This embodiment provides a specific implementation of a defect generation method and system based on four-point mapping and edge fusion. This solution is particularly suitable for defect data enhancement of precision industrial products with regular or semi-regular contours, such as semiconductor chips.
[0021] Figure 1 This is a schematic diagram of the structure of a defect generation system provided in an embodiment of this application. Figure 1 As shown, the system can be implemented as a software system running on a general-purpose computer, server, or dedicated industrial computer. It includes a data loading module 10, a product region vertex acquisition module 20, a perspective transformation mapping module 30, an edge weighted fusion module 40, and a composite image output module 50. It is understood that these modules can be software functional units executed via processor calls.
[0022] Figure 2 This is a flowchart illustrating a defect generation method provided in an embodiment of this application. The following will be combined with... Figure 1 and Figure 2 and refer to Figure 3 and Figure 4 The specific process of this embodiment will be clearly and completely described.
[0023] The subject executing this method can be Figure 1 The defect generation system is illustrated. First, in the image acquisition step S101, the system performs a data import operation through the data loading module 10. Specifically, the operator can select and load a target image 100 to be synthesized with defects from local storage, a network file system, or a database via the user interface provided by the system. This target image 100 is typically a product image that has been judged to be qualified by quality inspection and does not contain defects; it can also be referred to as an "OK image" in this document. At the same time, the system loads a source image 200 from a pre-built defect image library. This source image 200 is a product image of the same type containing real defects; it can also be referred to as an "NG image." In addition to the image itself, the system also loads the annotation information associated with the source image 200. This annotation information precisely defines the location, shape, and category of the defect area in the source image 200; for example, it can be a binarized mask image or a polygon file containing contour point coordinates.
[0024] Subsequently, step S102, which involves acquiring four pairs of corresponding vertices, is performed by the product region vertex acquisition module 20. This step is crucial for achieving accurate geometric alignment later on. Figure 3 The process of vertex acquisition is illustrated in detail. Operators interact with the system interface, targeting both the target image 100 and the source image 200. Specifically, the user roughly selects the target product region 110 in the target image 100 using interactive methods such as mouse dragging. Upon receiving the selection command, the product region vertex acquisition module 20 automatically executes an image processing algorithm (e.g., calculating the minimum bounding rectangle of the selected region) and uses the four corner points of this rectangle as target vertices ptok1, ptok2, ptok3, and ptok4. It should be noted that the order of these four vertices is predefined, for example, top left, top right, bottom right, and bottom left. Using a similar method, the user selects the source product region 210 in the source image 200, and the system similarly calculates its corresponding four source vertices ptng1, ptng2, ptng3, and ptng4. Since the products in the two images are of the same type, these four pairs of vertices (ptok1 and ptng1, ptok2 and ptng2, ptok3 and ptng3, ptok4 and ptng4) correspond one-to-one in terms of the physical structure of the products.
[0025] As a preferred implementation, to further improve vertex accuracy, this embodiment also provides an interactive editing function. After the system automatically calculates the initial coordinates of the eight vertices, it will visually display these vertices on the image, for example, marked with small circles or crosses. Operators can check whether these vertices accurately fall on the feature points of the product outline (such as the four corners of a chip). If there is a deviation, the user can directly drag a vertex marker to a more accurate position using the mouse. Correspondingly, the product area vertex acquisition module 20 will update the coordinates of the vertex in real time. This human-machine combination approach balances the efficiency of automation with the accuracy of manual calibration.
[0026] After accurately determining the four pairs of corresponding vertices, the process proceeds to step S103, which calculates the homography matrix, and step S104, which are executed by the perspective transformation mapping module 30. The homography matrix is a 3x3 matrix used to describe the projective transformation relationship between two planes. In this embodiment, it can accurately describe how the product plane in the source image 200 is transformed to the product plane in the target image 100. This transformation can simultaneously handle translation, rotation, scaling, and perspective distortion. The perspective transformation mapping module 30 uses the coordinates of the four pairs of vertices obtained in step S102 as input and calls a numerical calculation method (e.g., a direct linear transformation algorithm) to solve for the homography matrix H.
[0027] After obtaining the homography matrix H, the perspective transformation mapping module 30 uses this matrix to perform a geometric transformation on the defect region in the source image 200. Specifically, for each pixel in the defect region of the source image 200, its coordinates (x_ng, y_ng) can be mapped to new coordinates (x_ok, y_ok) = (x_ok' / w', y_ok' / w') in the target image coordinate system through matrix multiplication (x_ok', y_ok', w')^T = H*(x_ng, y_ng, 1)^T. By performing this perspective transformation on all pixels of the defect region and its annotation information (such as the mask image), the system can generate a "transformed defect region" that perfectly matches the product posture in the target image 100 in shape, orientation, and position. This process ensures that even if there are slight differences in the shooting angle and distance between the source image and the target image, the synthesized defect can perfectly fit the product contour, thereby avoiding shape distortion problems that may be caused by simple translation or affine transformation.
[0028] After the defect is accurately mapped to the corresponding position in the target image 100, directly overlaying the pixels of the transformed defect area onto the target image will produce harsh and abrupt stitching marks at the defect edges. To solve this problem, this embodiment performs an edge-weighted fusion step S105, which is executed by the edge-weighted fusion module 40.
[0029] Figure 4The principle of edge-weighted fusion is illustrated. The edge-weighted fusion module 40 first extends a certain number of pixels inwards and outwards (e.g., 5 pixels) around the contour line of the transformed defect region, thus defining a ring-shaped fusion band 330. This fusion band 330 covers the outermost portion of the defect region 310 and a portion of the adjacent background region 320. For each pixel within the fusion band 330, the system calculates its final pixel value according to the following fusion formula. :
[0030] in, It is the pixel value (e.g., grayscale value or RGB color vector) of the defect area at that pixel location after transformation. It is the original pixel value of the target image 100 at that pixel location. This is a fusion weight with a value ranging from [0, 1], and it is key to achieving a smooth transition. This fusion weight... The distance is determined based on the distance from the current pixel to the original edge of the transformed defect region. Specifically, a distance transformation algorithm can be used to calculate the distance d from each pixel within the fusion band 330 to the nearest edge point. Then, this distance is normalized so that: when the pixel is located on the original edge of the defect region, α=1. = The pixel information of the defect is completely preserved; when the pixel is located at the outermost boundary of the fusion band 330, α=0. = The background pixel information is completely preserved; while for pixels in between, the value of α smoothly and continuously changes from 1 to 0. For example, a linear function can be used. ,in It is half the width of the blending zone. In this way, the edges of defects blend naturally with the background, thereby eliminating perceptible seams.
[0031] Accordingly, after the fusion operation is completed, a new synthetic image with high-fidelity defects is generated. Finally, in the output result step S106, the synthetic image output module 50 saves the generated image to a specified storage location and can selectively update its metadata to record information such as the source defect ID and fusion parameters used. The image generated using the method of this embodiment seamlessly integrates the defects with the product background, with natural edge transitions and a strong sense of visual realism.
[0032] Furthermore, as an optional implementation, the system in this embodiment can also provide an interactive defect editing function. Before performing the edge-weighted fusion step S105, the perspective transformation mapping module 30 or a separate interactive editing module can provide a toolbar that allows the user to fine-tune the transformed defect area that has been mapped onto the target image, such as performing small-range translation, rotation, scaling, or adjusting its grayscale, contrast, brightness, etc. This provides great flexibility for generating more diverse or specific defect samples.
[0033] Example 2 This embodiment, based on Embodiment 1, provides a variant with a higher degree of automation. The main difference lies in the method of obtaining the product region vertices. This embodiment aims to reduce manual operations and improve efficiency when batch processing images.
[0034] In this embodiment, the method flow is the same as... Figure 2 The system structure is basically the same as shown, and the system structure is also similar. Figure 1 Similar. However, the behavior of the product area vertex acquisition module 20 is different when performing step S102 of acquiring four pairs of corresponding vertices.
[0035] After the data loading module 10 loads the target image 100 and the source image 200, the product region vertex acquisition module 20 first calls a pre-trained object detection model. This model is specifically trained for a particular type of product (such as the chip in this embodiment) and can quickly and accurately locate the position and range of the product in the input image. The object detection models that can be used include, but are not limited to, YOLO series models, Faster R-CNN, SSD, etc.
[0036] The object detection model infers the bounding box coordinates of the target image 100 and outputs the bounding box coordinates of the target product region 110. Similarly, it infers the bounding box coordinates of the source image 200 and outputs the bounding box coordinates of the source product region 210. The product region vertex acquisition module 20 then directly extracts the coordinates of the four corner points of these two bounding boxes as the initial target vertices ptok1, ptok2, ptok3, ptok4 and source vertices ptng1, ptng2, ptng3, ptng4.
[0037] This automated method allows the system to quickly locate and extract vertices of product areas without any manual selection. In batch generation tasks involving hundreds or thousands of images, this can significantly save time and labor costs.
[0038] Understandably, considering the potential minor errors in the object detection model, this embodiment retains the interactive editing function described in Embodiment 1. After automatically detecting and extracting vertices, the system will visualize the results on the interface. If operators find deviations in the automatically located vertices, they can still manually fine-tune the vertex positions by dragging the mouse, as in Embodiment 1, to ensure the highest alignment accuracy.
[0039] After the vertex acquisition step S102 is completed, the subsequent homography matrix calculation step S103, perspective transformation mapping step S104, edge weighted fusion step S105, and output result step S106 are implemented in exactly the same way as described in Example 1.
[0040] This embodiment optimizes the vertex acquisition process by introducing automated target detection technology, achieving an upgrade from manual to "automatic detection + manual verification", which significantly improves the automation level and processing efficiency of the entire defect generation process. It is particularly suitable for scenarios where the product position is relatively fixed and the degree of standardization is high on the production line.
[0041] Example 3 This embodiment, based on Embodiment 1, provides a superior variant for the edge blending step, aiming to generate composite images with more delicate and seamless edge transitions. The main difference in this embodiment is that it uses the Poisson image editing algorithm instead of the linear weighted fusion algorithm.
[0042] In this embodiment, the preliminary steps of the method, namely the image acquisition step S101, the acquisition of four pairs of corresponding vertices step S102, the calculation of the homography matrix step S103, and the perspective transformation mapping step S104, are implemented in exactly the same way as in Embodiment 1. The system also first uses four-point mapping to accurately transform and locate the defective region in the source image 200 onto the target image 100.
[0043] The main difference lies in the edge-weighted fusion step S105. In this embodiment, this step is replaced by a "Poisson fusion step," which is performed by an edge fusion module 40 configured to perform Poisson fusion.
[0044] Poisson image editing is an image fusion technique based on solving the Poisson equation. Its core idea is to preserve the gradient field of the source image (i.e., the transformed defect area) as much as possible within the fusion region, while ensuring that the boundary of the fusion region is smooth and continuous with the pixel values of the target image (i.e., the background). The gradient represents the texture and detail information of the image. Therefore, preserving the gradient field means preserving the texture features of the defect itself, while the smooth continuity of the boundary ensures seamless stitching.
[0045] Specifically, the edge blending module 40 performs the following operations: 1. Define the solution domain: The contour of the defect region after perspective transformation is taken as the internal region Ω for solving the Poisson equation.
[0046] 2. Set the guiding field: Calculate the gradient field inside the defect region after transformation (i.e., the rate of change of intensity of each pixel in the x and y directions), as the guiding vector field v in the solution process.
[0047] 3. Set boundary conditions: Use the pixel values of the target image 100 on the boundary of region Ω as the Dirichlet boundary conditions of the Poisson equation.
[0048] 4. Solving the Poisson equation: Find an equation in the region Ω that makes the new image... gradient As close as possible to the guiding field The Poisson equation, i.e., solving... It satisfies the above boundary conditions. The solution to this equation... This refers to the new pixel values of all pixels within the merged region.
[0049] By solving this equation, the new pixel values obtained by the system not only blend perfectly into the surrounding background tone and brightness, but also preserve the details and textures inside the defect itself to the maximum extent.
[0050] Compared to the linear weighted fusion in Example 1, Poisson fusion has a particularly significant advantage in handling uneven lighting and complex background textures. Linear weighted fusion is essentially an averaging of pixel values, which may lead to blurring or decreased contrast in the fusion zone. Poisson fusion, on the other hand, operates in the gradient domain. It does not directly mix pixel values but rather "transplants" the structural information of the image. Therefore, it can produce a stitching effect that is almost indistinguishable to the naked eye, resulting in a more realistic and faithful composite image.
[0051] After the Poisson fusion step is completed, the method proceeds to the output result step S106, which outputs and saves the composite image with excellent fusion effect.
[0052] This embodiment provides a preferred solution for achieving the best possible fusion effect. Although its computational complexity is higher than that of linear weighted fusion, this embodiment has significant application value in application scenarios with extremely high requirements for the quality of synthesized samples, such as training sophisticated models for detecting subtle defects or identifying false defects.
[0053] Example 4 This embodiment aims to demonstrate the broad applicability of the method and system proposed in this application, proving that its core ideas are not limited to the generation of chip defects, but can also be extended to defect data augmentation tasks of other industrial products with definable profiles. This embodiment uses the generation of crack defects on solar cells as an example for illustration.
[0054] Solar cells are another type of product that requires rigorous surface defect inspection in industrial production. Defect types include cracks, broken grids, and black spots. Among these, obtaining samples of crack defects is also difficult and lacks diversity.
[0055] The method flow of this embodiment and Figure 2 The diagram is completely consistent with the diagram, and the system structure is also identical. Figure 1 The process is identical to that in Example 1, with only minor adjustments to the objects being processed and specific details.
[0056] First, in the image acquisition step S101, the data loading module 10 loads an image of a intact solar cell as the target image 100, and loads an image of a solar cell with a real crack from the defect library as the source image 200, while loading the annotation information of the crack area.
[0057] Next, in step S102 of obtaining four pairs of corresponding vertices, since solar cells are usually standard rectangles, their four corner points are very clear. Therefore, the product area vertex acquisition module 20 can very easily (whether through automatic detection in Embodiment 2 or manual clicking in Embodiment 1) determine the four corner points of the main body of the solar cell on the target image 100 and the source image 200, respectively, as target vertices ptok1-ptok4 and source vertices ptng1-ptng4.
[0058] Then, in the homography matrix calculation step S103 and the perspective transformation mapping step S104, the perspective transformation mapping module 30 calculates the homography matrix H based on these four pairs of precisely corresponding corner points, and precisely maps the crack defect area in the source image 200 to the corresponding position of the battery cell on the target image 100 through perspective transformation. Even if there are slight differences in angle or position between the two battery cell images during shooting, this transformation can ensure that the shape and direction of the crack match the orientation of the target battery cell.
[0059] Subsequently, in the edge-weighted fusion step S105, the edge-weighted fusion module 40 processes the mapped crack edges. Since cracks are typically thin, dark lines, the fusion of their edges with the battery cell background also requires a smooth transition to simulate the visual effect of real cracks under different lighting conditions. This embodiment uses the exact same edge-weighted fusion algorithm as Embodiment 1, that is, defining a fusion band 330 at the crack edge and using the formula based on the distance weight α. The final pixel value is calculated so that the crack can "grow" naturally on the surface of the intact battery cell.
[0060] Finally, in the output result step S106, the system outputs a composite image with realistic cracks generated on an intact solar cell.
[0061] The successful implementation of this embodiment demonstrates the excellent versatility of the core technical framework proposed in this application: "precise geometric alignment based on four-point mapping" and "natural appearance synthesis based on edge blending." As long as the industrial product being processed has a relatively stable contour or region (whether rectangular, trapezoidal, or other quadrilaterals) that can be defined by four vertices, the method and system of this application can be effectively applied, such as liquid crystal display panels, mobile phone glass covers, and printed circuit boards. This greatly expands the application scope and industrial value of the technology in this application.
[0062] Additionally, as an optional implementation, a similarity matching function can be further introduced in the above embodiments. For example, after the image acquisition step S101, a similarity matching step can be added. The system can extract global or local features of the target image 100 (such as color histogram, texture features, structural layout, etc.) and perform similarity calculations with the features of all source images in the defect library (e.g., calculating the cosine similarity or Euclidean distance between feature vectors). Then, the system can recommend a list of candidate source images that are most similar to the current target image 100 to the user. This ensures that the defects used for transfer are more closely matched with the target scene in terms of background, lighting, etc., thereby generating more realistic synthetic samples and further improving the intelligence level of defect generation.
[0063] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.
Claims
1. An image-based defect generation method, characterized in that, Includes the following steps: Obtain the target image of the defect to be synthesized, the source image containing the source defect, and the defect region in the source image; Product regions are determined on the target image and the source image respectively, and four pairs of corresponding vertices of the product regions are obtained; Based on the four pairs of corresponding vertices, calculate the geometric transformation matrix used to map the source image coordinate system to the target image coordinate system; Using the geometric transformation matrix, a geometric transformation is performed on the defective region to map the defective region onto the product region of the target image; On the target image, a pixel fusion operation is performed on the edges of the mapped defect region to achieve a smooth transition between the defect region and the region in the target image corresponding to the defect region.
2. The image-based defect generation method according to claim 1, characterized in that, The geometric transformation matrix is a homography matrix, and the geometric transformation is a perspective transformation.
3. The image-based defect generation method according to claim 1 or 2, characterized in that, The pixel fusion operation is edge-weighted fusion, which includes: A fusion band is defined at the edge of the mapped defect region, and a fusion formula is applied. Calculate the final pixel values of the pixels within the fusion band, where, For the final pixel value, This represents the pixel value of the mapped defect region at that location. The original pixel value of the target image at that location. The fusion weight is determined based on the distance from the pixel to the edge of the mapped defect region.
4. The image-based defect generation method according to claim 1, characterized in that, The step of acquiring the source image includes: The image features of the target image are extracted and similarity calculation is performed with the features of multiple images in a defect library to output one or more candidate source images containing the source defect; One of the one or more candidate source images is selected as the source image.
5. The image-based defect generation method according to claim 1, characterized in that, The method further includes: An interactive editing function is provided, which allows users to manually fine-tune the positions of the four pairs of corresponding vertices and / or to translate, rotate, scale, adjust grayscale or contrast of the mapped defect area.
6. An image-based defect generation system, characterized in that, include: The image acquisition module is used to acquire the target image of the defect to be synthesized, the source image containing the source defect, and the defect region in the source image; The product region vertex acquisition module is used to determine the product region on the target image and the source image respectively, and acquire four pairs of corresponding vertices of the product region; The geometric transformation mapping module is used to calculate the geometric transformation matrix based on the four pairs of corresponding vertices, and to perform a geometric transformation on the defect region using the geometric transformation matrix, so as to map the defect region to the product region of the target image; An edge blending module is used to perform pixel blending operations on the edges of the mapped defective region on the target image to achieve a smooth transition between the defective region and the region in the target image corresponding to the defective region.
7. The image-based defect generation system according to claim 6, characterized in that, The geometric transformation mapping module is configured to calculate the homography matrix and perform perspective transformation.
8. The image-based defect generation system according to claim 6 or 7, characterized in that, The edge blending module is configured to perform edge-weighted blending, which defines a blending band at the edge of the mapped defect region and follows a blending formula. Calculate the final pixel values of the pixels within the fusion band, where, For the final pixel value, This represents the pixel value of the mapped defect region at that location. The original pixel value of the target image at that location. The fusion weight is determined based on the distance from the pixel to the edge of the mapped defect region.
9. The image-based defect generation system according to claim 6, characterized in that, The system also includes: The similarity matching module is used to obtain the target image from the image acquisition module, extract the image features of the target image, and perform similarity calculation with the features of multiple images in a defect library to output one or more candidate source images containing source defects; Furthermore, the image acquisition module is configured to select one of the one or more candidate source images as the source image.
10. The image-based defect generation system according to claim 6, characterized in that, The system also includes: An interactive editing module is provided to offer an interactive interface that allows users to manually fine-tune the positions of the four pairs of corresponding vertices and / or to translate, rotate, scale, adjust grayscale or contrast of the mapped defect area.