Defect image data enhancement method, device and equipment and storage medium

Abstract

The invention discloses a data enhancement method and device for a defect image, equipment and a storage medium, and relates to the field of image processing. Acquiring a defect image set; extracting defect features of each defect image in the defect image set, and respectively generating a defect feature set and a background image set corresponding to the defect image set; and performing feature transformation on the defect feature set based on the feature change sequence, and generating a defect image sequence set in combination with the background image set. According to the scheme, all original defect images in a defect image set are subjected to feature extraction and are converged into a defect feature set and a background image set; and carrying out feature transformation on the defect feature set by adopting a preset feature change sequence, and expanding each defect feature into a defect feature sequence to realize quantity fission. And then full matching and defect fusion are carried out on each defect feature sequence and background images in the background image set, rich defect images are obtained through expansion, and a data enhancement effect is realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of document scanning, and in particular to a method, device, equipment and storage medium for data enhancement of defective images. Background Art

[0002] In the field of industrial inspection, data enhancement technology for defect images plays a key role in training detection algorithms, enhancing data sets, and simulating various defect situations. At present, the main methods for data enhancement of defect images include methods based on generative adversarial networks (GANs), methods based on diffusion models, and traditional image processing methods.

[0003] Whether it is generative adversarial networks, diffusion models or traditional image processing, the generation of defect images requires the use of a complete defect image library or database. A neural network model is used for simulation generation. However, the defect images generated by simulation training may not be fine enough in terms of details, and the training process is unstable, prone to problems such as mode collapse. In addition, in terms of constructing defect data sets, traditional collection of defect images is not convenient. Among the limited wafer images collected, the number and type of defect images are single, and it is difficult to screen out a rich variety of defect types. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for data enhancement of defect images, so as to solve the problem that the types and quantities of existing defect data sets are single.

[0005] In one aspect, the present application provides a method for data enhancement of a defective image, the method comprising:

[0006] Obtain a defect image set;

[0007] Extracting defect features of each defect image in the defect image set, and generating a defect feature set and a background image set corresponding to the defect image set respectively;

[0008] The defect feature set is subjected to feature transformation based on the feature change sequence, and a defect image sequence set is generated in combination with the background image set.

[0009] Specifically, extracting defect features of each defect image in the defect image set and generating a defect feature set and a background image set corresponding to the defect image set respectively include:

[0010] Extracting defect images from the defect image set, and identifying defect contours of defect features contained therein;

[0011] Extracting a defect image from a defect image according to the defect contour to generate a defect feature map; generating a background image based on an image obtained by extracting the defect image from the defect image, wherein the background image does not contain defect features;

[0012] All defect images in the defect image set are extracted in sequence, defect feature maps and corresponding background images are respectively generated, and the defect feature maps and the corresponding background images are summarized into a defect feature set and a background image set.

[0013] Specifically, the performing feature transformation on the defect feature set based on the feature change sequence includes:

[0014] Determine the feature transformation content of each defect feature image in the defect feature set based on the feature change sequence content; the feature transformation content includes size transformation and angle rotation transformation;

[0015] Defect feature maps are extracted in sequence, and size transformation and / or angle rotation are performed according to feature transformation content to generate a feature map sequence; the feature map sequence contains defect feature maps of different sizes and / or rotation angles.

[0016] Specifically, after generating the feature graph sequence, the method further includes:

[0017] The target size of the defect image in the defect image set is obtained, and the feature trajectory sequence of the feature image sequence is determined according to the target size and the feature change sequence; the feature trajectory sequence is used to determine the fusion position of each defect feature image.

[0018] Specifically, the step of generating a defect image sequence set by combining a background image set includes:

[0019] Selecting a target background image from the background image set;

[0020] The defect feature images in the defect feature set are numbered in sequence according to the feature trajectory sequence, and the target background images are respectively fused according to the sequence numbers to generate a defect image sequence;

[0021] Different background images are selected from the background image set in sequence, and defect feature images are fused according to a feature trajectory sequence to obtain the defect image sequence set.

[0022] Specifically, determining the feature trajectory sequence of the feature map sequence according to the image size and feature change sequence includes:

[0023] Determine the number of defect feature graphs in the defect feature set and determine the trajectory points;

[0024] Generate a defect trajectory frame based on the target size, and generate a trajectory curve within the defect trajectory frame;

[0025] A target trajectory point is selected according to the number of the defect feature graphs, and a feature trajectory sequence of the feature graph sequence is generated based on the target trajectory point.

[0026] Specifically, the step of generating a background image based on an image obtained by extracting the defect image from the defect image includes:

[0027] Determine the image coordinates of the defect contour and expand the pixel area of ​​the target range width based on the defect contour;

[0028] The pixel area of ​​the target range width is cut into several color picking frames, and the contour pixel value is determined according to the pixel average value of the color picking frames;

[0029] Select blank pixels from the defect outline inward, and use the blank pixels as the corner points of a rectangle to expand a rectangular pixel frame; the rectangular pixel frame contains at least 4*4 pixels;

[0030] Taking the average of the remaining pixels in the rectangular pixel frame, and determining the pixel average as the pixel value of the blank pixel;

[0031] Blank pixels are selected layer by layer along the defect contour and pixel values ​​are calculated until all defect contours are filled to obtain a repaired background image.

[0032] On the other hand, the present application provides a data enhancement device for a defective image, the device comprising:

[0033] An acquisition module, used for acquiring a defect image set;

[0034] A first generating module is used to extract defect features of each defect image in the defect image set, and respectively generate a defect feature set and a background image set corresponding to the defect image set;

[0035] The second generating module is used to perform feature transformation on the defect feature set based on the feature change sequence, and generate a defect image sequence set in combination with the background image set.

[0036] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the data enhancement method for defective images described in the above aspects.

[0037] On the other hand, the present application provides a computer-readable storage medium, wherein the readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the data enhancement method for defective images described in the above aspects.

[0038] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least: the present application extracts features from all original defect images in the defect image set and aggregates them into a defect feature set and a background image set. For the defect feature set, a preset feature change sequence is used to perform feature transformation, and each defect feature is expanded into a defect feature sequence to achieve quantitative fission. Then, each defect feature sequence is fully matched and defect-fused with the background image in the background image set, and a rich defect image is obtained by expansion to achieve data enhancement effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a method for data enhancement of defective images provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram showing the characteristics of a fusion defect;

[0041] Figure 3 Schematic diagram showing characteristics of different types of defects;

[0042] Figure 4 is a schematic diagram of extracting defect features and background images provided by an exemplary embodiment;

[0043] Figure 5 A schematic diagram showing a defect trajectory curve generated in the Pad region;

[0044] Figure 6 A schematic diagram showing the generation of a defect image sequence by mapping and fusing a feature trajectory sequence with the same background image is shown;

[0045] Figure 7 A schematic diagram of collecting a rectangular pixel frame to calculate a target pixel point is shown;

[0046] Figure 8 It is a schematic diagram of restoring target pixels of multiple color blocks;

[0047] Fig. 9 A structural block diagram of a defect image data enhancement device provided in an embodiment of the present application is shown;

[0048] Fig.10 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0050] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0051] Figure 1 : is a flow chart of a method for enhancing defective images provided in an embodiment of the present application, comprising the following steps:

[0052] S1, obtaining a defect image set;

[0053] The defect image set described in this embodiment is an image of a wafer or material board obtained after being photographed and screened by a camera, and the image contains at least one determined defect feature (point). These original defect images with defect features are aggregated to obtain a defect image set. In particular, the defect image set of this application can be images of different magnifications or different sizes. In order to facilitate unified operation, this application is illustrated by taking the same image size as an example.

[0054] S2, extracting defect features of each defect image in the defect image set, and generating a defect feature set and a background image set corresponding to the defect image set respectively;

[0055] Different from the traditional generative adversarial network and Diffusion model modes of generating defect images, these modes use AI to generate images with poor authenticity. However, the method of extracting features from existing defect images and regenerating them can ensure authenticity requirements. The core idea is to extract the original defect images in the defect image set one by one, identify and extract the defect features therein, that is, to extract the defect feature map. The extracted defect feature map forms a defect feature set, and the remaining part of the image after extracting the defect features forms the background image. The background image here is a pure image after completely removing the defect features. All background images are gathered to obtain a background image set, which is used for subsequent data enhancement to enrich the database of defect images.

[0056] S3. Perform feature transformation on the defect feature set based on the feature change sequence, and generate a defect image sequence set in combination with the background image set.

[0057] Because the original defect image is selected and the data volume is small, simply combining the extracted defect feature set with the background image cannot meet the data richness. Therefore, this application chooses to perform some feature transformations on all defect features covered in the defect feature set, and the feature change sequence contains specific feature transformation rules, such as how to enlarge or reduce defect features, and how to integrate defect features into the background image.

[0058] All defect features in the defect feature set are transformed based on the feature change sequence, that is, each defect feature can be transformed to expand a feature sequence, and the defect feature maps in the feature sequence are obtained after transformation and have different forms. Then these defect feature maps are used as foreground images and combined with each background image in the background image set to generate a corresponding defect image sequence for each defect feature sequence, and all defect feature sequences can expand a defect image sequence set to achieve a richer image data set enhancement effect.

[0059] Figure 2 The diagram shows the schematic diagram of fused defect features, where the defect features circled in the upper part of the left figure are the extracted and cut-out parts, while the defect image on the right is obtained after the feature transformation and re-fusion of the background image (the background image can be the same as or different from the original background). After this transformation, the defect points in the original defect image can be fully utilized and fully expanded to obtain a rich defect image.

[0060] In summary, the present application extracts features from all original defect images in the defect image set and aggregates them into a defect feature set and a background image set. For the defect feature set, a preset feature change sequence is used for feature transformation, and each defect feature is expanded into a defect feature sequence to achieve quantitative fission. Each defect feature sequence is then fully matched and defect-fused with the background image in the background image set to expand and obtain rich defect images, thereby achieving data enhancement effects.

[0061] In some possible implementations, the process of generating the defect feature set and the background image set may be implemented using the following steps:

[0062] a. Extract the original defect image from the defect image set and identify the defect contour of the defect features contained therein;

[0063] b. Extracting defect features from the original defect image according to the defect contour to generate a defect feature map; generating a background image based on the image after the defect features are extracted from the original defect image, and the background image does not contain the defect features;

[0064] Figure 3A schematic diagram shows different types of defect features, including board loss, electroplating resist, solder mask peeling, solder mask scratches, post-gold scratches, gold surface agglomeration, nickel leakage on gold surface, copper exposure on gold surface, gold surface depression, discoloration, foreign matter on pad, and ink on pad. The screening of these defects can be done manually or by other AI model detection, and then a defect feature set is generated based on the contour range.

[0065] Assuming that the defect feature set includes A defect, B defect and C defect, then the feature change sequence is transformed to form A sequence {A1, A2, A3, A4}, B sequence {B1, B2, B3, B4}, and C sequence {C1, C2, C3, C4}. Each feature in these sequences can be defect fused with all background images. A defect image sequence is generated for A sequence features, B defect image sequence is generated for B sequence features, and C defect image sequence is generated for C sequence features. Figure 4 It is a schematic diagram of extracting defect features and background images provided by an exemplary embodiment. The background image generation process needs to repair the remaining part after the defect features are extracted, which is equivalent to a normal image.

[0066] c. Extract all original defect images in the defect image set in turn, extract and generate defect feature maps and corresponding background images respectively, and summarize them into a defect feature set and a background image set.

[0067] Furthermore, the defect feature set is transformed based on the feature change sequence, which may specifically include the following steps:

[0068] d. Determine the feature transformation content of each defect feature image in the defect feature set based on the feature change sequence content;

[0069] The feature change sequence specifies the feature transformation of the defect feature set and the rules for defect fusion with the background image set. Based on the rules, it is clear which feature transformations are performed on the defect feature map. In the embodiment of the present application, the feature transformation content includes size transformation and angle rotation transformation, such as magnification by 2 times, 4 times, reduction by 1 times, image compression, image rotation by 90 degrees, etc.

[0070] e. Extract defect feature maps in sequence, perform size transformation and / or angle rotation according to feature transformation content, and generate a feature map sequence;

[0071] For a selected defect feature map, the corresponding feature map sequence will be generated after feature transformation using the feature transformation content. These feature map sequences contain defect feature maps of different sizes and / or rotation angles. This step is mainly to expand the defect feature content, so as to ensure the diversity of subsequent defect data.

[0072] In some embodiments, the feature change sequence also records defect fusion rules, and defect images are generated based on these rules, rather than randomly generated, to avoid duplication. In particular, the fusion position of the sequence feature image will directly determine the defect effect after fusion. For this reason, the present application also continues to perform step f.

[0073] f. Obtain the target size of the defect image in the defect image set, and determine the feature trajectory sequence of the feature image sequence according to the target size and the feature change sequence.

[0074] The image sizes in the defect image set can be the same or different. Different feature trajectory sequences can be established for different image sizes. When the defect is fused, the matching feature trajectory sequence is selected according to the size of the background image. This application is described with a unified target size. After obtaining the target size, the feature trajectory is generated according to the coverage range of the target size. When fused, the defect feature sequence will be integrated into the target position of the background image in sequence according to the feature trajectory. The process can be further described as follows:

[0075] 1. Determine the number of defect feature graphs in the defect feature set and determine the trajectory points;

[0076] The number of defect feature maps represents the number of trajectory points contained in the feature trajectory. Considering the maximum number, for a selected background image, an N:1 mapping mode will be used for defect fusion, where N is the number of defect feature maps in the defect feature set.

[0077] 2. Generate a defect trajectory frame based on the target size, and generate a trajectory curve within the defect trajectory frame;

[0078] Figure 5 The schematic diagram of the defect trajectory curve generated in the Pad area is shown. The trajectory curve must be drawn within the size range of the defect trajectory box. In fact, the defect trajectory box does not consider the background image content when drawing the trajectory curve, that is, a blank box with the same size as the background image, and then the trajectory curve is drawn in the blank box. Figure 5 What is shown is the path effect after mapping the real background image.

[0079] 3. Select target trajectory points according to the number of defect feature maps, and generate a feature trajectory sequence of the feature map sequence based on the target trajectory points.

[0080] The target trajectory point is the fusion position of the defect feature map on the trajectory curve. After determining the target trajectory point, the target trajectory point (including the coordinates of the target trajectory point in the defect trajectory box) can be associated and matched with the defect feature map, for example, the trajectory point and the feature sequence image are set in a numbering order, and the feature trajectory sequence is generated according to the number matching.

[0081] In particular, a trajectory curve can be set to correspond to a defect feature sequence, and each target trajectory point corresponds to a defect feature map in the defect feature set. Of course, when the background image Pad or gold surface is rich, multiple trajectory curves can be set to include multiple defect feature sequences at the same time, or multiple defect feature sequences can be used to draw a trajectory curve together.

[0082] Based on the above embodiments, the process of generating a defect image sequence set by combining a background image set can be achieved by the following steps:

[0083] 1. Select the target background image from the background image set;

[0084] 2. The defect feature images in the defect feature set are numbered in sequence according to the feature trajectory sequence, and the target background images are fused according to the sequence numbers to generate a defect image sequence;

[0085] Note that this step can randomly select different background images of the same size or use the same background image for fusion, that is, N defect feature images of a feature trajectory sequence correspond to N background images, and these N background images can be copied from one target background image, or from several target background images. Figure 5 What is shown in the figure is that a background image is used to map a feature trajectory sequence for display.

[0086] 3. Select different background images from the background image set in turn, and fuse the defect feature maps according to the feature trajectory sequence to obtain a defect image sequence set.

[0087] Figure 6 The schematic diagram of generating a defect image sequence by mapping a fusion feature trajectory sequence with the same background image is shown. In the above technical solution, by extracting a limited number of original defect images, expanding them with a small number of defect features to the maximum extent, and using a trajectory sequence to determine the fusion position, the maximum number of defects can be fused for each background image, thereby improving data richness.

[0088] The above parts are all explained using the repaired background image as an example, because this process does not introduce other irrelevant normal images, but directly uses the image after the defect features are removed. In fact, in order to achieve the data enhancement effect, additional images of normal areas can be introduced or photographed, but this method is not conducive to the subsequent defect image annotation, because the produced defect images are used to train AI models. While providing a test training set for the AI ​​model, it is also necessary to ensure that there is a corresponding verification set for verification, that is, to ensure that there is a completely matching defect annotated image. Therefore, this application does not use direct shooting or introduction of other normal images, but uses the image after the defect is removed for repair, so that the fused defect image can be directly used for subsequent generation of annotated images.

[0089] The process of extracting defective images to generate background images can be implemented through the following steps:

[0090] 1. Determine the image coordinates of the defect contour;

[0091] 2. Select blank pixels from the defect outline inward, and use the blank pixels as the corner points of the rectangle to expand the rectangular pixel frame; the rectangular pixel frame must contain at least 4*4 pixels;

[0092] 3. Take the average of the remaining pixels in the rectangular pixel frame, and determine the pixel average as the pixel value of the blank pixel;

[0093] Figure 7 The schematic diagram of collecting rectangular pixel frames to calculate target pixel points is shown. The scheme uses a successive approximation strategy to calculate pixel values ​​inward in sequence. The internal pixels are completely determined by the size of the pixel values ​​in the surrounding area, and there will be no situation where the color difference with the outer contour is too large.

[0094] Figure 8 This is a schematic diagram of restoring target pixels in multiple color blocks. For the case where defect features span multiple color areas at the same time, the mode of first restoring the background contour edge line and then performing pixel value approximation calculation can be adopted. Figure 8 The dotted line inside the defect contour is a virtual contour restored based on the background contour, and then the contour is defined as a color boundary. The S1 and S2 areas on the left and right of the color boundary correspond to two completely different color blocks. For the S1 and S2 blocks, blank pixels are also selected inward according to the curve of the outermost defect contour, and the blank pixels are used as the rectangular corner points to expand the rectangular pixel frame to calculate the pixel value, but the color boundary line is used as a sign to stop the approximation calculation, that is, the S1 area is approximated and calculated from left to right as a whole, and the S2 area is approximated and calculated from right to left as a whole. In this way, when the two areas approach the color boundary line, they stop, thereby restoring two different color blocks. For other forms, such as the case of dividing the color block up and down, this solution is also applicable, and the details will not be repeated.

[0095] 4. Select blank pixels layer by layer along the defect contour and calculate the pixel values ​​until all defect contours are filled to obtain the repaired background image.

[0096] Fig. 9 The structural block diagram of the data enhancement device for defective images provided in an embodiment of the present application is shown, and the device includes:

[0097] An acquisition module 910 is used to acquire a defect image set;

[0098] A first generating module 920 is used to extract defect features of each defect image in the defect image set, and respectively generate a defect feature set and a background image set corresponding to the defect image set;

[0099] The second generating module 930 is used to perform feature transformation on the defect feature set based on the feature change sequence, and generate a defect image sequence set in combination with the background image set.

[0100] The data enhancement device for defective images provided in the embodiments of the present application can be applied to the data enhancement method for defective images provided in the above embodiments. For relevant details, refer to the above method embodiments. The implementation principles and technical effects are similar and will not be repeated here.

[0101] It should be noted that the data enhancement device for defective images provided in the embodiments of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In practical applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the data enhancement device for defective images is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the data enhancement method for defective images provided in the above-mentioned method embodiment and the implementation method of the data enhancement device for defective images provided in this embodiment belong to the same concept. The specific implementation process of the data enhancement device for defective images provided in this embodiment is detailed in the above-mentioned method embodiment and will not be repeated here.

[0102] Fig.10The block diagram of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory may be connected by a bus or in other ways. Among them, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0103] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0104] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0105] In some embodiments, the computer device may further optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit, a display screen and a keyboard.

[0106] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0107] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen is a touch screen, the display screen also has the ability to collect touch signals on the surface or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in other embodiments, the display screen can be at least two, respectively arranged on different surfaces of the computer device or in a folding design; in other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0108] The power supply is used to power various components in the computer device. The power supply can be AC, DC, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0109] Those skilled in the art will appreciate that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0110] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0111] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A data enhancement method for a defective image, characterized in that: The method comprises: Obtain a defect image set; Extracting defect features of each defect image in the defect image set, and generating a defect feature set and a background image set corresponding to the defect image set respectively; The defect feature set is subjected to feature transformation based on the feature change sequence, and a defect image sequence set is generated in combination with the background image set.

2. The method according to claim 1, characterized in that The step of extracting defect features of each original defect image in the defect image set and generating a defect feature set and a background image set corresponding to the defect image set respectively includes: Extracting an original defect image from the defect image set, and identifying a defect contour of a defect feature contained therein; Extracting defect features from the original defect image according to the defect contour to generate a defect feature map; generating a background image based on the image after the defect features are extracted from the original defect image, and the background image does not contain the defect features; All original defect images in the defect image set are extracted in sequence, defect feature maps and corresponding background images are respectively generated, and the defect feature maps and the corresponding background images are summarized into a defect feature set and a background image set.

3. The method according to claim 1, characterized in that The performing feature transformation on the defect feature set based on the feature change sequence includes: Determine the feature transformation content of each defect feature image in the defect feature set based on the feature change sequence content; the feature transformation content includes size transformation and angle rotation transformation; Defect feature maps are extracted in sequence, and size transformation and / or angle rotation are performed according to feature transformation content to generate a feature map sequence; the feature map sequence contains defect feature maps of different sizes and / or rotation angles.

4. The method according to claim 3, characterized in that: After generating the feature map sequence, the method further includes: The target size of the defect image in the defect image set is obtained, and the feature trajectory sequence of the feature image sequence is determined according to the target size and the feature change sequence; the feature trajectory sequence is used to determine the fusion position of each defect feature image.

5. The method according to claim 4, characterized in that The step of combining the background image set to generate a defect image sequence set includes: Selecting a target background image from the background image set; The defect feature images in the defect feature set are numbered in sequence according to the feature trajectory sequence, and the target background images are respectively fused according to the sequence numbers to generate a defect image sequence; Different background images are selected from the background image set in sequence, and defect feature images are fused according to a feature trajectory sequence to obtain the defect image sequence set.

6. The method according to claim 4, characterized in that The step of determining the feature trajectory sequence of the feature graph sequence according to the image size and feature change sequence comprises: Determine the number of defect feature graphs in the defect feature set and determine the trajectory points; Generate a defect trajectory frame based on the target size, and generate a trajectory curve within the defect trajectory frame; A target trajectory point is selected according to the number of the defect feature graphs, and a feature trajectory sequence of the feature graph sequence is generated based on the target trajectory point.

7. The method according to claim 2, characterized in that The generating of the background image based on the image after the defect image is extracted from the defect image comprises: Determine the image coordinates of the defect contour; Select blank pixels from the defect outline inward, and use the blank pixels as the corner points of a rectangle to expand a rectangular pixel frame; the rectangular pixel frame contains at least 4*4 pixels; Taking the average of the remaining pixels in the rectangular pixel frame, and determining the pixel average as the pixel value of the blank pixel; Blank pixels are selected layer by layer along the defect contour and pixel values ​​are calculated until all defect contours are filled to obtain a repaired background image.

8. A data enhancement device for a defective image, characterized in that: The device comprises: An acquisition module, used for acquiring a defect image set; A first generating module is used to extract defect features of each defect image in the defect image set, and respectively generate a defect feature set and a background image set corresponding to the defect image set; The second generating module is used to perform feature transformation on the defect feature set based on the feature change sequence, and generate a defect image sequence set in combination with the background image set.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the data enhancement method for defective images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the data enhancement method for defective images as described in any one of claims 1 to 7.

Images