Image data processing method, apparatus and device, and storage medium

Abstract

Disclosed herein are an image data processing method, apparatus and device, and a storage medium. The processing method comprises: acquiring a product defect image corresponding to a defect of a target category and a product background image of a target product; acquiring a pre-constructed semantic model, the semantic model having a capability of determining in the product background image regions in which defects of various categories are produced; using the semantic model to determine in the product background image a target region corresponding to the defect of the target category; and fusing the product defect image into the target region to obtain a defect image sample of the target product corresponding to the defect of the target category.

Description

Image data processing method, device, equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application number CN202311280386.1, filed on September 28, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure belongs to the field of industrial detection technology, and in particular relates to image data processing, devices, equipment and storage media. Background Art

[0004] During the industrial production process, a large number of defective products are randomly generated. To ensure product quality and reduce production costs, defective products must be immediately removed through industrial inspection methods. With the increasing maturity of artificial intelligence technology, deep learning technology is widely used in industrial inspection to detect, identify, and segment defects in product images. This technology requires a sufficient number of defective image samples.

[0005] However, in industrial scenarios, there are many scenarios where samples are difficult to collect. The sub-scenarios are as follows: 1. The core indicators of the manufacturing industry are production capacity and yield. Companies reduce scrap rates and maintain continuous profitability by maintaining high yields, but an excessively high yield will make it impossible to collect enough samples; 2. When producing products in small batches and with multiple varieties, there may be situations where defective samples have not been collected yet, while the product has been changed; 3. Defects are diverse in form and have a long-tail effect, making it impossible to traverse all types of defects.

[0006] Therefore, how to increase product defect image samples in industrial scenarios is an urgent problem that needs to be solved.

[0007] Summary of the Invention

[0008] The embodiments of the present disclosure provide a method, apparatus, device, and storage medium for processing image data, which can realize the automatic generation of defect image samples in industrial scenarios and facilitate the rapid start-up of industrial inspection projects.

[0009] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0010] According to a first aspect of an embodiment of the present disclosure, a method for processing image data is provided, including: obtaining a product defect image corresponding to a target category defect, and a product background image of a target product; obtaining a pre-constructed semantic model, wherein the semantic model has the ability to determine the occurrence area of ​​various categories of defects in the product background image; using the semantic model to determine a target area corresponding to the target category defect in the product background image; and fusing the product defect image to the target area to obtain a defect image sample of the target product corresponding to the target category defect.

[0011] In some embodiments of the present disclosure, based on the aforementioned scheme, the semantic model is constructed by the following steps: obtaining an image sample to be trained, and determining the boundary information and category information of the target object in the image sample to be trained; determining the circumscribed graphic area corresponding to the target object in the image sample to be trained based on the boundary information; obtaining a first image containing the circumscribed graphic area; setting the RGB value of each pixel of the circumscribed graphic area in the first image to a first preset value to obtain a second image; and training the initial model based on the category information and the second image to obtain the semantic model.

[0012] In some embodiments of the present disclosure, based on the aforementioned scheme, obtaining the first image containing the circumscribed graphic area includes: dilating the circumscribed graphic area in the image sample to be trained to obtain an expanded area; and extracting an image from the expanded area in the image sample to be trained to obtain the first image.

[0013] In some embodiments of the present disclosure, based on the aforementioned solution, the area of ​​the expansion region is smaller than a second preset value.

[0014] In some embodiments of the present disclosure, based on the aforementioned scheme, the number of image samples to be trained obtained is positively correlated with the complexity of the target semantic relationship, wherein the target semantic relationship is the semantic relationship between the third image and the fourth image, the third image is the image corresponding to the target object in the image samples to be trained, and the fourth image is the image in the image samples to be trained other than the third image.

[0015] In some embodiments of the present disclosure, based on the aforementioned scheme, determining the target area corresponding to the target category defect in the product background image using the semantic model includes: determining multiple candidate areas and a dilated area corresponding to each candidate area in the product background image, wherein the RGB value of each pixel in the candidate area is set to the first preset value; scoring the dilated area corresponding to each candidate area using the semantic model to obtain a first score; and determining the target area corresponding to the target category defect from the multiple candidate areas based on the first score of the dilated area corresponding to each candidate area.

[0016] In some embodiments of the present disclosure, based on the aforementioned scheme, each candidate region corresponds to multiple dilated regions, the first score is used to represent the probability of the target category defect occurring in the corresponding candidate region, and determining the target region corresponding to the target category defect from the multiple candidate regions based on the first score of the dilated region corresponding to each candidate region includes: taking an average of the first scores of the multiple dilated regions corresponding to each candidate region as the second score corresponding to each candidate region; and determining the candidate region with the highest second score as the target region corresponding to the target category defect.

[0017] In some embodiments of the present disclosure, based on the aforementioned scheme, each candidate region has multiple dilated regions corresponding to it, and each dilated region has multiple first scores corresponding to it, wherein each first score is used to characterize the probability of a corresponding candidate category defect occurring in the corresponding candidate region. Determining the target region corresponding to the target category defect from the multiple candidate regions based on the first score of the dilated region corresponding to each candidate region includes: taking the average of the first scores of the multiple dilated regions corresponding to each candidate region on each candidate category defect as the second score for the occurrence of each candidate category defect in each candidate region; determining, based on the second scores for the occurrence of each candidate category defect in each candidate region, the candidate category defect with the highest second score as the target category defect corresponding to each candidate region; and determining the candidate region corresponding to the target category defect as the target region corresponding to the target category defect.

[0018] In some embodiments of the present disclosure, based on the aforementioned scheme, obtaining a product defect image corresponding to a target category defect includes: obtaining an original image of a product containing the target category defect, and a mask image containing the target category defect; and fusing the original image of the product with the mask image to obtain the product defect image.

[0019] In some embodiments of the present disclosure, based on the aforementioned solution, obtaining a product defect image corresponding to a target category defect includes:

[0020] Acquire an original image of a product containing the target category defect; and extract an image corresponding to the target category defect from the original image of the product to obtain the product defect image.

[0021] In some embodiments of the present disclosure, based on the aforementioned scheme, the product defect image is fused to the target area to obtain a defect image sample of the target product corresponding to the target category defect, including: determining a first gradient field of the product defect image and a second gradient field of the product background image; replacing the gradient field corresponding to the target area in the second gradient field with the first gradient field to obtain a third gradient field of the initial fused image; determining a transformation matrix coefficient based on the third gradient field; and constructing a defect image sample of the target product corresponding to the target category defect based on the transformation matrix coefficient.

[0022] In some embodiments of the present disclosure, based on the aforementioned scheme, the method for processing image data also includes: performing multiple deformation processing on the product defect image to obtain multiple fifth images; and fusing the multiple fifth images into the target area respectively to obtain multiple defect image samples of the target product corresponding to the target category defects.

[0023] In some embodiments of the present disclosure, based on the aforementioned scheme, the method for processing image data also includes: obtaining an original image of a product containing defects of the target category selected by a user based on a first control; generating a mask image containing defects of the target category based on the original image of the product in response to the user's operation of a second control; fusing the original image of the product with the mask image to obtain the product defect image; and executing the step of performing multiple deformation processing on the product defect image in response to the user's operation of a third control.

[0024] In some embodiments of the present disclosure, based on the aforementioned scheme, the step of performing multiple deformation processing on the product defect image in response to the user's operation of the third control includes: displaying a batch deformation interface in response to the user's operation of the third control; and performing the step of performing multiple deformation processing on the product defect image according to the deformation strength and deformation quantity input by the user in the batch deformation interface.

[0025] In some embodiments of the present disclosure, based on the aforementioned scheme, the image data processing method further includes: when the color of the product defect image is inconsistent with the color of the product background image, modifying the color of the product defect image according to the color of the product background image.

[0026] In some embodiments of the present disclosure, based on the aforementioned solution, the defect image sample is used for defect detection of the target product.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an image data processing device, comprising: an image acquisition unit, for acquiring a product defect image corresponding to a target category defect, and a product background image of a target product; a model acquisition unit, for acquiring a pre-built semantic model, wherein the semantic model has the ability to determine the occurrence area of ​​various categories of defects in the product background image; a target area determination unit, for determining a target area corresponding to the target category defect in the product background image using the semantic model; and an image fusion unit, for fusing the product defect image into the target area to obtain a defect image sample of the target product corresponding to the target category defect.

[0028] According to a third aspect of an embodiment of the present disclosure, there is provided an image data processing device, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the steps of the method described in any one of the first aspects above are implemented.

[0029] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, in which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor is prompted to implement the steps of the method described in any one of the first aspects above.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0032] FIG1 is a schematic flow chart showing a method for processing image data in one embodiment;

[0033] FIG2 shows a demonstration diagram of the generation process of the defect image sample in FIG1 ;

[0034] FIG3 shows another demonstration diagram of the generation process of the defect image sample in FIG1 ;

[0035] FIG4 is a schematic diagram showing a comparison between the effects of generating image samples using the semantic model in FIG1 and randomly generating image samples;

[0036] FIG5 is a schematic diagram showing an expansion region during the construction of the semantic model in FIG1 ;

[0037] FIG6 shows a schematic diagram of an application scenario of the semantic model in FIG1 ;

[0038] FIG7 is a schematic flow chart showing a method for processing image data in another embodiment;

[0039] FIG8 shows a schematic diagram of an operation interface for generating multiple defect image samples in FIG7 ;

[0040] FIG9 shows a schematic diagram of an operation interface for batch deformation in FIG8 ;

[0041] FIG10 shows a schematic diagram of an operation interface for batch fusion in FIG8 ;

[0042] FIG11 is a schematic diagram showing an operation interface for generating the defect image sample in FIG1 ;

[0043] FIG12 shows a schematic diagram of another operation interface for generating the defect image sample in FIG1 ;

[0044] FIG13 shows a schematic diagram of another operation interface for generating the defect image sample in FIG1 ;

[0045] FIG14 shows a schematic diagram of another operation interface for generating the defect image sample in FIG1 ;

[0046] FIG15 shows a block diagram of an apparatus for processing image data in one embodiment;

[0047] FIG16 shows a schematic structural diagram of an image data processing device in one embodiment. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0049] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0050] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0052] It should also be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be implemented in an order other than that shown or described.

[0053] In order to enable those skilled in the art to better understand the present disclosure, the application scenarios involved in the present disclosure are first briefly described.

[0054] With the increasing maturity of artificial intelligence technology, industrial inspection has higher and higher requirements for defect image samples in industrial scenes. However, the market is changing rapidly and product iterations are frequent, resulting in the problem that defect image samples are difficult to collect in industrial scenes. Based on this, the present disclosure proposes a method for processing image data, which can automatically generate defect image samples in industrial scenes, and can quickly prepare a large number of defect image samples based on a very small number of defect image samples, so that when the number of defect image samples is small or there are no defect image samples, the industrial inspection project can still be started quickly, which speeds up the convergence cycle of product quality. For scenarios where the number of good products is relatively large, the defect information is relatively small, and the industrial inspection project needs to be started quickly and the delivery cycle is short, this image data processing method can achieve good benefits.

[0055] FIG1 shows a flow chart of a method for processing image data in one embodiment. As shown in FIG1 , the method for processing image data may include the following steps 101 to 104 .

[0056] In step 101, a product defect image corresponding to a target category defect and a product background image of a target product are obtained.

[0057] It's understandable that different products often have different types of cosmetic defects, and these defects correspond to different product defect images. For example, display panels may have defects such as broken pieces, bubbles, scratches, chipped corners, and cracks, and these defects correspond to different product defect images.

[0058] During the implementation process, the product defect map corresponding to the target category defect can be obtained in a variety of ways.

[0059] In some embodiments, an original image of a product containing a target category defect may be obtained; and an image corresponding to the target category defect may be extracted from the original image of the product to obtain a product defect image.

[0060] Referring also to Figure 2, Figure 2 illustrates a demonstration of the generation process of the defect image samples in Figure 1. As shown in Figure 2, from left to right, are the original product image, the product defect image, and the defect image sample. By extracting the product defect image corresponding to the target defect category from the original product image and then augmenting it with the product background image, a more realistic defect image sample can be obtained.

[0061] In other embodiments, an original product image containing defects of the target category and a mask image containing defects of the target category may be obtained, and the original product image and the mask image may be fused to obtain a product defect image.

[0062] Referring also to Figure 3, another illustration of the generation process of the defect image sample in Figure 1 is shown. As shown in Figure 3, from left to right, are the original product image, the mask image, the product background image, and the defect image sample. By fusing the original product image with the mask image to generate the product defect image, and then augmenting it with the product background image, a more realistic defect image sample can be obtained.

[0063] In some embodiments, product defect images corresponding to various types of defects may be pre-saved in a defect database, and then product defect images corresponding to target type defects may be obtained from the defect database.

[0064] It is understandable that the product background image is an image containing product information, and may be a good product image or other images, and the embodiments of the present disclosure are not limited to this.

[0065] During the implementation process, a plurality of product background images of the target product that needs to be industrially inspected may be saved in a memory in advance, and then the product background image of the target product may be obtained from the memory.

[0066] In step 102 , a pre-built semantic model is obtained, wherein the semantic model has the ability to determine the occurrence areas of various types of defects in the product background image.

[0067] It is understood that the occurrence area refers to the area in the product background image where various types of defects occur. Taking the target product as an example, assuming that Type I defects typically occur in Area A and Area B in the product background image, the occurrence areas of Type I defects in the product background image can be Area A and Area B.

[0068] The inventors of this disclosure have discovered that typical product appearance defects often have a spatial relationship with their background. When using these defects to augment defect image samples, the spatial relationship between the product appearance defects and the background can be established first. The disclosed embodiments can model the semantic relationship between the target object and its surroundings to obtain a semantic model. This semantic model is then used to determine the relative position of the product defect image corresponding to the target defect within the product background image, ultimately yielding defect image samples that meet the requirements.

[0069] Refer also to Figure 4, which shows the effects of generating image samples using two different methods. As can be seen from Figure 4, the solution for randomly generating image samples is to randomly paste the target object into the background image. Obviously, this solution is not suitable for scenarios with complex background image content and where the target object appears in a specific position in the background image. In this scenario, when randomly generated image samples are applied to industrial inspection based on deep learning networks, not only will they not help improve deep learning capabilities, but they may even have the opposite effect, that is, reduce the deep learning capabilities of the network. The solution for generating image samples using a semantic model can place the target object in the appropriate position in the background image based on the semantic relationship between the target object and the background image.

[0070] In some embodiments, the semantic model can be constructed by the following steps: obtaining an image sample to be trained, and determining the boundary information and category information of the target object in the image sample to be trained; determining the circumscribed graphic area corresponding to the target object in the image sample to be trained based on the boundary information; obtaining a first image containing the circumscribed graphic area; setting the RGB value of each pixel of the circumscribed graphic area in the first image to a first preset value to obtain a second image; training the initial model based on the category information and the second image to obtain a semantic model.

[0071] During the implementation process, the image samples to be trained can be obtained by processing images of products with various categories of defects produced on the product line, or by processing product defect images that are similar to the defects that may appear in the target product and are screened out from the defect database, or even by processing images that do not contain defects and products.

[0072] When processing images of products with various types of defects produced on a product line, it is necessary to manually segment and label the target objects in the image (the target objects in this scenario are various types of defects) to obtain product defect images, and then augment the product defect images with other images to obtain image samples to be trained.

[0073] When processing product defect images that are screened out from the defect database and are similar to defects that may occur in the target product, the screened product defect images can be directly augmented into other images to obtain image samples to be trained.

[0074] When processing images that do not contain defects and products, it is necessary to manually segment and label the target objects in the image to obtain the target object image, and then augment the target object image into other images to obtain the image sample to be trained.

[0075] Labeling refers to marking the outer contour of the target object using a freeform, polygonal, spline curve or other common geometric shape frame. The boundary information of the target object in the training image sample can be the frame used to label the target object.

[0076] The circumscribed graphic area may be a circumscribed rectangular area, a circumscribed circular area, etc. In order to obtain a better training effect, the circumscribed graphic area may be a minimum circumscribed rectangular area.

[0077] During the implementation process, the first preset value can be 0, 255 or other values, that is, the RGB values ​​of each pixel in the external graphic area in the first image can be set to black, white or other colors to obtain the second image, and then the category information and the second image are input into the initial model for training. When the model converges, a semantic model is obtained.

[0078] It should be noted that the above initial model can be a context-based convolutional neural network.

[0079] In some embodiments, the number of image samples to be trained obtained may be positively correlated with the complexity of the target semantic relationship, wherein the target semantic relationship is the semantic relationship between the third image and the fourth image, the third image is the image corresponding to the target object in the image samples to be trained, and the fourth image is the image in the image samples to be trained other than the third image.

[0080] It can be understood that for images with simple target semantic relationships, the number of acquired training image samples can be relatively small. For example, about 100 training image samples can be acquired. For images with complex target semantic relationships, in order to obtain sufficient image information, the number of acquired training image samples needs to be larger. For example, about 1000 training image samples can be acquired.

[0081] In some embodiments, the circumscribed graphic region in the training image sample can be dilated to obtain a dilated region; an image is extracted from the dilated region in the training image sample to obtain a first image.

[0082] It can be understood that the dilated region is a region larger than the circumscribed graphic region. By extracting an image from the dilated region in the training image sample, more environmental information about the target object can be obtained from the training image sample.

[0083] In the implementation process, the circumscribed graphic region can be randomly dilated, or the length and width of the circumscribed graphic region can be enlarged proportionally, or the circumscribed graphic region can be dilated based on other algorithms. The embodiments of the present disclosure do not limit the dilation method.

[0084] Referring to FIG. 5 together, FIG. 5 shows a schematic diagram of the dilated region in the construction process of the semantic model in FIG. 1. As shown in FIG. 5, when the circumscribed graphic region is the minimum circumscribed rectangle region, the dilated region is the region defined by the dashed box. The image defined by the dilated region can actually be a second image similar to a "hui" character shape. In some embodiments, the area of the dilated region is less than a second preset value to avoid excessive environmental information due to the overly large area of the dilated region, thereby affecting the accuracy of the semantic model.

[0085] Based on the semantic model constructed in the above steps, the occurrence regions of different categories of target objects in different background images can be determined, and thus it also has the ability to determine the occurrence regions of various types of defects in the product background image.

[0086] In step 103, the semantic model is used to determine the target region corresponding to the target category defect in the product background image.

[0087] In the implementation process, the product background image can be processed based on processing steps similar to those in the construction process of the semantic model, and the processed image is input into the semantic model for semantic relationship reasoning to determine the target region corresponding to the target category defect.

[0088] In some embodiments, multiple candidate regions and a dilated region corresponding to each candidate region can be determined in a product background image, wherein the RGB value of each pixel in the candidate region is set to a first preset value; the dilated region corresponding to each candidate region is scored using a semantic model to obtain a first score; and based on the first score of the dilated region corresponding to each candidate region, a target region corresponding to a target category defect is determined from the multiple candidate regions.

[0089] During the implementation process, multiple candidate boxes can be generated in the product background image, and the area defined by the candidate boxes is used as the candidate area. Correspondingly, the expansion area is the area defined by the outer box corresponding to the candidate box. The number of candidate areas can be 100, 200, or other values. One candidate area corresponds to one expansion area, or can correspond to multiple expansion areas.

[0090] It should be noted that during the training process of the semantic model, the RGB value of each pixel in the external graphic area in the first image will be set to the first preset value. When using the semantic model for reasoning, the RGB value of each pixel in the candidate area also needs to be set to the first preset value accordingly.

[0091] It is understandable that after the above processing is performed on the product background image, it can be input into the semantic model to obtain the first score of the expansion area corresponding to each candidate area output by the semantic model.

[0092] In some embodiments, each candidate region may correspond to multiple dilated regions, the first score is used to represent the probability of a target category defect occurring in the corresponding candidate region, and determining the target region corresponding to the target category defect from the multiple candidate regions based on the first score of the dilated region corresponding to each candidate region may include: taking an average of the first scores of the multiple dilated regions corresponding to each candidate region as the second score corresponding to each candidate region; and determining the candidate region with the highest second score as the target region corresponding to the target category defect.

[0093] For example, let's assume that there are three candidate regions, namely, region A, region B, and region C, and each candidate region corresponds to three dilated regions. If the semantic model outputs the first scores of the dilated regions corresponding to region A as 0.4, 0.5, and 0.6, respectively, and the semantic model outputs the first scores of the dilated regions corresponding to region B as 0.45, 0.55, and 0.68, respectively, and the semantic model outputs the first scores of the dilated regions corresponding to region A as 0.60, 0.51, and 0.60, respectively, then the average first score of the three dilated regions corresponding to region A is 0.50, meaning that the second score for region A is 0.50. The average first score of the three dilated regions corresponding to region B is 0.56, meaning that the second score for region B is 0.56. The average first score of the three dilated regions corresponding to region C is 0.57, meaning that the second score for region C is 0.57. Since region C has the highest second score, region C is selected as the target region for the target category defect.

[0094] In other embodiments, each candidate region may correspond to multiple dilated regions, and each dilated region may correspond to multiple first scores, wherein each first score is used to represent the probability of a corresponding candidate category defect occurring in the corresponding candidate region. Based on the first score of the dilated region corresponding to each candidate region, a target region corresponding to a target category defect is determined from the multiple candidate regions, including: taking the average of the first scores of the multiple dilated regions corresponding to each candidate region on each candidate category defect as the second score for the occurrence of the corresponding candidate category defect in each candidate region; determining the candidate category defect with the highest second score corresponding to each candidate region; and, if the candidate category defect with the highest second score is the target category defect, determining the candidate region corresponding to the candidate category defect with the highest second score as the target region corresponding to the target category defect.

[0095] It can be understood that if each expansion region corresponds to a first score for each candidate category defect, then the n expansion regions corresponding to the same candidate region will correspond to n first scores for the candidate category defect. At this time, it is necessary to calculate the average of these n first scores and use the calculated average as the second score for the candidate category defect occurring in the candidate region.

[0096] Taking the candidate regions as candidate region A and candidate region B, with the number of dilated regions corresponding to each candidate region being 2, and taking the target category defect as scratch for example. If the first score of the dilated region A1 corresponding to candidate region A output by the semantic model for the candidate category defect of scratch is 0.4 and the first score for the candidate category defect of crack is 0.5, and the first score of the dilated region A2 corresponding to candidate region A output by the semantic model for the candidate category defect of scratch is 0.3 and the first score for the candidate category defect of crack is 0.6, then calculate the average value of the first score of dilated region A1 for the candidate category defect of scratch, which is 0.35, and use it as the second score for candidate region A to be classified as scratch for the candidate category defect; calculate the average value of the first score of dilated region A1 for the candidate category defect of crack, which is 0.56, and use it as the second score for candidate region A to be classified as crack for the candidate category defect.

[0097] Since the second score for candidate region A to be classified as crack for the candidate category defect is greater than the second score for candidate region A to be classified as scratch for the candidate category defect, the candidate category defect with the highest second score corresponding to candidate region A is determined as crack.

[0098] Determine the candidate category defect with the highest second score corresponding to candidate region B in the same way. If the candidate category defect with the highest second score corresponding to candidate region B is scratch, then determine candidate region B as the target region.

[0099] Refer to Figure 6 together. Figure 6 shows a schematic diagram of an application scenario of the semantic model in Figure 1. As shown in Figure 5, first, label the target object in the original image sample to obtain the bounding box of the target object, obtain the image sample to be trained, and determine the category information of the target object; secondly, according to the bounding box, determine the minimum bounding rectangle region corresponding to the target object in the image sample to be trained, perform random dilation processing on the minimum bounding rectangle region to obtain the dilated region, extract the image from the dilated region in the image sample to be trained to obtain the first image containing the minimum bounding rectangle region, and set the RGB values of each pixel in the minimum bounding rectangle region in the first image to 0 to obtain the second image in the shape of a "hui" character; thirdly, based on the category information and the second image, train the initial model to obtain the semantic model.

[0100] After obtaining the semantic model, multiple candidate regions can be determined in the background image, use the semantic model to determine the target regions corresponding to different categories of target objects from the multiple candidate regions, and fuse the images corresponding to different categories of target objects to the target regions corresponding to the target objects to obtain the image sample for use in the later industrial inspection scenario.

[0101] In step 104 , the product defect image is fused to the target area to obtain a defect image sample of the target product corresponding to the target category defect.

[0102] During the implementation process, images can be fused in a variety of ways to make the final defect image sample smoother.

[0103] In some embodiments, the idea of ​​Poisson fusion can be adopted to determine the first gradient field of the product defect image and the second gradient field of the product background image; the gradient field corresponding to the target area in the second gradient field is replaced by the first gradient field to obtain the third gradient field of the initial fused image; the transformation matrix coefficients are determined based on the third gradient field; and based on the transformation matrix coefficients, defect image samples of the target product corresponding to the target category defects are constructed.

[0104] During implementation, the median difference method can be used to calculate the horizontal and vertical gradient fields of the product defect image. The sum of these two gradient fields is then used to perform a convolution operation on the RGB values ​​of the pixels in the product defect image to obtain the first gradient field of the product defect image. The same method can be used to calculate the second gradient field of the product background image.

[0105] By replacing the gradient field, the third gradient field of the initial fused image P' can be obtained. By taking the partial derivative of the third gradient field, the divergence Dori of the initial fused image P' can be obtained. Construct the equation M×Y=Dori, where Y is the transformation matrix coefficient to be solved, M can be a matrix with a center of -4, a diagonal of 1 and the same size as the image, and Dori is the divergence of the initial fused image P'. The transformation matrix coefficient Y can be obtained by the above equation, and then the transformation matrix coefficient Y is multiplied by the RGB value of each pixel in the initial fused image P' to obtain the RGB value of the pixel of the target fused image P (i.e., the defect image sample), and then the defect image sample is obtained.

[0106] In the above-mentioned image data processing method, by utilizing the semantic model to determine the target area of ​​the product defect image corresponding to the target category defect in the product background image of the target product, not only the automatic generation of defect image samples in the industrial scene is realized, but also the generated defect image samples can meet the requirements of subsequent use. When used for neural network learning, the deep learning ability of the network can be improved.

[0107] FIG7 shows a flow chart of a method for processing image data in another embodiment. As shown in FIG7 , the method for processing image data may further include the following steps:

[0108] Step 701, performing multiple deformation processes on the product defect image to obtain multiple fifth images;

[0109] Step 702 : fuse the plurality of fifth images into the target area respectively to obtain a plurality of defect image samples corresponding to the target category defects of the target product.

[0110] It is understandable that in order to increase the diversity of defect image samples, the product defect images can be deformed to achieve batch preparation of defect image samples. Deformation refers to geometric transformations such as rotation, stretching, and scaling.

[0111] In some embodiments, the number of product background images of the target product is 1, the product defect image is deformed N1 times, and then the N1 fifth images obtained after deformation are fused with 1 product background image to obtain N1 defect image samples, where N1 is a positive integer greater than 1.

[0112] In some embodiments, the image data processing method may further include the following steps: obtaining an original image of a product containing defects of a target category selected by a user based on a first control; generating a mask image containing defects of the target category based on the original image of the product in response to the user's operation of a second control; fusing the original image of the product with the mask image to obtain a product defect image; and performing multiple deformation processing steps on the product defect image in response to the user's operation of a third control.

[0113] Figure 8 shows a schematic diagram of an operation interface for generating multiple defect image samples in Figure 7. As shown in Figure 8, the user can click the first control (i.e., open the image control) to select the original product image containing the target category defect, then click the second control (i.e., generate a mask image containing the target category defect, and then click the third control (i.e., batch deformation control) to perform N-fold deformation processing on the product defect images corresponding to the target category defect.

[0114] In some embodiments, in response to the user's operation on the third control, a batch deformation interface may be displayed; and the product defect image may be deformed multiple times according to the deformation strength and deformation quantity input by the user in the batch deformation interface.

[0115] Figure 9 shows a schematic diagram of an operation interface for the batch deformation process in Figure 8. As shown in Figure 9, the user can select the save path, deformation intensity, and deformation quantity. The deformation quantity determines the number of defect image samples ultimately generated. After making these selections, click the batch deformation control and the progress of the batch deformation process can be viewed in real time through the progress bar. After the batch deformation process is completed, the software can create an "image" directory to store the deformed product defect images. Clicking the result filter control will open the "image" folder, where the user can view the deformation effect and delete images with unsatisfactory deformation effects. In other embodiments, after the batch deformation process is completed, the software can also create two directories, "image" and "mask," to store product images containing deformed product defect images and mask images corresponding to the deformed product defect images, respectively. Clicking the result filter control will open the "image" folder, where the user can view the deformation effect and delete images with unsatisfactory deformation effects. The user can then clean up the corresponding "mask" folder based on the filtered "image" folder, ensuring that the images in the "image" and "mask" folders correspond to each other.

[0116] Figure 10 shows a schematic diagram of an operational interface for the batch fusion shown in Figure 8. The user can click the batch fitting control in the operational interface shown in Figure 8 and select the good path, result path, and / or fusion location in the pop-up operational interface shown in Figure 10. The good path is the storage path for the product background image, and the result path is the storage path for the defective image samples. The fusion location can be automatically determined based on the semantic model or selected by the user. The number of defective image samples generated by the fusion is equal to the product of the number of deformations and the number of product background images.

[0117] In some embodiments, the number of product background images of the target product is M1, the product defect image is deformed N2 times, and then the N2 fifth images obtained after deformation are fused with the M1 product background images respectively, to obtain N2×M1 defect image samples, where M1 and N2 are both positive integers greater than 1.

[0118] In some embodiments, the number of product defect images corresponding to the target category defects is N3, and the number of product background images of the target product is M2. The N3 product defect images are deformed K times, and then the N3×K fifth images obtained after deformation are fused with the M2 product background images respectively, and N3×K×M2 defect image samples can be obtained, where N3, K and M2 are all positive integers greater than 1.

[0119] During the implementation process, users can prepare defect image samples and mask images corresponding to product defect images in the defect image samples in advance, and store them in the image folder and mask folder respectively, and the defect image samples and mask images correspond one to one.

[0120] The user can select the saving path, deformation intensity, and deformation amount based on the same or similar operation interface as FIG9 to obtain the product defect image after deformation.

[0121] Users can also select a defect path, a qualified path, a result path, and / or a fusion location using an interface similar to or identical to Figure 10. The fusion location can be automatically determined based on the semantic model or user-selected. The number of defect image samples generated by fusion is equal to the product of the number of defect image samples, the number of deformations, and the number of product background images. Users can also view, filter, and organize the fused defect image samples.

[0122] By preparing defect image samples in batches, the diversity of defect image samples is increased and the preparation efficiency of defect image samples is improved.

[0123] In some embodiments, the image data processing method may further include the step of modifying the color of the product defect image according to the color of the product background image when the color of the product defect image is inconsistent with the color of the product background image.

[0124] It is understood that the color of the product defect image may be different from the color of the product background image. When the color of the product defect image is grayscale, the color of the product background image may be grayscale or color, and vice versa. In order to better integrate the product defect image with the target area of ​​the product background image, the color of the product defect image can be modified.

[0125] FIG11 shows a schematic diagram of an operation interface for generating the defect image sample in FIG1 . As shown in FIG11 , when the color of the product background image and the color of the product defect image are both grayscale, the color of the product defect image may not be modified.

[0126] FIG12 shows another schematic diagram of an operation interface for generating the defect image sample in FIG1 . As shown in FIG12 , when the color of the product defect image is grayscale and the color of the product background image is color, the color of the product defect image can be modified to color.

[0127] FIG13 shows another schematic diagram of an operation interface for generating the defect image sample in FIG1 . As shown in FIG13 , when the color of the product defect image is in color and the color of the product background image is in grayscale, the color of the product defect image can be modified to grayscale.

[0128] FIG14 shows another schematic diagram of an operation interface for generating the defect image sample in FIG1 . As shown in FIG14 , when the color of the product background image and the color of the product defect image are both in color, the color of the product defect image may not be modified.

[0129] By adaptively modifying the color of the product defect image according to the color of the product background image, the product defect image can be better integrated into the target area of ​​the product background image.

[0130] In some embodiments, the defect image samples may be used for defect detection of a target product.

[0131] Understandably, in the industrial inspection industry, there are multiple methods for defect detection of target products, with the most common method being the use of deep learning networks. After obtaining defect image samples, they can be applied to the deep learning network. Because the various defects in these defect image samples are located in appropriate locations, they meet the deep learning network's training data requirements and can therefore enhance the network's deep learning capabilities.

[0132] It should be noted that the current difficulty in collecting defect image samples in industrial scenarios can be solved by the image data processing method proposed in this disclosure. This method can automatically generate defect image samples corresponding to various types of defects in target products, and can quickly and massively produce defect image samples. This allows industrial inspection of target products to be initiated based on the generated defect image samples, achieving the goal of accelerating the launch of industrial inspection projects.

[0133] The following describes an embodiment of the device disclosed herein, which can be used to execute the image data processing method described in the above embodiment of the present disclosure. For details not disclosed in the embodiment of the device disclosed herein, please refer to the embodiment of the image data processing method described in the above embodiment of the present disclosure.

[0134] Referring to FIG15 , a block diagram of an image data processing apparatus according to an embodiment of the present disclosure is shown. As shown in FIG15 , the image data processing apparatus according to an embodiment of the present disclosure includes: an image acquisition unit 1501, a model acquisition unit 1502, a target region determination unit 1503, and an image fusion unit 1504. The image acquisition unit 1501 is configured to acquire a product defect image corresponding to a target category defect and a product background image of a target product; the model acquisition unit 1502 is configured to acquire a pre-built semantic model, wherein the semantic model has the ability to determine the occurrence area of ​​various categories of defects in the product background image; the target region determination unit 1503 is configured to use the semantic model to determine a target region corresponding to the target category defect in the product background image; and the image fusion unit 1504 is configured to fuse the product defect image into the target region to obtain a defect image sample of the target product corresponding to the target category defect.

[0135] In some embodiments, the model acquisition unit 1502 is also used to obtain image samples to be trained, and determine the boundary information and category information of the target object in the image samples to be trained; determine the circumscribed graphic area corresponding to the target object in the image samples to be trained based on the boundary information; obtain a first image containing the circumscribed graphic area; set the RGB value of each pixel of the circumscribed graphic area in the first image to a first preset value to obtain a second image; and train the initial model based on the category information and the second image to obtain a semantic model.

[0136] In some embodiments, the model acquisition unit 1502 is further used to acquire a first image containing an external graphic area, including: dilating the external graphic area in the image sample to be trained to obtain an expanded area; extracting an image from the expanded area in the image sample to be trained to obtain a first image.

[0137] In some embodiments, the area of ​​the expansion region is smaller than a second preset value.

[0138] In some embodiments, the number of image samples to be trained is positively correlated with the complexity of the target semantic relationship, wherein the target semantic relationship is the semantic relationship between the third image and the fourth image, the third image is the image corresponding to the target object in the image samples to be trained, and the fourth image is the image in the image samples to be trained other than the third image.

[0139] In some embodiments, the target area determination unit 1503 is further configured to determine multiple candidate areas and a dilated area corresponding to each candidate area in the product background image, wherein the RGB value of each pixel in the candidate area is set to a first preset value; score the dilated area corresponding to each candidate area using a semantic model to obtain a first score; and determine a target area corresponding to the target category defect from the multiple candidate areas based on the first score of the dilated area corresponding to each candidate area.

[0140] In some embodiments, each candidate region corresponds to multiple dilated regions, and the first score is used to represent the probability of a target category defect occurring in the corresponding candidate region. The target region determination unit 1503 is further configured to calculate an average of the first scores of the multiple dilated regions corresponding to each candidate region as the second score corresponding to each candidate region; and determine the candidate region with the highest second score as the target region corresponding to the target category defect.

[0141] In some embodiments, each candidate region corresponds to multiple dilated regions, and each dilated region corresponds to multiple first scores, where each first score is used to represent the probability of a corresponding candidate category defect occurring in the corresponding candidate region. The target region determination unit 1503 is further configured to use the average of the first scores of the multiple dilated regions corresponding to each candidate region for each candidate category defect as the second score for the occurrence of each candidate category defect in each candidate region; based on the second scores for the occurrence of each candidate category defect in each candidate region, determine the candidate category defect with the highest second score as the target category defect corresponding to each candidate region; and determine the candidate region corresponding to the target category defect as the target region corresponding to the target category defect.

[0142] In some embodiments, the image acquisition unit 1501 is further used to acquire an original image of a product containing defects of the target category and a mask image containing defects of the target category; and fuse the original image of the product with the mask image to obtain a product defect image.

[0143] In some embodiments, the image acquisition unit 1501 is further configured to acquire an original image of a product containing a target category defect; and extract an image corresponding to the target category defect from the original image of the product to obtain a product defect image.

[0144] In some embodiments, the image fusion unit 1504 is further used to determine a first gradient field of the product defect image and a second gradient field of the product background image; replace the gradient field corresponding to the target area in the second gradient field with the first gradient field to obtain a third gradient field of the initial fused image; determine the transformation matrix coefficients based on the third gradient field; and construct a defect image sample of the target product corresponding to the target category defect based on the transformation matrix coefficients.

[0145] In some embodiments, the image data processing device also includes a batch preparation unit (not shown) for performing multiple deformation processing on the product defect image to obtain multiple fifth images; and fusing the multiple fifth images into the target area respectively to obtain multiple defect image samples of the target product corresponding to the target category defects.

[0146] In some embodiments, the batch preparation unit is also used to obtain an original image of a product containing defects of a target category selected by a user based on a first control; in response to the user's operation on a second control, generate a mask image containing defects of the target category based on the original image of the product; fuse the original image of the product with the mask image to obtain a product defect image; and in response to the user's operation on a third control, perform the steps of performing multiple deformation processing on the product defect image.

[0147] In some embodiments, the batch preparation unit is also used to display a batch deformation interface in response to the user's operation on the third control; and execute the steps of performing multiple deformation processing on the product defect image according to the deformation strength and deformation quantity input by the user in the batch deformation interface.

[0148] In some embodiments, the image data processing device also includes an image fitting unit (not shown) for modifying the color of the product defect image according to the color of the product background image when the color of the product defect image is inconsistent with the color of the product background image.

[0149] In some embodiments, the defect image samples are used for defect detection of a target product.

[0150] Based on the same concept, an embodiment of the present disclosure also provides an image data processing device. Referring to Figure 16, a structural schematic diagram of the image data processing device in an embodiment of the present disclosure is shown. The image data processing device includes one or more memories 1604, one or more processors 1602, and at least one computer program (computer program instruction) stored on the memory 1604 and executable on the processor 1602. When the processor 1602 executes the computer program, the method described above is implemented.

[0151] In Figure 16, a bus architecture (represented by bus 1600) is shown. Bus 1600 can include any number of interconnected buses and bridges. Bus 1600 links various circuits together, including one or more processors represented by processor 1602 and memory represented by memory 1604. Bus 1600 can also link various other circuits together, such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and, therefore, will not be described further herein. Bus interface 1605 provides an interface between bus 1600 and receiver 1601 and transmitter 1603. Receiver 1601 and transmitter 1603 can be the same component, namely a transceiver, which provides a unit for communicating with various other devices over a transmission medium. Processor 1602 is responsible for managing bus 1600 and general processing, while memory 1604 can be used to store data used by processor 1602 when performing operations.

[0152] Based on the same concept, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a processor, the processor is prompted to implement the steps of the method described above.

[0153] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code. Other examples and implementations are within the scope and spirit of the present disclosure and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hardwiring, or a combination of any of these. Furthermore, the functional units may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0154] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0155] The units described as separate components may or may not be physically separate, and the components of the control device may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store computer program instructions, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0157] The foregoing description is merely an embodiment of the present disclosure and is not intended to limit the present disclosure. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present disclosure are intended to be within the scope of the claims of the present disclosure.

Claims

1. A method for processing image data, comprising: Obtain a product defect image corresponding to a target category defect and a product background image of the target product; Obtaining a pre-built semantic model, wherein the semantic model has the ability to determine occurrence areas of various types of defects in the product background image; Determining a target region corresponding to the target category defect in the product background image using the semantic model; and The product defect image is fused to the target area to obtain a defect image sample of the target product corresponding to the target category defect.

2. The method for processing image data according to claim 1, wherein: The semantic model is constructed by the following steps: Acquire a to-be-trained image sample, and determine boundary information and category information of a target object in the to-be-trained image sample; Determining, according to the boundary information, a circumscribed graphic area corresponding to the target object in the image sample to be trained; Acquire a first image including the circumscribed graphic area; Setting the RGB value of each pixel in the circumscribed graphic area in the first image to a first preset value to obtain a second image; as well as Based on the category information and the second image, an initial model is trained to obtain the semantic model.

3. The method for processing image data according to claim 2, wherein: The step of acquiring the first image including the circumscribed graphic area comprises: Performing expansion processing on the circumscribed graphic area in the image sample to be trained to obtain an expansion area; and An image is extracted from the dilated area in the image sample to be trained to obtain the first image.

4. The method for processing image data according to claim 3, wherein: The area of ​​the expansion region is smaller than a second preset value.

5. The method for processing image data according to claim 2, wherein: The number of image samples to be trained obtained is positively correlated with the complexity of the target semantic relationship, wherein the target semantic relationship is the semantic relationship between the third image and the fourth image, the third image is the image corresponding to the target object in the image samples to be trained, and the fourth image is the image in the image samples to be trained except the third image.

6. The method for processing image data according to any one of claims 2 to 5, wherein: The determining the target area corresponding to the target category defect in the product background image by using the semantic model includes: Determining a plurality of candidate regions and a dilation region corresponding to each of the candidate regions in the product background image, wherein the RGB value of each pixel in the candidate region is set to the first preset value; Scoring the dilated region corresponding to each candidate region by using the semantic model to obtain a first score; and According to the first score of the dilated region corresponding to each of the candidate regions, a target region corresponding to the target category defect is determined from the plurality of candidate regions.

7. The method for processing image data according to claim 6, wherein: There are multiple dilation regions corresponding to each candidate region, the first score is used to characterize the probability of the target category defect occurring in the corresponding candidate region, and determining the target region corresponding to the target category defect from the multiple candidate regions according to the first score of the dilation region corresponding to each candidate region includes: taking an average of the first scores of the plurality of dilated regions corresponding to each of the candidate regions as a second score corresponding to each of the candidate regions; and The candidate region with the second highest score is determined as the target region corresponding to the target category defect.

8. The method for processing image data according to claim 6, wherein: There are multiple dilated regions corresponding to each of the candidate regions, and there are multiple first scores corresponding to each of the dilated regions, wherein each first score is used to characterize the probability of a defect of a corresponding candidate category occurring in the corresponding candidate region. Determining a target region corresponding to the defect of the target category from the multiple candidate regions according to the first score of the dilated region corresponding to each of the candidate regions includes: Taking the average of the first scores of the multiple expansion regions corresponding to each candidate region on each candidate category defect as the second score of the corresponding candidate category defect occurring in each candidate region; Determine a candidate category defect with the second highest score corresponding to each of the candidate regions; and When the defect in the candidate category with the second highest score is the defect in the target category, the candidate region corresponding to the defect in the candidate category with the second highest score is determined as the target region corresponding to the defect in the target category.

9. The method for processing image data according to claim 1, wherein: The step of obtaining a product defect image corresponding to a target category defect includes: Acquire an original image of a product containing defects of the target category and a mask image containing defects of the target category; and The product original image is fused with the mask image to obtain the product defect image.

10. The method for processing image data according to claim 1, wherein: The step of obtaining a product defect image corresponding to a target category defect includes: Acquire an original image of a product containing defects of the target category; and An image corresponding to the target category defect is extracted from the original product image to obtain the product defect image.

11. The method for processing image data according to claim 9 or 10, wherein: The step of fusing the product defect image to the target area to obtain a defect image sample of the target product corresponding to the target category defect includes: Determine a first gradient field of the product defect image and a second gradient field of the product background image; Replacing the gradient field corresponding to the target area in the second gradient field with the first gradient field to obtain a third gradient field of the initial fused image; Determining transformation matrix coefficients according to the third gradient field; and According to the transformation matrix coefficients, a defect image sample of the target product corresponding to the target category defect is constructed.

12. The method for processing image data according to claim 1, further comprising: Performing multiple deformation processing on the product defect image to obtain multiple fifth images; as well as The plurality of fifth images are respectively fused to the target area to obtain a plurality of defect image samples of the target product corresponding to the target category defects.

13. The method for processing image data according to claim 12, further comprising: Acquire an original image of a product containing defects of the target category selected by a user based on a first control; In response to the user's operation on the second control, generating a mask image containing the target category defect based on the product original image; Fusion the product original image with the mask image to obtain the product defect image; as well as In response to the user's operation on the third control, the step of performing multiple deformation processing on the product defect image is executed.

14. The method for processing image data according to claim 13, wherein: The step of performing multiple deformation processing on the product defect image in response to the user's operation on the third control comprises: In response to the user's operation on the third control, displaying a batch deformation interface; and The step of performing multiple deformation processing on the product defect image is performed according to the deformation strength and deformation quantity input by the user in the batch deformation interface.

15. The method for processing image data according to claim 1, further comprising: In the case that the color of the product defect image is inconsistent with the color of the product background image, the color of the product defect image is modified according to the color of the product background image.

16. The method for processing image data according to claim 1, wherein the defect image sample is used for defect detection of the target product.

17. A device for processing image data, comprising: An image acquisition unit, used to acquire a product defect image corresponding to a target category defect and a product background image of a target product; A model acquisition unit, used to acquire a pre-built semantic model, wherein the semantic model has the ability to determine the occurrence area of ​​various types of defects in the product background image; a target region determining unit, configured to determine a target region corresponding to the target category defect in the product background image by using the semantic model; and An image fusion unit is used to fuse the product defect image into the target area to obtain a defect image sample of the target product corresponding to the target category defect.

18. An image data processing device, comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the steps of the method according to any one of claims 1 to 16 are implemented.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, prompt the processor to implement the steps of the method according to any one of claims 1 to 16.

Images