Defect detection model training method, device and defect detection method and device

Abstract

The present application discloses a defect detection model training method, device and defect detection method, device. The defect detection model training method of the present application includes: obtaining a product image, wherein the product image includes an annular area to be detected; extracting the annular area in the product image, and dividing the regional morphology of the annular area into a corner morphology and an edge morphology; performing regional segmentation and splicing on the annular area according to the corner morphology and the edge morphology to obtain a plurality of sub-images; using each sub-image to train a defect detection model corresponding to the sub-image. The defect detection of the present application includes: obtaining a product image to be detected, extracting the annular area in the product image, and performing regional segmentation and splicing on the annular area according to the corner morphology and the edge morphology of the annular area to obtain a plurality of sub-images; inputting each sub-image into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a defect detection model training method and device, and a defect detection method and device. Background Art

[0002] In the product imaging process of industrial manufacturing, due to the accuracy of mechanical positioning, unstable production line processes, changes in environmental factors such as lighting, the products produced have various defects of different forms. In the traditional precision manufacturing industry, product defects are generally classified and detected manually. In recent years, traditional manufacturing has gradually shifted to intelligent manufacturing. Object detection based on deep learning has the advantages of low detection cost, stable detection results, less labor demand, and easy maintenance. It has gradually become an important part of the transformation from traditional manufacturing to automation.

[0003] In the related art, when defect detection is performed on a product image based on a defect detection model, there is a problem of poor accuracy of defect detection results for key areas due to inconsistent detection standards for the entire product in the product image. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, and the following technical solution is specially proposed to improve the defect detection accuracy by obtaining the annular area that needs to be focused on from the product image for defect detection.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In one aspect of an embodiment of the present application, a defect detection model training method is provided, comprising: acquiring a product image, wherein the product image includes an annular area to be focused on detection;

[0007] Extracting the annular region in the product image, and dividing the region shape of the annular region into a corner shape and a sideline shape;

[0008] Performing region segmentation and splicing on the annular region according to the corner shape and edge shape to obtain a plurality of sub-graphs;

[0009] Each sub-graph is used to train the defect detection model corresponding to the sub-graph.

[0010] Another aspect of the embodiment of the present application further provides a defect detection method, including: acquiring a product image to be detected, extracting an annular region in the product image, and performing region segmentation and splicing on the annular region according to the corner shape and edge shape of the annular region to obtain multiple sub-images;

[0011] Each sub-image is input into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-image; wherein the defect detection model is trained by the above-mentioned defect detection model training method.

[0012] According to another aspect of the embodiments of the present application, a defect detection model training device is provided for implementing the above-mentioned defect detection model training method.

[0013] According to another aspect of the embodiments of the present application, a defect detection device is provided for implementing the above-mentioned defect detection method.

[0014] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0015] For scenarios where the defect detection standards on the entire product surface are inconsistent and the key area to be inspected is a ring-shaped area, the embodiments of the present application extract the ring-shaped area of ​​focus in the product image. Since the defect detection standards in the ring-shaped area are consistent, as long as the defect types in the product image are relatively comprehensive, a defect detection model suitable for the ring-shaped area can be quickly trained with the help of a small amount of training set data. In addition, by segmenting and splicing the ring-shaped area, the interference of the larger redundant area in the middle of the ring-shaped area can be removed, thereby improving the model training speed and model detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 This is a flow chart of the defect detection model training method shown in the embodiment of the present application;

[0018] Figure 2 A schematic diagram of the Unet network structure shown in an embodiment of the present application;

[0019] Figure 3 A mask schematic diagram shown in an embodiment of the present application;

[0020] Figure 4 This is a schematic diagram of annular area segmentation shown in an embodiment of the present application;

[0021] Figure 5 This is a schematic diagram of the first sub-diagram shown in the embodiment of the present application;

[0022] Figure 6 This is a schematic diagram of a second sub-image before splicing processing shown in an embodiment of the present application;

[0023] Figure 7A schematic diagram of a second sub-image after splicing shown in an embodiment of the present application;

[0024] Figure 8 A schematic diagram of a defect detection process shown in an embodiment of the present application;

[0025] Fig. 9 This is a flow chart of a defect detection method shown in an embodiment of the present application;

[0026] Fig.10 This is a structural block diagram of a defect detection model training device shown in an embodiment of the present application;

[0027] Fig.11 It is a structural block diagram of a defect detection device shown in an embodiment of the present application;

[0028] Fig.12 It is a schematic diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0030] The inventors of the present application discovered during their research and practice of product defect detection that some areas of a product have a greater impact on performance and efficiency and a lower tolerance for defects. For example, the magnetic gap area of ​​a magnetic circuit product has a lower tolerance for defects than other areas. If a global product image including this area and other areas of the product is used as the image to be inspected for defect detection, the defect detection accuracy of this area will be affected.

[0031] In response to the above problems, the inventors of the present application came up with the idea that in application scenarios where the defect detection standards on the entire product surface are inconsistent and the key area to be inspected is a ring-shaped area, the ring-shaped area can be segmented and spliced ​​to obtain a stitched image that can cover the original ring-shaped area with the smallest area. The characteristics of the stitched image where defects are more concentrated and defect deformations are smaller can be used to improve the accuracy and efficiency of defect detection.

[0032] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0033] An embodiment of the present application provides a defect detection model training method, Figure 1 This is a flow chart of the defect detection model training method shown in the embodiment of the present application, such as Figure 1As shown, the method at least includes the following steps S110 to S140:

[0034] Step S110, obtaining a product image, wherein the product image includes a ring-shaped area to be focused on.

[0035] When training the defect detection model in the embodiment of the present application, a certain number of product images need to be obtained as original training samples. The specific method for obtaining product images can be flexibly set by those skilled in the art according to actual needs and is not specifically limited here.

[0036] Step S120: extracting a ring-shaped area in the product image, and dividing the region shape of the ring-shaped area into a corner shape and a sideline shape.

[0037] After obtaining the original product image, it is necessary to perform region detection on the product image to determine whether there is a ring region in the product image that has a lower tolerance for defects than other regions.

[0038] In the case where such an annular area exists, the regional morphology of the annular area is further divided into corner morphology and edge morphology, so that the image of the annular area can be segmented and spliced ​​according to the corner morphology and edge morphology, so as to achieve the purpose of removing interference from a large redundant area in the middle of the annular area, and avoid wasting computing power and reasoning time when learning the defect detection model.

[0039] Step S130 , segmenting and splicing the annular region according to the corner shape and the edge shape to obtain a plurality of sub-images.

[0040] In order to remove the larger redundant area in the middle of the annular area, this embodiment performs region segmentation on the annular area based on the corner shape and the edge shape, and segments the annular area into corner areas that conform to the corner shape and edge areas that conform to the edge shape. The corner areas and edge areas segmented based on this method can completely cover the original annular area.

[0041] Step S140: Use each sub-graph to train a defect detection model corresponding to the sub-graph.

[0042] After obtaining a sub-graph with more concentrated defects and consistent detection standards, the sub-graph can be input into the corresponding defect detection model for training. After the detection effect reaches the expected level, the training can be stopped.

[0043] It should be noted that in order to improve the accuracy of the defect detection model, this embodiment constructs a corresponding defect detection model for each sub-graph. For example, when two sub-graphs are spliced, two defect detection models are constructed, and the defect detection models are trained using sub-graphs of corresponding types.

[0044] exist Figure 1In the illustrated embodiment, an annular area of ​​focus in the product image is extracted. Since the defect detection standard in the annular area is consistent, as long as the defect types in the product image are relatively comprehensive, a defect detection model suitable for the annular area can be quickly trained with the help of a small amount of training set data. In addition, by segmenting and splicing the annular area, the interference of a large redundant area in the middle of the annular area can be removed, thereby improving the model training speed and model detection efficiency.

[0045] In one embodiment of the present application, extracting the annular area in the product image includes: processing the product image using a trained image segmentation network, generating a mask image corresponding to the annular area through the image segmentation network, and extracting the annular area from the product image according to the mask image.

[0046] When extracting the annular area, this embodiment can use the image segmentation algorithm to extract the annular area to be focused on from the product image. It is preferred to use a shallower Unet network structure. The Unet network can quickly complete the fitting task of the specified area of ​​the image on a small sample data of dozens of images.

[0047] When using the Unet network to extract the annular region, the Unet network needs to be trained using sample data. The sample data here refers to: the original sample image and its corresponding mask image. The labeling tool labelme can be used to mark the annular region to be detected in the original sample image, generate a json annotation file, and generate a mask image of the same size as the original sample image based on the json annotation file. The grayscale value of the annular region to be detected in the mask image is 1, and the grayscale value of other positions is 0. Therefore, each sample data in the sample set has a corresponding mask image.

[0048] When the Unet network is trained, the original sample image and the corresponding mask image are input into the Unet network at the same time, such as Figure 2 As shown in the figure, the first half of the Unet network extracts features from the original sample image through 3x3 convolution kernels. The number of the first two convolution kernels is 64. After each two convolution operations, the number of convolution kernels will double. Every two convolution kernels will cooperate with a 2x2 maximum pooling operation. After each maximum pooling, the length and width of the feature map will be halved. The number of convolution kernels in the second half starts to decrease from 1024. After each convolution operation, the number of convolution kernels is reduced by half. With upsampling, the length and width of the feature map will double after each upsampling.

[0049] This embodiment also adds a new splicing (copy and crop) feature fusion method in the second half of the Unet network, and performs channel-level splicing on the feature maps collected in the first half, that is, the size and number of channels of the spliced ​​feature map are the sum of the two feature maps to be spliced.

[0050] Afterwards, the mask image generated by the second half of the Unet network is compared with the mask image corresponding to the original sample image, the loss function is calculated, and the model parameters of the Unet network are continuously corrected.

[0051] Here, the loss function uses the Dice-coefficient loss function. Compared with the original Unet loss function, the Dice-coefficient loss function is more effective for the category imbalance problem, and can quickly measure the similarity between the mask image generated by the Unet network and the original mask image.

[0052] After the Unet network is trained, the acquired product image can be input into the Unet network, and the mask image corresponding to the annular area can be obtained through the output of the Unet network. Since the size of the output mask image is not necessarily the same as the size of the original product image, before extracting the annular area based on the mask image, it is necessary to adjust the image size of the mask image to obtain the adjusted mask image. The size of the adjusted mask image is the same as the size of the product image.

[0053] In this way, the annular area of ​​the pixel position in the adjusted mask image can be marked out, and the annular area of ​​the pixel position in the product image can be obtained. Figure 3 The annular area shown can be extracted from the product image using image clipping technology.

[0054] from Figure 3 It can be seen that the extracted annular area includes a large redundant area in the middle, which not only wastes the computing power and reasoning time when learning the defect detection model, but also is not conducive to defect detection in the annular area. To address this problem, the embodiment of the present application divides the annular area according to the corner shape and edge shape of the annular area to obtain four corner areas and four edge areas, and splices the four corner areas into Figure 5 The first subgraph shown, and the four edge regions are spliced ​​as shown Figure 7 The second sub-figure is shown.

[0055] When segmenting the annular region, four corner regions that conform to the corner form are extracted according to the first size ratio, two first-class borderline regions that conform to the borderline form are extracted according to the second size ratio, and two second-class borderline regions that conform to the borderline form are extracted according to the third size ratio, wherein the first size ratio, the second size ratio, and the third size ratio indicate the ratio of the length and width of the region extracted according to the first, second, or third size ratio to the annular region in the product image. Taking the extraction of the corner region as an example, the length of the corner region is proportional to the length of the annular region, and the width of the corner region is proportional to the width of the annular region, and these two ratio values ​​are the first size ratio.

[0056] In practical applications, the annular area is also segmented according to the size information of the product defect, so that the extracted corner area and edge area have overlapping areas, and the overlapping area can be set according to the size of the minimum product defect on the annular area.

[0057] by Figure 4 Taking the segmentation scenario shown in FIG. 1 as an example, the length and width of the corner area are set to 1 / 5 and 1 / 4 of the length and width of the original annular area, and the size of the overlapping area is set to the size of the smallest product defect on the annular area. In this way, the following can be obtained: Figure 4 The corner areas and edge areas shown are, in clockwise direction, the four corner areas of the marked annular area are areas 1, 2, 3, and 4 respectively; in the direction of the long side of the annular area, the two first-type edge areas of the marked annular area are areas 5 and 6 respectively; in the direction of the wide side of the annular area, the two second-type edge areas of the marked annular area are areas 7 and 8 respectively.

[0058] Since the four corner areas of the annular area constitute Figure 3 The main reason for the large redundant area is that the four corner areas are spliced ​​together to effectively reduce the area of ​​the redundant area. When splicing, the four corner areas are spliced ​​into a ring area according to their original orientation in the ring area. Figure 4 , the four corner areas 1, 2, 3, and 4 are spliced ​​clockwise to form a first sub-image that is nearly a square.

[0059] Since the segmented edge regions include the first type edge regions and the second type edge regions, the image morphology of the two first type edge regions is completely the same, and the image morphology of the two second type edge regions is also completely the same, therefore, before the edge regions are spliced, the two first type edge regions can be spliced ​​to obtain the first edge splicing region, and the two second type edge regions can be spliced ​​to obtain the second edge splicing region. Figure 5 and Figure 6 As shown, the length of the first type of edge line area is greater than that of the second type of edge line rectangle, and the long sides of the first type of edge line area and the second type of edge line are in different directions.

[0060] Based on this, it is necessary to adjust the length and direction of the two side line splicing areas obtained by splicing the side line areas to be consistent. On the one hand, the length directions of the first side line splicing area and the second side line splicing area are processed to be the same direction, for example, the first side line splicing area is rotated by a preset angle so that it is the same as the length direction of the second side line splicing area; on the other hand, the lengths of the first side line splicing area and the second side line splicing area are adjusted and aligned, for example, the first side line splicing area is shortened, stretched or spliced ​​so that it is aligned in length with the second side line splicing area.

[0061] In one embodiment, after the length directions of the first edge line stitching area and the second edge line stitching area are processed to be in the same direction, the length of the first edge line stitching area is set to be the same as the length of the second edge line stitching area, and the first edge line stitching area and the second edge line stitching area with the same length and length direction are regionally stitched to obtain a second sub-image.

[0062] refer to Figure 7 , this embodiment Figure 4 After the sideline areas 7 and 8 are joined, they are rotated 90 degrees counterclockwise and then set at the bottom of the sideline areas 5 and 6. Because the length of the sideline areas 5 and 6 is greater than that of the sideline areas 7 and 8, the height of the sideline areas 5 and 6 remains unchanged and the length is resized to the length of the sideline areas 7 and 8; then they are joined with the sideline areas 7 and 8 to form the final second sub-area. Figure 2 In the process of splicing the second sub-image, the length of the two edge splicing areas is adjusted to be flush, which will cause a small amount of deformation of the defect.

[0063] Based on the second sub-image stitching method for shortening the length of the first edge region, if there is a defect in the first type of edge stitching region, the width of the defect remains unchanged and the length is shortened, that is, the defect will undergo a small amount of deformation. Figure 6 and Figure 7 The defect indicated by the circle in the border area 6 has a constant width but a shortened length, whereas the different defects indicated by the diamonds in the border area 8 do not deform.

[0064] Of course, in practical applications, the height of the edge areas 7 and 8 can be kept unchanged and the length can be resized to the length of the edge areas 5 and 6, or a blank area of ​​equal width can be spliced ​​at one end of the edge areas 7 and 8 to align the length direction of the two edge splicing areas through the blank area. Generally, when splicing sub-graphs, the final spliced ​​sub-layer should be as close to a square as possible, so that the deformation of the defect will be smaller during the model learning process.

[0065] It should be noted that in the above embodiment, four edge regions are spliced ​​into one sub-image. In practical applications, two first-class edge regions can be spliced ​​into one sub-image, and two second-class edge regions can be spliced ​​into another sub-image. The specific splicing scheme can refer to the edge region aspect ratio. For example, when the edge region aspect ratio is 2:1, the first-class edge regions can be spliced ​​into one sub-image.

[0066] The first sub-graph and the second sub-graph obtained by the above embodiment have at least the following advantages:

[0067] (1) The first sub-image and the second sub-image can cover the annular area to be detected with the smallest splicing area, while avoiding the interference of the larger black redundant area in the middle;

[0068] (2) The same defect types in the two sub-graphs are more concentrated. For example, the same defect types in the first sub-graph (such as impurities, hair fibers, etc.) are more concentrated, and several unique defect types in the second sub-graph (such as washer crush, edge magnetic damage, etc.) are also more concentrated, which is conducive to the centralized training and learning of the defect detection model;

[0069] (3) Compared with the original product image size of 2448x2048, the sub-image size is smaller, only about 400x400 pixels, which saves more time during training and prediction;

[0070] (4) The aspect ratio of the two sub-images obtained is close to 1, so that the deformation of the defect during training and defect detection is minimal, the impact on the defect morphology is minimal, and the defect detection accuracy can be improved.

[0071] After obtaining the first sub-graph corresponding to the corner shape of the annular area and the second sub-graph corresponding to the edge shape of the annular area, the first sub-graph can be used to train the first defect detection model, and the second sub-graph can be used to train the second defect detection model. Here, the defect detection model can be a yolov4 network structure, and of course it can be other forms of network structures. It can be flexibly selected in actual applications, and the embodiments of the present application do not make specific limitations.

[0072] The embodiment of the present application performs annular area recognition, extraction, segmentation, splicing and other processing on the original product image, and constructs a training set based on the sub-images obtained by the processing, so that a high-precision and high-efficiency defect detection model can be trained quickly when the defect types in the training set are comprehensive.

[0073] like Figure 8As shown, a defect detection process diagram of an embodiment of the present application is provided. First, the Unet network is trained, and the annular area to be detected is extracted using the trained Unet network, and the pixels of other areas outside the annular area in the product image are set to zero, four corner areas are segmented from the annular area and spliced ​​into a first sub-image, and four edge areas are segmented from the annular area and spliced ​​into a second sub-image, and the first defect detection model is trained using the first sub-image, and the second defect detection model is trained using the second sub-image, thereby completing the model training.

[0074] After the model training is completed, the trained defect detection model can be used to perform defect detection on the product images to be inspected.

[0075] An embodiment of the present application provides a defect detection method. Fig. 9 This is a flow chart of the defect detection method shown in the embodiment of the present application, such as Fig. 9 As shown, the method at least includes the following steps S910 to S920:

[0076] Step S910, obtaining a product image to be detected, extracting a ring area in the product image, and performing region segmentation and splicing on the ring area according to the corner shape and edge shape of the ring area to obtain a plurality of sub-images.

[0077] After the image of the product to be detected is obtained, the image of the product to be detected can be processed according to the processing process of the training data in the model training process to obtain multiple sub-images.

[0078] Step S920, input each sub-graph into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-graph; wherein the defect detection model is trained based on the defect detection model training method described above.

[0079] In one embodiment of the present application, each sub-image is input into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-image, including: marking the defect detection results in multiple sub-images in the product image to be inspected according to the pixel position of the sub-image in the product image to be inspected.

[0080] In production line applications, if you only need to detect NG (Not Good) products and NG categories without locating the specific location of the defect in the product, in this case, in order to reduce time consumption, you can directly feedback the detection results of the two sub-images without merging the sub-image detection results into the original product image. If you need to emphasize the location of the defect in theoretical research or application, you need to feed back the defect detection results of the two sub-images to the original product image according to the cropped coordinates and mark the original product image to determine the defect location.

[0081] An embodiment of the present application provides a defect detection model training device, Fig.10 This is a structural block diagram of the defect detection model training device shown in the embodiment of the present application, such as Fig.10 As shown, the device 1000 in this embodiment includes:

[0082] An image acquisition unit 1010 is used to acquire a product image, wherein the product image includes a ring-shaped area to be focused on.

[0083] The region processing unit 1020 is used to extract the annular region in the product image, divide the region shape of the annular region into a corner shape and a sideline shape; perform region segmentation and splicing on the annular region according to the corner shape and the sideline shape to obtain a plurality of sub-images;

[0084] The model training unit 1030 is used to use each sub-graph to train the defect detection model corresponding to the sub-graph.

[0085] In some embodiments, the region processing unit 1020 includes: a segmentation module and a splicing module;

[0086] A segmentation module, used for segmenting the annular area according to the corner shape and the edge shape to obtain four corner areas and four edge areas;

[0087] The splicing module is used to splice the four corner areas into a first sub-image, and splice the four edge areas into a second sub-image.

[0088] In some embodiments, the segmentation module is further used to extract four corner areas that conform to the corner shape according to the first size ratio, extract two first-type edge line areas that conform to the edge shape according to the second size ratio, and extract two second-type edge line areas that conform to the edge shape according to the third size ratio.

[0089] In some embodiments, the segmentation module is further used to segment the annular area according to the size information of the product defect, so that the extracted corner area and edge area have overlapping areas.

[0090] In some embodiments, a splicing module is used to splice four corner areas into an annular area according to their original orientations in the annular area; and to splice two first-type edge line areas to obtain a first edge line splicing area, and to splice two second-type edge line areas to obtain a second edge line splicing area; after processing the directions of the lengths of the first edge line splicing area and the second edge line splicing area to be the same direction, the length of the first edge line splicing area is set to be the same as the length of the second edge line splicing area; the first edge line splicing area and the second edge line splicing area with the same length and length direction are region-stitched to obtain a second sub-image.

[0091] In some embodiments, the region processing unit 1020 also includes: an extraction module, which processes the product image using a trained image segmentation network, generates a mask image corresponding to the annular region through the image segmentation network; and extracts the annular region from the product image according to the mask image.

[0092] In some embodiments, the extraction module is further used to adjust the image size of the mask image to obtain an adjusted mask image, the size of the adjusted mask image is the same as the size of the product image; and according to the pixel position of the annular area in the adjusted mask image, the annular area of ​​the pixel position in the product image is marked.

[0093] It can be understood that the above-mentioned defect detection model training device can implement the various steps of the defect detection model training method provided in the aforementioned embodiment, and the relevant explanations about the defect detection model training method are applicable to the defect detection model training device and will not be repeated here.

[0094] An embodiment of the present application provides a defect detection device, Fig.11 This is a structural block diagram of a defect detection device shown in an embodiment of the present application, such as Fig.11 As shown, the device 1100 in this embodiment includes:

[0095] The image processing unit 1110 is used to obtain a product image to be detected, extract an annular region in the product image, and perform region segmentation and splicing on the annular region according to the corner shape and edge shape of the annular region to obtain multiple sub-images;

[0096] The defect detection unit 1120 is used to input each sub-image into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-image;

[0097] Wherein, the defect detection model is trained based on the defect detection model training method described above.

[0098] Fig.12 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Fig.12 At the hardware level, the electronic device includes a processor and a memory, and optionally also includes an internal bus and a network interface. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0099] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0100] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0101] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a defect detection model training device or a defect detection device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0102] Acquire a product image, wherein the product image includes a ring-shaped area to be focused on;

[0103] Extracting the annular region in the product image, and dividing the region shape of the annular region into a corner shape and a sideline shape;

[0104] Performing region segmentation and splicing on the annular region according to the corner shape and edge shape to obtain a plurality of sub-graphs;

[0105] Each sub-graph is used to train the defect detection model corresponding to the sub-graph.

[0106] Or, to perform the following operations:

[0107] Acquire a product image to be detected, extract a ring area in the product image, and segment and splice the ring area according to the corner shape and edge shape of the ring area to obtain multiple sub-images;

[0108] Each sub-graph is input into a corresponding defect detection model to obtain a defect detection result corresponding to each sub-graph; wherein the defect detection model is trained based on the defect detection model training method described above.

[0109] The above application Figure 1The method performed by the defect detection model training device disclosed in the embodiment shown can be applied to a processor, or the above-mentioned Fig. 9 The method performed by the defect detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0110] The electronic device may also perform Figure 1 The defect detection model training device is implemented in the method, and the defect detection model training device is implemented in Figure 1 Alternatively, the electronic device may also perform Fig. 9 The method is performed by the defect detection device in Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0111] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the defect detection model training device in the embodiment shown, or for performing Fig. 9The method performed by the defect detection device in the illustrated embodiment.

[0112] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0116] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0117] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0118] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0119] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0121] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A defect detection model training method, characterized in that: include: Acquire a product image, wherein the product image includes an annular area to be inspected, and the tolerance of the annular area to defects is lower than that of other areas; Extracting the annular region in the product image, and dividing the region shape of the annular region into a corner shape and a sideline shape; The annular region is segmented and spliced ​​according to the corner shape and the edge shape to obtain a plurality of sub-images, wherein the sub-images can cover the annular region with a minimum area, and the sub-images have the characteristics of defect concentration and defect deformation reduction; Using each sub-graph to train the defect detection model corresponding to the sub-graph; The annular region is segmented and spliced ​​to obtain multiple sub-images, including: According to the corner shape and the edge shape, the annular area is segmented to obtain four corner areas and four edge areas, the edge areas include first-type edge areas and second-type edge areas, the length of the first-type edge areas is greater than that of the second-type edge areas, the image shapes of the two first-type edge areas are completely the same, and the image shapes of the two second-type edge areas are also completely the same; The four corner areas are spliced ​​into a first sub-image, specifically, the four corner areas are spliced ​​into a first sub-image according to their original orientations in the annular area where they are located; and the four sideline areas are spliced ​​into a second sub-image, specifically, two first-category sideline areas are spliced ​​to obtain a first sideline splicing area, and two second-category sideline areas are spliced ​​to obtain a second sideline splicing area; after processing the directions of the lengths of the first sideline splicing area and the second sideline splicing area to be the same direction, the length of the first sideline splicing area is set to be the same as the length of the second sideline splicing area; the first sideline splicing area and the second sideline splicing area having the same length and length direction are regionally spliced ​​to obtain a second sub-image.

2. The method according to claim 1, characterized in that According to the corner shape and the edge shape, the annular area is segmented to obtain four corner areas and four edge areas, including: Extract four corner areas that conform to the corner shape according to the first size ratio, Extract two first-class edge line regions that conform to the edge line shape according to the second size ratio, And extract two second-type edge line regions that conform to the edge line shape according to the third size ratio.

3. The method according to claim 2, characterized in that When the annular area is segmented according to the corner shape and the edge shape, the method further includes: The annular area is segmented according to the size information of the product defect, so that the extracted corner area and edge area have overlapping areas.

4. The method according to claim 1, characterized in that Extracting the annular region in the product image, including: Processing the product image using the trained image segmentation network, and generating a mask image corresponding to the annular area through the image segmentation network; A ring area is extracted from the product image according to the mask image.

5. The method according to claim 4, characterized in that Extracting a ring area from the product image according to the mask image includes: Adjusting the image size of the mask image to obtain an adjusted mask image, wherein the size of the adjusted mask image is the same as the size of the product image; According to the pixel positions of the annular area in the adjusted mask image, the annular area at the pixel positions in the product image is marked.

6. A defect detection method, characterized in that: include: An image of a product to be inspected is acquired, and an annular area in the product image is extracted, wherein the tolerance of the annular area to defects is lower than that of other areas, and the annular area is segmented and spliced ​​according to the corner shape and edge shape of the annular area to obtain a plurality of sub-images, wherein the sub-images can cover the annular area with a minimum area, and the sub-images have the characteristics of defect concentration and smaller defect shape. Specifically, the annular area is segmented according to the corner shape and the edge shape to obtain four corner areas and four edge areas, wherein the edge area includes a first type of edge area and a second type of edge area, wherein the length of the first type of edge area is greater than that of the second type of edge area, the image morphology of two first type of edge areas is exactly the same, and the image morphology of two second type of edge areas is exactly the same. The image morphology of the line area is also exactly the same; the four corner areas are spliced ​​into a first sub-image, specifically, the four corner areas are spliced ​​into a first sub-image according to the original orientations in the annular area where they are located, and the four sideline areas are spliced ​​into a second sub-image, specifically, two first-category sideline areas are spliced ​​to obtain a first sideline splicing area, and two second-category sideline areas are spliced ​​to obtain a second sideline splicing area; after processing the directions of the lengths of the first sideline splicing area and the second sideline splicing area into the same direction, the length of the first sideline splicing area is set to be the same as the length of the second sideline splicing area; the first sideline splicing area and the second sideline splicing area having the same length and length direction are spliced ​​to obtain a second sub-image; Input each sub-image into the corresponding defect detection model to obtain the defect detection result corresponding to each sub-image; Wherein, the defect detection model is trained based on the defect detection model training method according to any one of claims 1 to 5.

7. A defect detection model training device, characterized in that: The device is used to implement any one of the defect detection model training methods of claims 1 to 5.

8. A defect detection device, characterized in that: The device is used to implement the defect detection method described in claim 6.

Images