Method for improving visual inspection accuracy, electronic device, and storage medium

Abstract

A method for improving visual inspection accuracy, comprising: labeling collected product images to generate an initial dataset; generating a training dataset from the initial dataset, and training an Artificial Intelligence Generated Content (AIGC) model according to the training dataset; using different algorithms of the AIGC model to control the image production process; adjusting parameters corresponding to each algorithm in the AIGC model; generating a plurality of new product images through the AIGC model; training a visual inspection model according to the plurality of new product images; and using the visual inspection model to detect whether a product is abnormal. The present application also provides an electronic device and a storage medium for implementing the method, which improves the visual inspection accuracy of images by training the dataset of the AIGC.

Description

Technical Field

[0001] This invention relates to chip inspection methods, and more particularly to a method, electronic device, and storage medium for improving the accuracy of visual inspection. Background Technology

[0002] The rapid movement of products on the packaging line presents several challenges for anomaly detection. Current inspection methods are inefficient and inaccurate. Inspection speed is slow. Images become blurry due to high-speed product movement. Images taken at different stations have varying quality. Inventory counts may contain errors.

[0003] Known product visual inspection methods include data augmentation techniques. However, in most deep learning-based visual inspection software, the initially acquired dataset cannot cover all situations of the object being inspected. Therefore, data augmentation techniques are applied to simulate images of the same object type under different conditions, including left / right movement, zooming in / out, rotation, horizontal / vertical flipping, and contrast adjustment. While these techniques can provide a large number of images, the characteristics of some of these images may change; for example, moving, flipping, rotating, and scaling an image may result in blank areas at the image edges. Summary of the Invention

[0004] In view of the above, it is necessary to provide a method, electronic device, and storage medium to improve the accuracy of visual inspection by training a dataset of Artificial Intelligence Generated Content (AIGC) to improve the accuracy of visual inspection of images.

[0005] This invention provides a method for improving the accuracy of visual inspection, applied in an electronic device, comprising: collecting multiple product images; labeling each collected product image to generate an initial dataset; generating a training dataset based on the initial dataset, and training an Artificial Intelligence Generated Content (AIGC) model based on the training dataset; controlling the image production process using different algorithms of the AIGC model; adjusting the parameters corresponding to each algorithm in the AIGC model; generating multiple new product images through the AIGC model; training a visual inspection model based on the multiple new product images; and using the visual inspection model to detect whether the products are abnormal.

[0006] This invention also provides an electronic device, which includes a memory, a processor, and a program for improving visual detection accuracy stored in the memory and executable on the processor. The electronic device further includes an image collection and labeling module, a model training module, and a visual detection module. When executed by the processor, the program for improving visual detection accuracy performs the following steps: collecting multiple product images; labeling each collected product image to generate an initial dataset; generating a training dataset based on the initial dataset and training an artificial intelligence-generated content model based on the training dataset; controlling the image production process using different algorithms of the artificial intelligence-generated content model; adjusting the parameters corresponding to each algorithm in the artificial intelligence-generated content model; generating multiple new product images through the artificial intelligence-generated content model; training a visual detection model based on the multiple new product images; and using the visual detection model to detect whether the products are abnormal.

[0007] This invention also provides a storage medium storing a computer program that, when executed, implements the steps of the method for improving the accuracy of visual detection as described above.

[0008] The method, electronic device, and storage medium for improving visual detection accuracy according to embodiments of the present invention can reduce training costs, including the time and labor required for data collection and labeling. Models trained using rich datasets have high stability and significantly reduce verification time. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the steps of a method for improving the accuracy of visual detection according to an embodiment of the present invention.

[0010] Figure 2 This is a schematic diagram of the product status according to an embodiment of the present invention.

[0011] Figure 3 This is a schematic diagram of the architecture of the Packing Station according to an embodiment of the present invention.

[0012] Figure 4 This is a flowchart illustrating the steps involved in collecting the dataset according to an embodiment of the present invention.

[0013] Figure 5 This is a flowchart illustrating the steps of dataset division and AIGC model training in an embodiment of the present invention.

[0014] Figure 6 This is a flowchart illustrating the steps of selecting a preprocessing model and setting relevant parameters in an embodiment of the present invention.

[0015] Figure 7This is a flowchart illustrating the parameter adjustment steps for the AIGC model to generate a new image according to an embodiment of the present invention.

[0016] Figure 8 This is a flowchart illustrating the steps of anomaly detection using a visual inspection model according to an embodiment of the present invention.

[0017] Figure 9 This is a schematic diagram of the hardware architecture of the electronic device according to an embodiment of the present invention.

[0018] Figure 10 This is a functional block diagram of an electronic device according to an embodiment of the present invention.

[0019] Explanation of main component symbols Product 1 Carrier 2 Black Hole 3 Scroll 4 Detection device 5 Camera 6 Light source 7 Production Line 8 Computer 9 Electronic devices 200 Processor 210 Memory 220 System 230 for improving the accuracy of visual inspection Image collection and labeling module 310 Model training module 320 Visual inspection module 330 Steps S1-S8, S10-S33 The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0021] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention. The described embodiments are merely some, not all, of the embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0023] It should be noted that the descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of the stated features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0024] Figure 1 This is a flowchart illustrating the steps of a method for improving the accuracy of visual inspection according to an embodiment of the present invention. The method is applied to an electronic device, which is a server for improving the accuracy of visual inspection. Depending on different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0025] Step S1: Collect images of multiple products (e.g., chips). For example, 5 products, each with three different states. Take pictures of the products at three different brightness levels according to each product state, and collect a total of 45 images.

[0026] Figure 2 This is a schematic diagram of the product states according to an embodiment of the present invention. Product states include Normal, Reverse, and Missing, as shown below. Figure 2 The image shows the left, middle, and right chips, where 1 represents the product (e.g., chip), 2 represents the chip's pocket, 3 represents the black hole, and 4 represents the chip's reel.

[0027] Figure 3 This is a schematic diagram of the architecture of a packing station according to an embodiment of the present invention. The present invention operates a series of production lines 8 for different product types. Figure 3In this system, the control program is stored in computer 9, and the inspection box 5 contains an imaging subsystem (not shown). During the packaging process, the controller module (not shown) controls the motors (not shown) of production line 8 to package the products into reels 4 corresponding to each product type, and then the camera 6 records the video feed of production line 8. Adjusting the lens (not shown) of camera 6 allows it to capture the maximum amount of light from light source 7, thereby reducing the camera's capture time. During image (dataset) collection, products continuously move along production line 8 and are loaded into reels 4.

[0028] Figure 4 This is a flowchart illustrating the steps involved in collecting the dataset according to an embodiment of the present invention.

[0029] Step S10: Place each product (chip) on the production line.

[0030] Step S11: Adjust the light source to different brightness levels.

[0031] Step S12: Take product images of each product in different states according to different light source brightness.

[0032] Step S13: Determine whether the number of captured images has reached the preset number. If not, return to step S10.

[0033] For example, for the same product type, only images of 5 products are needed to generate a dataset. Therefore, with 5 products, 3 product states, and 3 light source brightness levels, a total of 45 product images are required. The number of products to be photographed is not limited to 45; the number of product images can be changed according to actual needs.

[0034] Step S14: Once a preset number of product images have been acquired, the dataset collection is complete.

[0035] Step S2: Label each collected image to generate an initial dataset.

[0036] Step S3: Generate a training dataset based on the initial dataset, and train an Artificial Intelligence Generated Content (AIGC) model based on the training dataset.

[0037] Figure 5 This is a flowchart illustrating the steps of dataset division and AIGC model training in an embodiment of the present invention.

[0038] Step S15: Adjust the image resolution of the initial dataset.

[0039] In this embodiment of the invention, the required image resolution is 512 x 512 pixels. Therefore, multiple images from the initial dataset need to be adjusted to this resolution for training the AIGC model. It should be noted that the images can be adjusted to other resolutions according to other requirements.

[0040] Step S16: Label each image according to its product status.

[0041] For example, images with a product status of Reverse are labeled as Chip_Reverse(1), Chip_Reverse(2), etc. Similarly, images with product statuses of Normal and Missing are labeled as Chip_Normal and Chip_Missing, respectively.

[0042] Step S17: Generate a training dataset for training the AIGC model based on the labeled product images.

[0043] Step S18: Train the AIGC model based on the training dataset.

[0044] The training process of the AIGC model consists of two stages: U-net training and Textcoder training. It should be noted that U-net training and Textcoder training are familiar AIGC model training techniques to those skilled in the art, and will not be elaborated upon in this article.

[0045] Step S4: Based on different product states, different algorithms of the AIGC model are used to control the image production process for different product images.

[0046] Figure 6 This is a flowchart illustrating the steps of selecting a preprocessing model and setting relevant parameters in an embodiment of the present invention.

[0047] The AIGC model is trained on a large dataset, thus possessing strong generative capabilities. The difference between the generated image and the original image depends on the degree of denoising. To control the image generation of the AIGC model, it is necessary to select appropriate intermediate preprocessing algorithms, such as the Canny composite edge detection algorithm and the Maximum Likelihood Sequence Detection (MLSD) algorithm.

[0048] Step S19: Obtain the initial dataset.

[0049] Step S20: Obtain one of the images from the initial dataset and determine whether the image is in the Chip_Missing state.

[0050] Step S21: If the image is in the Chip_Missing state, perform image processing on the image using the Canny algorithm and the corresponding Canny parameters. Note that the Canny algorithm is an image processing technique familiar to those skilled in the art and will not be described in detail herein.

[0051] Step S22: If the image is not in the Chip_Missing state, for example, in the Chip_Normal or Chip_Reverse state, image processing is performed on the image using the MLSD algorithm and the corresponding MLSD parameters. It should be noted that the MLSD algorithm is an image processing technique familiar to those skilled in the art, and will not be described in detail herein.

[0052] Step S23: Store the Canny parameters and the MLSD parameters into the AIGC model.

[0053] Step S5: Adjust the parameters corresponding to each algorithm in the AIGC model.

[0054] Step S6: Generate a new product image using the AIGC model.

[0055] Figure 7 This is a flowchart illustrating the parameter adjustment steps for generating new images using the AIGC model according to an embodiment of the present invention. The trained AIGC model is used and its parameters are adjusted to generate new images based on the initial dataset.

[0056] Step S24: Set the model parameters of the AIGC model.

[0057] Step S25: The AIGC model processes an image from the initial dataset according to the model parameters to generate a new image.

[0058] Step S26: Determine whether the quality of the new image meets the requirements of visual detection.

[0059] In step S27, if the quality of the new image does not meet the requirements of visual detection, the values ​​of the seed and classifier free guidance scale (CFGs) parameters of the model are adjusted, and the process returns to step S25 to generate a better quality image in the next iteration.

[0060] Step S28: If the quality of the new image meets the requirements of visual detection, the AIGC model processes the remaining images in the initial dataset according to the pre-set model parameters to generate new images. It should be noted that the images in the initial dataset can be used to generate more new images, for example, 100 new images.

[0061] Step S7: Train the visual detection model based on the new product image.

[0062] Step S8: Use the visual inspection model to detect whether the product is abnormal.

[0063] Figure 8 This is a flowchart illustrating the steps of anomaly detection using a visual inspection model according to an embodiment of the present invention.

[0064] Step S30: When the product is on the production line, the product is photographed by the camera of the inspection box to capture images.

[0065] Step S31: Determine whether an abnormal image has been detected.

[0066] Step S32: If no abnormal image is detected, it means that the product is judged to be normal, and the product is packaged into a roll.

[0067] Step S33: If an abnormal image is detected, it indicates that the product is judged to be abnormal, and the detection device issues an instruction to stop the production line.

[0068] Step S34: Store the image of the product in the workshop management system.

[0069] Figure 9 This is a schematic diagram of the hardware architecture of an electronic device according to an embodiment of the present invention. The electronic device 200, for example, but not limited to, a server for improving visual inspection accuracy, can be interconnected with the processor 210, memory 220, and system 230 for improving visual inspection accuracy via a system bus. Figure 9 Only the electronic device 200 with components 210-230 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0070] The memory 220 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 220 may be an internal storage unit of the electronic device 200, such as the hard disk or memory of the electronic device 200. In other embodiments, the memory may also be an external storage device of the electronic device 200, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 200. Of course, the memory 220 may also include both internal storage units and external storage devices of the electronic device 200. In this embodiment, the memory 220 is typically used to store the operating system and various application software installed on the electronic device 200, such as the program code of the system 230 for improving the accuracy of visual inspection. Furthermore, the memory 220 can also be used to temporarily store various types of data that have been output or will be output.

[0071] In some embodiments, the processor 210 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 210 is typically used to control the overall operation of the electronic device 200. In this embodiment, the processor 210 is used to run program code stored in the memory 220 or process data, for example, to run the system 230 for improving visual inspection accuracy.

[0072] It should be noted that, Figure 9 The electronic device 200 is merely an example. In other embodiments, the electronic device 200 may also include more or fewer components, or have different component configurations.

[0073] If the modules / units integrated in the electronic device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, disks, optical discs, computer memory, read-only memory, random access memory, electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0074] Figure 10 This is a functional block diagram of an electronic device according to an embodiment of the present invention, which is used to execute a method for improving the accuracy of visual inspection. The method for improving the accuracy of visual inspection according to an embodiment of the present invention can be implemented by a computer program stored in a storage medium, such as memory 220 in the electronic device 200. When the computer program implementing the method of the present invention is loaded into memory 220 by processor 210, it drives processor 210 of the electronic device 200 to execute the method for improving the accuracy of visual inspection according to an embodiment of the present invention.

[0075] The electronic device 200 of this invention includes an image collection and labeling module 310, a model training module 320, and a visual detection module 330.

[0076] The image collection and labeling module 310 collects images of multiple products (e.g., chips), for example, 5 products, each product has three different states, and the products are photographed at three different brightness levels according to each product state, for a total of 45 images; and labels each collected image to generate an initial dataset.

[0077] The model training module 320 generates a training dataset based on the initial dataset and trains the AIGC model based on the training dataset; according to different product states, it uses different algorithms of the AIGC model to control the image production process for different product images; it adjusts the parameters of each algorithm in the AIGC model; it generates new product images through the AIGC model; and it trains a dataset for the visual detection model based on the new product images.

[0078] The visual inspection module 330 detects whether the five products are abnormal.

[0079] It is understood that the module division described above is merely a logical functional division, and other division methods may be used in actual implementation. Furthermore, the functional modules in the various embodiments of this application can be integrated into the same processing unit, or each module can exist physically separately, or two or more modules can be integrated into the same unit. The integrated modules described above can be implemented in hardware or in a combination of hardware and software functional modules.

[0080] For those skilled in the art, other corresponding changes or adjustments can be made to the technical solutions and concepts provided in the embodiments of the present invention in combination with actual needs, and all such changes and adjustments should fall within the protection scope of the claims of the present invention.

Claims

1. A method for improving the accuracy of visual inspection, applied in electronic devices, characterized in that, The method includes: Collect multiple product images; Each collected product image is labeled to generate an initial dataset; A training dataset is generated based on the initial dataset, and an AIGC (Artificial Intelligence Generate Content) model is trained based on the training dataset. Different algorithms of the aforementioned artificial intelligence-generated content model are used to control the image production process; Adjust the parameters of each algorithm in the AI-generated content model; Multiple new product images are generated through the aforementioned artificial intelligence-generated content model; A visual detection model is trained based on the aforementioned multiple new product images; and The visual inspection model is used to detect whether the product is abnormal.

2. The method for improving the accuracy of visual inspection as described in claim 1, characterized in that, The step of collecting multiple product images also includes: Images of the multiple products are collected, wherein each product includes different product states, including Normal, Reverse, and Missing.

3. The method for improving the accuracy of visual detection as described in claim 2, characterized in that, The step of collecting multiple product images also includes: The product is photographed at different brightness levels according to its condition, and a preset number of images are collected in total.

4. The method for improving the accuracy of visual detection as described in claim 2, characterized in that, The step of labeling each collected product image to generate the initial dataset further includes: Images with a product status of "Normal" are labeled "Chip_Normal", images with a product status of "Reverse" are labeled "Chip_Reverse", and images with a product status of "Missing" are labeled "Chip_Missing". The training dataset for training the AIGC model is generated based on the labeled product images.

5. The method for improving the accuracy of visual detection as described in claim 1, characterized in that, The step of controlling the image production process using different algorithms of the AIGC model further includes: The image production process is controlled using the Canny composite edge detection algorithm and the Maximum Probability Sequence Detection (MLSD) algorithm.

6. The method for improving the accuracy of visual inspection as described in claim 1, characterized in that, The step of using the visual inspection model to detect whether a product is abnormal also includes: Capture images of the product to be tested; Determine whether the image of the product to be detected is an abnormal image; If the image of the product to be tested is not an abnormal image, then the product is packaged into a roll; If the image of the product to be inspected is an abnormal image, an instruction to stop the production line is issued; and The image of the product to be tested is stored in the workshop management system.

7. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a program for improving visual detection accuracy stored in the memory and executable on the processor. The electronic device also includes an image acquisition and labeling module, a model training module, and a visual detection module. When the program for improving visual detection accuracy is executed by the processor, it performs the following steps: Collect multiple product images; Each collected product image is labeled to generate an initial dataset; A training dataset is generated based on the initial dataset, and an AIGC (Artificial Intelligence Generate Content) model is trained based on the training dataset. Different algorithms of the aforementioned artificial intelligence-generated content model are used to control the image production process; Adjust the parameters of each algorithm in the AI-generated content model; Multiple new product images are generated through the aforementioned artificial intelligence-generated content model; A visual detection model is trained based on the aforementioned multiple new product images; and The visual inspection model is used to detect whether the product is abnormal.

8. The electronic device as claimed in claim 7, characterized in that, When the program for improving visual detection accuracy is executed by the processor, it also performs the following steps: Images of the multiple products are collected, wherein each product includes different product states, including normal, reversed, and missing; and The product is photographed at different brightness levels according to its condition, and a preset number of images are collected in total.

9. The electronic device as claimed in claim 7, characterized in that, When the program for improving visual detection accuracy is executed by the processor, it also performs the following steps: Film one of the products. Capture images of the product to be tested; Determine whether the image of the product to be detected is an abnormal image; If the image of the product to be tested is not an abnormal image, then the product is packaged into a roll; If the image of the product to be inspected is an abnormal image, an instruction to stop the production line is issued; and The image of the product to be tested is stored in the workshop management system.

10. A storage medium storing at least one computer instruction thereon, characterized in that, The instructions are executed by the processor and loaded to perform the method for improving visual detection accuracy as described in any one of claims 1-6.

Images