Defect detection method, electronic device, and storage medium

Abstract

The application provides a defect detection method, an electronic device and a storage medium. The method comprises the following steps: obtaining a product type and a product positive sample image of a product; determining at least one defect detection tool through a preset application program according to the product type; determining labeling information of the product positive sample image according to the at least one defect detection tool, wherein the labeling information comprises a labeling position and a labeling method; obtaining a product image to be detected; and performing defect detection on the product image to be detected according to the at least one defect detection tool and the labeling information of the product positive sample image, to obtain a defect detection result. The application can be applied to different industrial product detection, and effectively reduces the defect detection cost.

Description

Technical Field

[0001] This application relates to the field of product testing technology, and more particularly to a defect detection method, electronic device, and storage medium. Background Technology

[0002] In the field of industrial inspection, to improve the quality of industrial products, defect detection is typically performed on them before packaging. Current defect detection methods offer native applications that, in conjunction with cameras, perform various inspections on the industrial products requiring defect detection, such as surface defect detection and packaging pattern defect detection.

[0003] However, developing native applications is too costly, and due to the complexity of the images being inspected, problems arise when annotating images and obtaining defect detection results, such as annotations obscuring the background, unclear annotation lines, and difficulty in viewing defect detection results. Furthermore, native applications are bundled with specific defect detection tools, but different production lines require different defect detection tools. Moreover, the defect detection needs of different industrial products vary, thus a complete and specific native application may not necessarily be necessary. Summary of the Invention

[0004] In view of the above, it is necessary to provide a defect detection method, electronic device, and storage medium to solve the problems of bundled defect detection tools with native applications and excessive costs.

[0005] This application provides a defect detection method, which includes: acquiring a product type and a sample image of the product; determining at least one defect detection tool based on the product type using a preset application; determining annotation information of the sample image of the product based on the at least one defect detection tool, wherein the annotation information includes annotation location and annotation method; acquiring a product image to be detected; and performing defect detection on the product image to be detected based on the at least one defect detection tool and the annotation information of the sample image of the product to obtain a defect detection result.

[0006] In one preferred embodiment, the step of performing defect detection on the product image to be detected based on the defect detection tool and the annotation information of the positive sample image of the product to obtain a defect detection result includes: determining the detection information of the product image to be detected based on the annotation information of the positive sample image of the product; and performing defect detection on the product image to be detected using the at least one defect detection tool based on the detection information to obtain a defect detection result.

[0007] In an optional implementation, the method further includes: annotating the product sample image according to the annotation information of the product sample image; detecting whether the annotation of the product sample image matches the preset annotation information in the at least one defect detection tool, and obtaining a matching result; if the matching result shows a mismatch, determining that the annotation of the product sample image is incorrect, issuing a warning message, and prompting the user to re-annotate the product sample image; and displaying the matching result, the product sample image, and the annotation of the product sample image on the display screen using a first stacking method.

[0008] In an optional implementation, the method further includes displaying the defect detection results of the positive sample image of the product.

[0009] In an optional implementation, the method further includes setting a display mode for the defect detection results via the application.

[0010] In one optional implementation, setting the display mode of the defect detection results through the application includes: providing display mode options, responding to user input on the display mode options, and determining the display mode corresponding to the display mode options, wherein the display modes include a highlighted display mode, a hidden display mode, and a scaled display mode.

[0011] In an optional implementation, the method further includes:

[0012] The defect detection results are displayed on the screen in the set display mode.

[0013] In an optional implementation, the method further includes: displaying the defect detection results on the display screen using a second stacking method, wherein the second stacking method includes stacking and displaying the defect detection results of different defect detection tools on the display screen.

[0014] This application also provides an electronic device, which includes a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the defect detection method.

[0015] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the defect detection method.

[0016] This application utilizes the aforementioned defect detection method, which can be flexibly applied to the inspection of different industrial products, and enables quick and convenient viewing of defect detection results, thereby reducing defect detection costs. Attached Figure Description

[0017] Figure 1 This is a flowchart of a defect detection method provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the application structure provided in an embodiment of this application.

[0019] Figure 3a This is a schematic diagram provided for an embodiment of this application, without the use of a white outline.

[0020] Figure 3b This is a schematic diagram with a white outline provided for an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of a first stacking method structure provided in an embodiment of this application.

[0022] Figure 5 This is a schematic diagram of a highlighting display mode provided in an embodiment of this application.

[0023] Figure 6 This is a schematic diagram of a hidden display mode provided in an embodiment of this application.

[0024] Figure 7 This is a schematic diagram of a scaling display mode provided in an embodiment of this application.

[0025] Figure 8 This is a schematic diagram of a second stacking method provided in an embodiment of this application.

[0026] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0028] Numerous specific details are set forth in the following description to provide a thorough understanding of this application. The described embodiments are merely some, not all, of the embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0029] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of some embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in some embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0031] See Figure 1 As shown, Figure 1 A flowchart illustrating a defect detection method provided in an embodiment of this application. The method is applied to electronic devices (e.g., Figure 9 In the electronic device 9) shown, the electronic device can be any kind of electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (Internet Protocol Television, IPTV), smart wearable device, etc.

[0032] The electronic device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to: microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0033] The electronic device may also include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0034] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0035] The method specifically includes the following steps.

[0036] 101. Obtain the product type and a positive sample image of the product.

[0037] In at least one embodiment of this application, the product type includes, but is not limited to, wafers, crystal boats, plastic products, etc.

[0038] In at least one embodiment of this application, a positive sample image is obtained in response to user input. In at least one embodiment of this application, the positive sample image may also be pre-stored in a database. The positive sample image includes images of flawless products. For example, the image of a flawless product could be an image of a plastic product without scratches.

[0039] 102. Based on the product type, at least one defect detection tool is determined through a preset application.

[0040] In at least one embodiment of this application, the application is developed using a preset programming language and web technologies. In this embodiment, applications developed using web technologies can reduce costs and more easily incorporate different types of defect detection tools. In this embodiment, the preset programming language includes JavaScript, and the web technologies include Hyper Text Markup Language (HTML) and Cascading Style Sheets (CSS).

[0041] like Figure 2 The diagram shown is a schematic representation of the application structure provided in this embodiment of the application, including a program module 201, a defect detection tool module 202, a webpage display module 203, and an application setting module 204. The program module 201 is used to design the application's pages and functions, as well as the overall visual structure and operational logic of the application.

[0042] The defect detection tool module 202 stores at least one defect detection tool. The program module 201 calls at least one defect detection tool within the defect detection tool module 202. In this embodiment, each defect detection tool in the defect detection tool module 202 has a corresponding identifier, and the program module 201 directly calls the at least one defect detection tool based on the identifier. For example, when the program module 201 performs defect detection on a pattern on product packaging and uses a pattern defect detection tool, the identifier of the pattern defect detection tool is Pattern1, and the program module 201 can call the pattern defect detection tool through the corresponding code of the defect detection tool identifier.

[0043] The webpage display module 203 is used to display the content developed in the program module 201 using a webpage. In this embodiment, the webpage display module 203 loads and displays the content via a browser or application.

[0044] The application setting module 204 is used to configure the application. In this embodiment, configuring the application includes setting the display mode of the defect detection results. In this embodiment, the display mode includes a highlighted display mode, a hidden display mode, and a zoomed display mode.

[0045] In at least one embodiment of this application, determining at least one defect detection tool via an application based on the product type includes:

[0046] At least one defect detection tool is determined from the defect detection tool module 202 based on the product type. For example, if the product type is a wafer, the defect detection tool is determined to be a wafer defect detection tool.

[0047] The application stores various defect detection tools, allowing users to select the appropriate tool based on the type of product to be inspected. This provides flexibility in defect detection across different products and facilitates easy viewing of the results.

[0048] 103. Determine the annotation information of the positive sample image of the product based on the at least one defect detection tool.

[0049] In at least one embodiment of this application, the annotation information includes the annotation location and the annotation method.

[0050] In at least one embodiment of this application, the method further includes: annotating the product positive sample image according to the annotation information of the product positive sample image. In this embodiment, the annotation of the product positive sample image includes automatic annotation by an electronic device or manual annotation by a user.

[0051] In this embodiment, the annotation positions of the positive product sample image are determined according to the different defect detection tools, and the positive product sample image is annotated according to the annotation positions. For example, if the defect detection tool is a text defect detection tool, only the positions of the text in the positive product sample image are annotated. If the defect detection tool is a pattern defect detection tool, the positions of the pattern in the positive product sample image are annotated.

[0052] In this embodiment, the annotation method for the positive product sample image is determined based on the different defect detection tools, and the positive product sample image is annotated according to the annotation method. In this embodiment, the annotation method includes, but is not limited to, using polygons, text, lines, etc., to annotate the positive product sample image. When the image is complex, a white outline is used for annotation to make the annotation clearer and improve readability. For example, as... Figure 3a The diagram shown is a schematic representation of an embodiment of this application without a white outline. Figure 3a It can be seen that the labeling is unclear and difficult to read. For example... Figure 3b The diagram shown is a schematic representation of an embodiment of this application, marked with a white outline. Figure 3b The annotations are clear and highly readable.

[0053] In at least one embodiment of this application, the method further includes detecting whether the annotation of the positive sample image of the product matches the preset annotation information in the at least one defect detection tool, and obtaining a matching result;

[0054] If the matching result shows a mismatch, it is determined that the labeling of the positive sample image of the product is incorrect, a warning message is issued, and the user is prompted to re-label the positive sample image of the product.

[0055] In at least one embodiment of this application, the step of detecting whether the annotation of the positive sample image of the product matches the preset annotation information in the at least one defect detection tool, and obtaining the matching result, includes:

[0056] Depending on the defect detection tool, the preset annotation position or annotation method of the defect detection tool will also be different. The annotation position of the positive sample image of the product is matched with the preset annotation position of the defect detection tool, or the annotation method of the positive sample image of the product is matched with the preset annotation method of the defect detection tool to obtain the matching result.

[0057] In at least one embodiment of this application, the step of determining that the labeling of the positive sample image of the product is incorrect if the matching result shows a mismatch, issuing a warning message, and prompting the user to re-label the positive sample image of the product includes:

[0058] If the matching result shows that the annotation position in the positive sample image of the product does not match the preset annotation position in the at least one defect detection tool, it is determined that the annotation position of the positive sample image of the product is incorrect, a warning message is issued, and the user is prompted to re-annotate the positive sample image of the product; or

[0059] If the matching result shows that the annotation method of the positive sample image of the product does not match the preset annotation method in the at least one defect detection tool, it is determined that the annotation method of the positive sample image of the product is incorrect, a warning message is issued, and the user is prompted to re-annotate the positive sample image of the product; or

[0060] If the matching result shows that the annotation position and annotation method of the positive sample image of the product do not match the preset annotation position and preset annotation method in the at least one defect detection tool, it is determined that the annotation position and annotation method of the positive sample image of the product are incorrect, a warning message is issued, and the user is prompted to re-annotate the positive sample image of the product.

[0061] In this embodiment, the warning information can be displayed and accompanied by a prompt sound via a terminal (e.g., a computer or mobile phone), such as by outputting a preset voice or a buzzer sound.

[0062] In at least one embodiment of this application, the method further includes: displaying the matching result, the positive sample image of the product, and annotations on the positive sample image of the product on the display screen. For example, as Figure 4 As shown, Figure 4 This is a schematic diagram of a first stacking method provided in an embodiment of this application. The diagram shows that the positive sample image of the product is at the bottom layer, the matching result is in the middle layer, and the annotation of the positive sample image of the product is at the top layer. It should be noted that the diagram is only an example; the positive sample image of the product can be in the middle layer or at the top layer, and this application does not impose any specific limitations.

[0063] 104. Obtain the image of the product to be inspected, and perform defect detection on the image of the product to be inspected based on the annotation information of the positive sample image of the product, and obtain the defect detection result.

[0064] In at least one embodiment of this application, the product image to be detected refers to an image of a product for which defects need to be detected.

[0065] In at least one embodiment of this application, acquiring the image of the product to be inspected includes: controlling a camera device to take pictures of the product to be inspected at the same position and angle, and obtaining the image of the product to be inspected.

[0066] In at least one embodiment of this application, the step of performing defect detection on the product image to be detected based on the annotation information of the positive sample image of the product, and obtaining the defect detection result, includes:

[0067] Based on the annotation information of the positive product sample, the detection information of the image of the product to be detected is determined;

[0068] Based on the detection information, the at least one defect detection tool is used to perform defect detection on the product image to be detected, and the defect detection result is obtained.

[0069] In one embodiment, the detection information includes the marked positions in the image of the product to be detected.

[0070] In this embodiment, determining the detection information of the product image to be detected based on the annotation information of the positive product sample includes:

[0071] The annotation position in the product to be tested is determined based on the annotation position in the positive sample image of the product.

[0072] Specifically, the labeled positions in the positive sample image of the product are consistent with the labeled positions in the image of the product to be detected.

[0073] In at least one embodiment of this application, based on the detection information, the at least one defect detection tool is used to perform defect detection on the product image to be detected, and the defect detection result is obtained as follows:

[0074] The at least one defect detection tool is used to detect defects at the marked locations on the product image to be inspected. For example, if the defect detection tool is a pattern defect detection tool, and a pattern is marked on the positive sample image of the product, the defect detection tool will only detect the same marked locations in the product image to be inspected as those in the positive sample image, thus obtaining a defect detection result, which is then displayed on a screen. It should be noted that the product image to be inspected and the positive sample image are taken from the same angle and position.

[0075] In at least one embodiment of this application, the method further includes: providing a display mode option, responding to a user's input operation on the display mode option, and determining a display mode corresponding to the display mode option.

[0076] Specifically, the display modes include a highlighted display mode, a hidden display mode, and a scaled display mode.

[0077] In this embodiment, multiple display mode options, along with corresponding illustrations and / or text descriptions, can be provided for users to refer to and select the appropriate display mode.

[0078] In at least one embodiment of this application, if the user selects the highlighted display mode as the display mode for the defect detection result, it is determined that the defect detection result will be displayed on the screen in the highlighted display mode. Figure 5 As shown, Figure 5 This is a schematic diagram of a highlighting display mode provided in an embodiment of this application. The figure shows the defect detection result in the highlighting display mode, with a white outer frame used as a highlighting mark.

[0079] In at least one embodiment of this application, if the user selects the hidden display mode as the display mode for the defect detection result, it is determined that the defect detection result will be displayed on the screen in the hidden display mode. Figure 6 As shown, Figure 6 This is a schematic diagram illustrating a hidden display mode provided in an embodiment of this application. The defect detection results are hidden in the annotation box. Clicking the annotation box displays a detailed description of the defect detection results; clicking the annotation box again hides the detailed description of the defect detection results. In this embodiment, the detailed description of the defect detection results is displayed with a blurred background, a solid background, or a semi-transparent background.

[0080] In at least one embodiment of this application, if the user selects the zoomed display mode as the display mode for the defect detection result, it is determined that the defect detection result will be displayed on the screen in the zoomed display mode. Figure 7 As shown, Figure 7 This is a schematic diagram of a zoomed display mode provided in an embodiment of this application. The figure shows the defect detection result in zoomed display mode, with the defect detection result magnified. In this embodiment, the detailed defect detection result description can be zoomed in and out, and the font size and outer frame width can be adjusted, or it can remain at the same screen size to improve the readability of the defect detection result. In this embodiment, the detailed defect detection result description is displayed with a blurred background, a solid background, or a semi-transparent background.

[0081] It should be noted that users can select at least one display mode, and multiple display modes can be combined and used together.

[0082] In at least one embodiment of this application, the method further includes displaying the defect detection results on a display screen using a second stacking method. In this embodiment, the second stacking method includes stacking and displaying the defect detection results of different defect detection tools. For example, if three defect detection tools—a barcode defect detection tool, a text defect detection tool, and a pattern defect detection tool—are used to inspect the product, the defect detection results of the three tools are stacked and displayed. Figure 8 As shown, Figure 8 This is a schematic diagram of a second stacking method provided in an embodiment of this application. The diagram shows that the defect detection results of the pattern defect detection tool are at the bottom layer, the defect detection results of the text defect detection tool are in the middle layer, and the defect detection results of the barcode defect detection tool are at the top layer. It should be noted that the diagram is merely an example; the defect detection results of the pattern defect detection tool can be in the middle layer or at the top layer, and this application does not impose any specific limitations.

[0083] It should be noted that the image to be detected will also be displayed stacked with the defect detection results.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. For those skilled in the art, improvements can be made without departing from the inventive concept of this application, but these all fall within the scope of protection of this application.

[0085] like Figure 9 As shown, Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 9 includes a memory 901, at least one processor 902, a computer program 903 stored in the memory 901 and executable on the at least one processor 902, and at least one communication bus 904.

[0086] Those skilled in the art will understand that Figure 9 The schematic diagram shown is merely an example of the electronic device 9 and does not constitute a limitation on the electronic device 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 9 may also include input / output devices, network access devices, etc.

[0087] The at least one processor 902 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The at least one processor 902 may be a microprocessor or any conventional processor, etc. The at least one processor 902 is the control center of the electronic device 9, connecting various parts of the entire electronic device 9 via various interfaces and lines.

[0088] The memory 901 can be used to store the computer program 903. The at least one processor 902 implements various functions of the electronic device 9 by running or executing the computer program 903 stored in the memory 901 and calling data stored in the memory 901. The memory 901 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 9 (such as audio data), etc. In addition, the memory 901 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0089] If the modules / units integrated in the electronic device 9 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0090] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A defect detection method, characterized in that, The defect detection method includes: Obtain the product type and a sample image of the product; Based on the product type, at least one defect detection tool is determined through a preset application. Based on the at least one defect detection tool, the annotation information of the positive sample image of the product is determined, wherein the annotation information includes the annotation location and annotation method; Acquire an image of the product to be inspected, and perform defect detection on the image of the product to be inspected based on the at least one defect detection tool and the annotation information of the positive sample image of the product, to obtain a defect detection result, including: determining the annotation position in the product to be inspected based on the annotation position in the positive sample image of the product, wherein the annotation position in the positive sample image of the product and the annotation position in the image of the product to be inspected are consistent; using the at least one defect detection tool to detect only the same positions in the image of the product to be inspected as those in the positive sample image, to obtain the defect detection result.

2. The defect detection method according to claim 1, characterized in that, The method further includes: The positive sample image of the product is annotated according to the annotation information of the positive sample image of the product; The system detects whether the annotations on the positive sample image of the product match the preset annotation information in the at least one defect detection tool, and obtains the matching result. If the matching result shows a mismatch, it is determined that the labeling of the positive sample image of the product is incorrect, a warning message is issued, and the user is prompted to re-label the positive sample image of the product. The matching results, the positive sample image of the product, and the annotations on the positive sample image of the product are displayed on the screen using a first stacking method.

3. The defect detection method according to claim 1, characterized in that, The method further includes: The defect detection results of the positive sample image of the product are displayed.

4. The defect detection method according to claim 1, characterized in that, The method further includes: The application allows you to set the display mode for defect detection results.

5. The defect detection method according to claim 4, characterized in that, The method of setting the display mode of defect detection results through the application includes: Provide display mode options, respond to user input on the display mode options, and determine the display mode corresponding to the display mode options, wherein the display modes include highlight display mode, hide display mode and zoom display mode.

6. The defect detection method according to claim 4, characterized in that, The method further includes: The defect detection results are displayed on the screen in the set display mode.

7. The defect detection method according to claim 6, characterized in that, The method further includes: The defect detection results are displayed on the display screen using a second stacking method, wherein the second stacking method includes stacking and displaying the defect detection results of different defect detection tools on the display screen.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the defect detection method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the defect detection method as described in any one of claims 1 to 7.

Images