System and method for automatically detecting products
Through automated detection systems and image analysis models, the problems of inconsistent manual detection and low efficiency are solved, efficient and accurate product quality inspection are achieved, and equipment complexity and cost are reduced.
Patent Information
- Application Number
- CN202410074083.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-18
- Publication Date
- 2025-07-18
AI Technical Summary
In the prior art, manual inspection of product quality has problems such as inconsistent detection results and poor efficiency, and it is difficult to effectively prevent defective products from flowing into the client.
An automated detection system is adopted, including computer devices and product detection devices, and a product image set is generated using image shooting modules and rotation modules, and automatic detection is performed through image analysis models, correcting the error judgment results and training the model to improve accuracy.
It realizes efficient and accurate product inspection, reduces the influence of human factors, improves detection efficiency and effectiveness, and reduces equipment complexity and cost.
Smart Images
Figure CN120334227A_ABST
Abstract
Description
Technical Field
[0001] This application relates to a system and method, particularly a system and method for automatically detecting products. Background Art
[0002] During the process of product assembly in manufacturers or foundries, etc., it is inevitable that there are human errors, missing parts, loose screws, and / or scratches on the product appearance. When the products are shipped and sent to the client, due to the defects of the products themselves, it often causes dissatisfaction among the clients, and then complaints are made to the head office.
[0003] Traditionally, in order to prevent defective products from being sent to the client, manual inspection is often adopted. For example, quality inspection personnel are hired to visually inspect the products before shipment to achieve the effect of quality control for the products before shipment. Summary of the Invention
[0004] Although the manual inspection method can achieve the effect of quality control for the products before shipment, this method is easily affected by human factors. For example, different judgment personnel may lead to inconsistent judgment criteria, that is, inconsistent detection quality. In addition, even if the judgment personnel are the same, the manual inspection method will also lead to problems such as limited detection effect and / or poor detection efficiency.
[0005] As described above, the purpose of this application is to solve the deficiencies of the prior art. Specifically, one purpose of this application is to solve the problems of limited detection effect and / or poor detection efficiency caused by human judgment.
[0006] To achieve the above purpose, this application provides a system for automatically detecting products. The system is suitable for detecting a product to be detected to generate a detection analysis result. The system includes a computer device and a product detection device, where the product detection device is configured to be coupled to the computer device. The product detection device includes a detection platform, a rotation module, and four image capture modules. The detection platform is configured to provide a receiving space. The rotation module is configured to operably rotate the detection platform. The four image capture modules are respectively disposed at different positions in the product detection device, and are respectively configured to capture images of the product to be detected placed in the receiving space to generate a product image set, where the product image set includes six views of the product to be detected. The computer device includes a storage module and a processing module, where the processing module is configured to be coupled to the storage module. The storage module is configured to store a computer program product. The processing module is configured to, after loading and executing the computer program product, be able to perform at least one image analysis step to generate the detection analysis result based on the product image set.
[0007] In some embodiments, at least one image analysis step that can be performed by the processing module of the computer device in the system includes receiving a product image set, inputting the product image set into an image analysis model, and analyzing the content of the product image set through the image analysis model to generate a detection and analysis result.
[0008] In some embodiments, at least one image analysis step that can be performed by the processing module of the computer device in the system further includes, when the detection and analysis result is a detection failure, further determining whether the detection and analysis result is a misjudgment. Among them, when the detection and analysis result is a misjudgment, correcting the detection and analysis result.
[0009] In some embodiments, at least one image analysis step that can be performed by the processing module of the computer device in the system further includes, when the detection and analysis result is a misjudgment, using the corrected detection and analysis result and the product image set as training data to further train the image analysis model with the training data.
[0010] In some embodiments, at least one image analysis step that can be performed by the processing module of the computer device in the system further includes storing the product identification code, the product image set, and the detection and analysis result of the product to be detected in a product detection database.
[0011] In addition, to achieve the above object, the present application also provides a method for automatically detecting products. The method is applicable to be executed by a system for automatically detecting products as described in the present application. The method includes generating a product image set through a product detection device, receiving the product image set through a computer device, inputting the product image set into an image analysis model through the computer device, and analyzing the content of the product image set through the image analysis model to generate a detection and analysis result. Among them, the product image set includes six-side views of the product to be detected.
[0012] In some embodiments, the method further includes, when the detection and analysis result is a detection failure, determining whether the detection and analysis result is a misjudgment through the computer device. Among them, when the detection and analysis result is a misjudgment, correcting the detection and analysis result through the computer device.
[0013] In some embodiments, the method further includes, when the detection and analysis result is a misjudgment, using the corrected detection and analysis result and the product image set as training data through the computer device to further train the image analysis model with the training data.
[0014] In some embodiments, the method further includes storing the product identification code, the product image set, and the detection and analysis result of the product to be detected in a product detection database through the computer device.
[0015] In some embodiments, the method further includes respectively performing image capturing on the product to be detected placed in the accommodation space by four image capturing modules, rotating the detection platform by a rotation module, and again respectively performing image capturing on the product to be detected placed in the accommodation space by the four image capturing modules.
[0016] Thereby, the technical means provided by the present application can achieve beneficial effects that cannot be achieved by the prior art. Specifically, one beneficial effect that the present application can achieve is to improve the detection effect and / or detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 Schematic diagram of a system for automatically detecting products according to an embodiment of the present application.
[0018] Figure 2 Schematic diagram of a product detection device according to an embodiment of the present application.
[0019] Figure 3 Schematic diagram of a computer device according to an embodiment of the present application.
[0020] Figure 4 Flowchart of each image analysis step executed by the processing module according to the first embodiment of the present application.
[0021] Figure 5 Flowchart of each image analysis step executed by the processing module according to the second embodiment of the present application.
[0022] Figure 6 Flowchart of each image analysis step executed by the processing module according to the third embodiment of the present application.
[0023] Figure 7 Flowchart of each image analysis step executed by the processing module according to the fourth embodiment of the present application.
[0024] Figure 8 Flowchart of each step of the method for automatically detecting products according to the first embodiment of the present application.
[0025] Figure 9 Detailed flowchart of step S810 of the method for automatically detecting products.
[0026] Figure 10 Schematic diagram of images of the detection and analysis results of two different products to be detected.
[0027] REFERENCE SIGNS
[0028] 50 System
[0029] 100 Product detection device
[0030] 110 Detection Platform
[0031] 130A First Image Shooting Module
[0032] 130B Second Image Shooting Module
[0033] 130C Third Image Shooting Module
[0034] 130D Fourth Image Shooting Module
[0035] 200 Computer Device
[0036] 210 Receiving Module
[0037] 220 Processing Module
[0038] 230 Storage Module
[0039] 240 Output Module
[0040] 250 Image Analysis Model
[0041] 260 Product Detection Database
[0042] 1010, 1020 Images
[0043] S410, S420, S430 Image Analysis Steps
[0044] S510, S515 Image Analysis Steps
[0045] S520, S525 Image Analysis Steps
[0046] S530 Image Analysis Step
[0047] S610 Image Analysis Step
[0048] S710 Image Analysis Step
[0049] S810, S820 Steps
[0050] S830, S840 Steps
[0051] S910, S920, S930 Steps Detailed Implementation Manner
[0052] The content of this application will be described in detail through the following embodiments and the accompanying drawings, so as to help those with ordinary knowledge in the technical field to which this application belongs understand the purpose, features and effects of this application. It should be noted that the various steps described herein can be executed sequentially, in reverse order, or by appropriately changing or skipping the order during the control process. It should be noted that the statement that "the first step can be executed after the second step" described herein may mean that the first step is directly executed after the second step is completed, or it may mean that after the second step is completed, other steps (for example, the third step) are first executed and then the first step is executed.
[0053] In addition, in the content described in this application, it should be noted that terms such as "first", "second" and "third" are used to distinguish different components, rather than to limit the components themselves or represent a specific order of the components. It should be noted that in the following description, the same components or steps can be represented by the same numbers.
[0054] In addition, the term "coupled" described in this application can be expressed as "directly connected" and / or "indirectly connected". Specifically, when the first component is configured to be coupled to the second component, it can mean that the first component is configured to be directly connected to the second component and / or the first component is configured to be indirectly connected to the second component.
[0055] For the purpose of simplicity, although each step in at least one image analysis step described in this application is executed by a computer device, in some embodiments, each step in at least one image analysis step can also be executed by multiple computer devices.
[0056] Please refer to Figure 1 , Figure 1 FIG. is a schematic diagram of a system 50 for automatically detecting products, which illustrates an embodiment of this application. The system 50 for automatically detecting products includes a product detection device 100 and a computer device 200, wherein the product detection device 100 is configured to be coupled to the computer device 200.
[0057] The product detection device 100 can be configured to take images of the product to be detected to generate a set of product images of the product to be detected. The computer device 200 can be configured to be able to execute at least one image analysis step to generate a detection analysis result based on the set of product images of the product to be detected.
[0058] In some embodiments, the computer device 200 may be connected to a programmable logic controller (PLC) via a physical signal line compliant with the Transmission Control Protocol / Internet Protocol (TCP / IP), and then operate each module of the product detection device 100 through the programmable logic controller. For example, the computer device 200 may place the product to be detected on the detection platform, move the detection platform, rotate the detection platform, move the image capture module, and / or remove the product to be detected from the detection platform through the programmable logic controller, and so on. In some embodiments, the programmable logic controller may be a finished product well-known to those with ordinary knowledge in the technical field to which this application pertains.
[0059] In some embodiments, the computer device 200 may be connected to the image capture module in the product detection device 100 via a Universal Serial Bus (USB) transmission line to receive the images captured by the image capture module in the product detection device 100. In some embodiments, the USB transmission line may be a finished product well-known to those with ordinary knowledge in the technical field to which this application pertains.
[0060] Since the product detection device 100 can capture images of the product to be detected and the computer device 200 can receive the images related to the product to be detected generated by the product detection device 100, the computer device 200 can further perform image analysis on the received images to generate detection analysis results related to the product to be detected. Thus, without manual inspection, the system 50 for automatically detecting products provided in this application can automatically detect the appearance of the product to be detected. At the same time, the system 50 for automatically detecting products provided in this application can have better detection effects and / or detection efficiency.
[0061] Please refer to Figure 2 , Figure 2 FIG. is a schematic diagram of the product detection device 100 illustrating an embodiment of this application. The product detection device 100 includes a detection platform 110, a rotation module, and four image capture modules (respectively denoted as the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D).
[0062] The detection platform 110 is configured to provide a receiving space such that the product to be detected can be placed in the receiving space of the detection platform 110, and then images of the product to be detected placed in the receiving space are captured. In some embodiments, the entire detection platform 110 may be a transparent platform, such as a glass plate, but is not limited thereto. In other embodiments, the overlapping portion of the receiving space and the detection platform 110 may be a transparent platform, such as a glass plate, but is not limited thereto.
[0063] The rotation module is configured to operably rotate the detection platform 110. More specifically, the computer device 200 can control the rotation module via a programmable logic controller such that the rotation module can rotate the detection platform 110. In some embodiments, the rotation module may include a fixed base plate and a rotating member. In some embodiments, the configuration relationship between the detection platform 110 and the rotation module may be as follows: from top to bottom are the detection platform 110, the rotating member of the rotation module, and the fixed base plate of the rotation module. That is, the rotation module can use the fixed base plate as the basis for rotation to rotate the detection platform 110 through the rotating member. In some embodiments, the rotation module can rotate the detection platform 110 in a horizontal rotation manner. In some embodiments, the rotation module can rotate the detection platform by 0 to 359 degrees in a fixed horizontal rotation direction (e.g., clockwise and / or counterclockwise). Since the rotation module can rotate the detection platform 110 after receiving a control signal from the programmable logic controller, the product to be detected placed in the accommodation space of the detection platform 110 can also be rotated as the detection platform 110 rotates.
[0064] Four image capture modules (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) are respectively disposed at different positions in the product detection device 100 and are respectively configured to perform image capture on the product to be detected placed in the accommodation space of the detection platform 110 to generate images related to the product to be detected (i.e., the product image set of the product to be detected). In some embodiments, the product image set of the product to be detected includes six views of the product to be detected. In some embodiments, the four image capture modules (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) may be industrial cameras, such as a2A3840-45ucBAS - Basler ace 2, but are not limited thereto. In some embodiments, the lenses (LENS) of the four image capture modules (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) may be, for example, ML - M0625UR, but are not limited thereto.
[0065] In a specific example, the first image capturing module 130A can be disposed above the inner side of the product detection device 100, such that the first image capturing module 130A can capture the product to be detected placed in the accommodation space of the detection platform 110 from above and generate a top view of the product to be detected. The second image capturing module 130B can be disposed below the inner side of the product detection device 100, such that the second image capturing module 130B can capture the product to be detected placed in the accommodation space of the detection platform 110 from below and generate a bottom view of the product to be detected. The third image capturing module 130C can be disposed behind the inner side of the product detection device 100, such that the third image capturing module 130C can capture the product to be detected placed in the accommodation space of the detection platform 110 from behind and generate a rear view of the product to be detected. The fourth image capturing module 130D can be disposed on the right side of the inner side of the product detection device 100, such that the fourth image capturing module 130D can capture the product to be detected placed in the accommodation space of the detection platform 110 from the right side and generate a right side view of the product to be detected. Then, by rotating the module, the detection platform 110 is horizontally rotated by 180 degrees, and the third image capturing module 130C can capture the product to be detected placed in the accommodation space of the detection platform 110 from behind again to generate a front view of the product to be detected, and the fourth image capturing module 130D can capture the product to be detected placed in the accommodation space of the detection platform 110 from the right side again to generate a left side view of the product to be detected. In this way, the product detection device 100 can generate six views of the product to be detected (i.e., the product image set described in the present application).
[0066] In another specific example, the first image capturing module 130A, the second image capturing module 130B, the third image capturing module 130C, and the fourth image capturing module 130D can also be respectively disposed above the inner side, below the inner side, on the right side of the inner side, and on the left side of the inner side of the product detection device 100 to respectively generate a top view, a bottom view, a right side view, and a left side view of the product to be detected. Then, by rotating the module, the detection platform 110 is horizontally rotated by 90 degrees, and the third image capturing module 130C and the fourth image capturing module 130D can further generate a front view and a rear view of the product to be detected. That is to say, in this specific example, the product detection device 100 can also generate six views of the product to be detected (i.e., the product image set described in the present application).
[0067] Accordingly, the product detection device 100 provided by the present application can generate six views of the product to be detected by only using four image capture modules, thereby reducing the components and configuration space required for the product detection device 100 and lowering the manufacturing cost of the product detection device 100. In addition, due to the configuration relationship among the detection platform 110, the rotation module, and the four image capture modules, the product detection device 100 provided by the present application can generate six views of the product to be detected by only horizontally rotating the product to be detected once, thereby reducing the device complexity and / or operation complexity of the product detection device 100. That is to say, the product detection device 100 provided by the present application can generate six views of the product to be detected while taking both cost and complexity into consideration.
[0068] In some embodiments, the product detection device 100 may further include a moving member. The moving member may be, for example, a slide rail, but is not limited thereto. In some embodiments, the four image capture modules (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) may be further disposed in the moving member and configured to respectively adjust the positions of the four image capture modules through the moving member. In a specific example, the first image capture module 130A may be disposed in the first moving member such that the distance between the first image capture module 130A and the product to be detected placed in the accommodation space of the detection platform 110 can be adjusted through the first moving member (the adjustable distance range may be, for example, between 0 and 20 centimeters, but is not limited thereto). The second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D may also be respectively disposed in the second moving member, the third moving member, and the fourth moving member such that the distances between the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D and the product to be detected placed in the accommodation space of the detection platform 110 can also be respectively adjusted through the second moving member, the third moving member, and the fourth moving member. Accordingly, the product detection device 100 having a moving member can respectively adjust the positions of the four image capture modules (i.e., the distances between the four image capture modules and the product to be detected placed in the accommodation space of the detection platform 110), thereby enabling the four image capture modules to more clearly and / or more completely capture images of the product to be detected placed in the accommodation space of the detection platform 110 to generate better six views of the product to be detected (i.e., the product image set described in the present application).
[0069] Please refer to Figure 3 , Figure 3FIG. 0 is a schematic diagram of a computer device 200 illustrating an embodiment of the present application. The computer device 200 includes a processing module 220 and a storage module 230. In some embodiments, the computer device 200 may further include a receiving module 210, an output module 240, an image analysis model 250, and / or a product detection database 260.
[0070] The receiving module 210 is configured to be able to receive data and / or instructions, etc. from an external device. In some embodiments, since the receiving module 210 of the computer device 200 can be respectively connected to the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D in the product detection device 100 through respective USB transmission lines, the receiving module 210 of the computer device 200 can receive a product image set of the product to be detected from the product detection device 100.
[0071] The processing module 220 is configured to be coupled to the storage module 230 and, after loading and executing a computer program product stored in the storage module 230, is configured to perform at least one image analysis step to generate a detection analysis result based on the product image set. In some embodiments, the processing module 220 may be a graphics processing unit (GPU), but is not limited thereto. More specifically, the processing module 220 may be, for example, Jetson AGX Xavier (GPU 512-core 1.37GHz), but is not limited thereto. Since the graphics processing unit has more powerful computing capabilities compared to the central processing unit (CPU), the processing module 220 provided in the present application can more effectively perform image analysis on the product image set.
[0072] In some embodiments, the processing module 220 may further be configured to be respectively coupled to the receiving module 210, the output module 240, the image analysis model 250, and / or the product detection database 260, such that the processing module 220 can receive the product image set through the receiving module 210, output the detection analysis result through the output module 240, input the product image set into the image analysis model 250, and / or store data related to the product to be detected in the product detection database 260, etc.
[0073] The storage module 230 is configured to be coupled to the processing module 220 and is configured to store a computer program product such that after the processing module 220 loads and executes the computer program product, it can perform at least one image analysis step to generate a detection analysis result based on the product image set. In some embodiments, the computer program product may be a series of program codes and / or instruction sets. In some embodiments, the storage module 230 may be configured to include one or more volatile storage media and one or more non-volatile storage media. In some embodiments, the volatile storage media may be any type of storage media known to those of ordinary skill in the art to which this application pertains, such as dynamic random access storage media or static random access storage media, etc., but not limited thereto. In some embodiments, the non-volatile storage media may be any type of storage media known to those of ordinary skill in the art to which this application pertains, such as read-only storage media, flash storage media, or non-volatile random access media, etc., but not limited thereto.
[0074] The output module 240 is configured to be coupled to the processing module 220 and is configured to be able to output the detection analysis result. In some embodiments, the output module 240 may output the detection analysis result generated by the processing module 220 to a display device and / or other computer devices, etc. In some embodiments, the output module 240 may further be configured to be coupled to the product detection database 260 such that the output module 240 can output the detection analysis result generated by the processing module 220 to the product detection database 260.
[0075] The image analysis model 250 is configured to be coupled to the processing module 220 and is configured to be able to perform image analysis on the product image set to generate a detection analysis result. In some embodiments, the processing module 220 may input the product image set into the image analysis model 250 such that the image analysis model 250 can analyze the content of the product image set and generate a detection analysis result. In some embodiments, the image analysis performed by the image analysis model 250 on each image in the product image set may be to use object detection to find the objects that need to be concerned in each image, and then further evaluate and judge these objects. In some embodiments, the content of the evaluation and judgment may include whether there are human errors, missing parts, missing screws, and / or scratches on the product appearance in each image, etc., but not limited thereto. In some embodiments, the image analysis model 250 may be a deep learning model such as YOLO (You Only Look Once) and / or a convolutional neural network model such as Mask-RCNN, etc., but not limited thereto. More specifically, the image analysis model 250 may be YOLOv4, but not limited thereto.
[0076] Since the image analysis model 250 has been pre-trained with multiple sets of data (each set of data can respectively include a product image set with annotation frames and its corresponding detection results), after receiving the product image set, the trained image analysis model 250 can analyze each image in the product image set and generate the detection and analysis results of the product image set. In some embodiments, the multiple sets of data can be, for example, at least 30 sets of image data, and each set of image data can have content such as correctly and / or incorrectly labeled screws, dust covers, stickers, and / or connection holes manually annotated.
[0077] In some embodiments, in order to confirm whether the trained image analysis model 250 has sufficient accuracy, a part of the data (such as one-tenth of the multiple sets of data) can be randomly selected from the multiple sets of data as test data, and the accuracy of the image analysis model 250 can be evaluated by using the test data. In some embodiments, the trained image analysis model 250 can have an accuracy of at least 0.990, preferably at least 0.996, so that the image analysis model 250 can more accurately generate detection and analysis results based on the product image set.
[0078] The product detection database 260 is configured to be coupled to the processing module 220 and is configured to store data related to the product to be detected. More specifically, the processing module 220 can store the data related to the product to be detected in the product detection database 260 through the output module 240. In addition, the processing module 220 can also read the data related to the previously stored product to be detected from the product detection database 260 through the receiving module 210.
[0079] In some embodiments, the product detection database 260 can be, for example, MySQL, but is not limited thereto. In some embodiments, the product detection database 260 can store structured data. More specifically, the product detection database 260 can use three tables to store the data related to the product to be detected, where the three tables can be a first table (such as a main table), a second table (such as an image table), and a third table (such as a detected object table). The first table can store, for example, the serial number (i.e., product identification code), product number, process, station, judgment result (i.e., detection and analysis result), error category, test time, and / or creation time of the product to be detected. The second table can store, for example, the product image set and / or image path of the product to be detected. The third table can store the category name and / or detection frame.
[0080] Thereby, the computer device 200 provided by the present application can generate a detection analysis result of the product to be detected based on the product image set of the product to be detected, and thus can automatically detect the appearance of the product to be detected without manual inspection, and can have better detection effect and / or detection efficiency.
[0081] In some embodiments, those with ordinary knowledge in the technical field to which the present application belongs can use VisualStudio Code as the development environment, and use python3.8 as the programming language for compilation to implement each image analysis step to be executed by the processing module 220. In some embodiments, those with ordinary knowledge in the technical field to which the present application belongs can further use function libraries such as OpenCV (image processing), NumPy (image processing), Darknet (image inference), PyQt5 (UI interface), dbr (barcode processing), pypylon (camera kit), and / or Requests (http request). In some embodiments, those with ordinary knowledge in the technical field to which the present application belongs can also use toolkits such as Django to process the backend data collection application programming interface (API).
[0082] Please refer to Figure 4 , Figure 4 which is a flowchart showing each image analysis step executed by the processing module 220 of the first embodiment of the present application. Specifically, the image analysis steps executed by the processing module 220 include image analysis steps S410, S420, and S430.
[0083] In the image analysis step S410, a product image set is received. More specifically, by executing the image analysis step S410, the processing module 220 can receive the product image set from an external device (such as the product detection device 100) through the receiving module 210. The received product image set includes six views of the product to be detected (i.e., top view, bottom view, front view, rear view, right side view, and left side view).
[0084] In the image analysis step S420, the product image set is input. More specifically, by executing the image analysis step S420, the processing module 220 can input each image in the received product image set into an image analysis model 250 such as YOLOv4. In some embodiments, the image analysis step S420 can be executed after the image analysis step S410.
[0085] In the image analysis step S430, the content of the product image set is analyzed, and a detection and analysis result is generated. More specifically, by executing the image analysis step S430, the image analysis model 250 such as YOLOv4 can perform image analysis on each image in the received product image set and generate a corresponding detection and analysis result. In some embodiments, the image analysis step S430 may be executed after the image analysis step S420. In some embodiments, the image analysis performed by the image analysis model 250 on each image in the product image set may be to use object detection to find the objects that need to be concerned in each image, and then further evaluate and judge these objects. In some embodiments, the content of the evaluation and judgment may include whether there are human errors, missing parts, missing screws, and / or scratches on the product appearance in each image, but is not limited thereto. In some embodiments, the detection and analysis result may be, for example, detection qualified and / or detection unqualified, but is not limited thereto.
[0086] Thereby, by executing each of the image analysis steps as Figure 4 shown, the processing module 220 of the computer device 200 can generate the detection and analysis result of the product to be detected based on the product image set of the product to be detected, and thus can automatically detect the appearance of the product to be detected without manual inspection, and can have better detection effect and / or detection efficiency.
[0087] Please refer to Figure 5 , Figure 5 which is a flowchart showing each image analysis step executed by the processing module 220 of the second embodiment of the present application. Specifically, the image analysis steps executed by the processing module 220 include image analysis steps S410, S420, S430, S510, S515, S520, S525, and S530, wherein the image analysis steps S410, S420, and S430 may be substantially the same as the Figure 4 image analysis steps shown.
[0088] In the image analysis step S510, it is determined whether the detection analysis result is a failed detection. More specifically, by performing the image analysis step S510, the processing module 220 can judge the detection analysis result generated by the image analysis model 250 such as YOLOv4 to determine the subsequent image analysis step to be executed by the processing module 220. In some embodiments, the image analysis step S510 may be executed after the image analysis step S430. Specifically, when the detection analysis result is a failed detection, the processing module 220 will subsequently execute the image analysis step S520; conversely, when the detection analysis result is not a failed detection (i.e., the detection analysis result is a passed detection), the processing module 220 will subsequently execute the image analysis step S515. In some embodiments, the detection analysis result may be represented, identified, and / or stored in the form of a flag. More specifically, a passed detection in the detection analysis result may be represented as a low logic level, while a failed detection in the detection analysis result may be represented as a high logic level.
[0089] In the image analysis step S515, no additional image analysis steps are performed (i.e., it ends). That is, when the detection analysis result is not a failed detection (i.e., the detection analysis result is a passed detection), the processing module 220 does not need to perform additional image analysis steps on the product image set of the product to be detected.
[0090] In the image analysis step S520, it is determined whether the detection analysis result is a misjudgment. More specifically, when the detection analysis result is a failed detection, by performing the image analysis step S520, the processing module 220 can perform a secondary judgment on the product to be detected that has already been determined to be a failed detection to determine the subsequent image analysis step to be executed by the processing module 220. Specifically, when the detection analysis result is a misjudgment, the processing module 220 will subsequently execute the image analysis step S520; conversely, when the detection analysis result is not a misjudgment, the processing module 220 will subsequently execute the image analysis step S525.
[0091] In some embodiments, when the detection analysis result is unqualified, the processing module 220 can actively send a notification message to notify relevant personnel to make a secondary judgment on the product to be detected that has been determined to be unqualified in the detection, and generate a secondary judgment result. In some embodiments, the secondary judgment result can be represented, identified, and / or stored in the form of a flag. More specifically, the qualified detection in the secondary judgment result can be represented as a low logic level, while the unqualified detection in the secondary judgment result can be represented as a high logic level. After the secondary judgment on the product to be detected is completed, the processing module 220 can determine whether the detection analysis result is a misjudgment based on the detection analysis result and the secondary judgment result. More specifically, the processing module 220 can compare the flag representing the detection analysis result with the flag representing the secondary judgment result. When the flags of both are high logic levels (i.e., unqualified in the detection), the processing module 220 will determine that the detection analysis result is not a misjudgment; otherwise, when the flag representing the detection analysis result is a high logic level (i.e., unqualified in the detection) and the flag representing the secondary judgment result is a low logic level (i.e., qualified in the detection), the processing module 220 will determine that the detection analysis result is a misjudgment.
[0092] In some embodiments, whether the detection analysis result is a misjudgment can be represented, identified, and / or stored in the form of a flag. More specifically, the detection analysis result being a misjudgment can be represented as a low logic level, while the detection analysis result not being a misjudgment can be represented as a high logic level.
[0093] In the image analysis step S525, no additional image analysis steps are performed (i.e., end). That is, when the detection analysis result is unqualified and the detection analysis result is not a misjudgment, the processing module 220 does not need to perform additional image analysis steps on the product image set of the product to be detected.
[0094] In the image analysis step S530, the detection analysis result is corrected. More specifically, when the detection analysis result is unqualified and the detection analysis result is a misjudgment, by performing the image analysis step S530, the processing module 220 can correct the detection analysis result that has been determined to be unqualified in the detection. That is, since the detection analysis result is a misjudgment, the processing module 220 can actively correct the unqualified detection to a qualified detection (i.e., correct the flag representing the detection analysis result from a high logic level to a low logic level).
[0095] Thus, by executing as Figure 5For each of the image analysis steps shown, the processing module 220 of the computer device 200 can further make a secondary judgment on the product to be detected that has been determined to be unqualified in the detection, and can correct the detection analysis result that has been determined to be unqualified in the detection, so as to avoid the influence caused by the misjudgment of the image analysis model 250, and further improve the accuracy of the detection analysis result.
[0096] Please refer to Figure 6 , Figure 6 which is a flowchart showing each image analysis step executed by the processing module 220 of the third embodiment of the present application. Specifically, the image analysis steps executed by the processing module 220 include image analysis steps S410, S420, S430, S510, S515, S520, S525, S530, and S610. Among them, the image analysis steps S410, S420, and S430 can be substantially the same as Figure 4 the image analysis steps shown, and the image analysis steps S510, S515, S520, S525, and S530 can be substantially the same as Figure 5 the image analysis steps shown.
[0097] In the image analysis step S610, the image analysis model 250 is trained. More specifically, when the detection analysis result is unqualified in the detection and the detection analysis result is a misjudgment, by executing the image analysis step S610, the processing module 220 can use the corrected detection analysis result and its corresponding product image set as training data to further train the image analysis model 250 that has been previously trained with multiple data, so as to improve the accuracy of the image analysis model 250. In some embodiments, the image analysis step S610 can be executed after the image analysis step S530.
[0098] Thereby, by executing each image analysis step as Figure 6 shown, the processing module 220 of the computer device 200 can further use the training data (new data) to further train the image analysis model 250, thereby improving the image analysis ability of the image analysis model 250 and increasing the accuracy of the image analysis model 250.
[0099] Please refer to Figure 7 , Figure 7 which is a flowchart showing each image analysis step executed by the processing module 220 of the fourth embodiment of the present application. Specifically, the image analysis steps executed by the processing module 220 include image analysis steps S410, S420, S430, and S710. Among them, the image analysis steps S410, S420, and S430 can be substantially the same as Figure 4 the image analysis steps shown.
[0100] In the image analysis step S710, data related to the product to be detected is stored. More specifically, by executing the image analysis step S710, the processing module 220 of the computer device 200 can store the data related to the product to be detected in the product detection database 260. In some embodiments, the image analysis step S710 may be executed after the image analysis step S430. In some embodiments, the data related to the product to be detected may include, for example, the product identification code of the product to be detected, the product image set, and the detection analysis result, but is not limited thereto.
[0101] Thereby, by executing each of the image analysis steps as Figure 7 shown, the processing module 220 of the computer device 200 can further store the data related to the product to be detected in the product detection database 260, so that authorized users (such as administrators) can track, view, and / or update these data, and make good use of these data (such as clustering the data or reverse tracking, etc.).
[0102] Please refer to Figure 8 , Figure 8 FIG. is a flowchart showing the steps of the method for automatically detecting products according to the first embodiment of the present application. The method for automatically detecting products may include steps S810, S820, S830, and S840. In some embodiments, the method for automatically detecting products may be executed by the system 50 for automatically detecting products described in the present application.
[0103] In step S810, a product image set is generated. More specifically, by executing step S810, the product detection device 100 can capture images of the product to be detected placed in the accommodation space of the detection platform 110 to generate a product image set. The generated product image set includes six views of the product to be detected (i.e., the top view, the bottom view, the front view, the rear view, the right side view, and the left side view).
[0104] In step S820, the product image set is received. More specifically, by executing step S820, the processing module 220 of the computer device 200 can receive the product image set from an external device (such as the product detection device 100) through the receiving module 210. The received product image set includes six views of the product to be detected (i.e., the top view, the bottom view, the front view, the rear view, the right side view, and the left side view). In some embodiments, step S820 may be executed after step S810.
[0105] In step S830, a product image set is input. More specifically, by executing step S830, the processing module 220 of the computer device 200 can input each image in the received product image set into an image analysis model 250 such as YOLOv4. In some embodiments, step S830 may be executed after step S820.
[0106] In step S840, the content of the product image set is analyzed, and a detection and analysis result is generated. More specifically, by executing step S840, an image analysis model 250 such as YOLOv4 can perform image analysis on each image in the received product image set and generate a corresponding detection and analysis result. In some embodiments, step S840 may be executed after step S830. In some embodiments, the image analysis performed by the image analysis model 250 on each image in the product image set may be to use object detection to find the objects that need to be concerned in each image, and then further evaluate and judge these objects. In some embodiments, the content of the evaluation and judgment may include whether there are human errors, missing parts, missing screws and / or scratches on the product appearance in each image, but is not limited thereto. In some embodiments, the detection and analysis result may be, for example, detection qualified and / or detection unqualified, but is not limited thereto.
[0107] Thus, by executing each step as Figure 8 shown, the system 50 for automatically detecting products can generate a product image set of the product to be detected, and generate a detection and analysis result of the product to be detected based on the product image set, so as to automatically detect the appearance of the product to be detected without manual inspection, and can have better detection effect and / or detection efficiency.
[0108] Please refer to Figure 9 , Figure 9 which is a detailed flowchart illustrating step S810 of the method for automatically detecting products. That is, generating a product image set (i.e., step S810) as Figure 8 shown can be completed through steps S910, S920 and S930 as Figure 9 shown. Steps S910, S920 and S930 can be executed by the product detection device 100 described in this application.
[0109] In step S910, an image of the product to be detected is taken. More specifically, by performing step S910, the four image capture modules of the product detection device 100 (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) can perform a first image capture of the product to be detected placed in the accommodation space of the detection platform 110, that is, first generate images of four faces of the product to be detected (for example, a top view, a bottom view, a rear view, and a right side view).
[0110] In step S920, the detection platform is rotated. More specifically, by performing step S920, after the product detection device 100 receives an instruction sent from the computer device 200 through a programmable logic controller, the rotation module of the product detection device 100 can rotate the detection platform 110. In some embodiments, step S920 can be executed after step S910. In some embodiments, the rotation module of the product detection device 100 can rotate the detection platform 110 in a fixed rotation direction (for example, rotate 180 degrees clockwise horizontally). Since the rotation module of the product detection device 100 can rotate the detection platform 110, the product to be detected placed in the accommodation space of the detection platform 110 can also be rotated as the detection platform 110 rotates.
[0111] In step S930, an image of the product to be detected is taken. More specifically, by performing step S930, the four image capture modules of the product detection device 100 (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) can perform a second image capture of the product to be detected placed in the accommodation space of the detection platform 110. In some embodiments, step S930 can be executed after step S920. Since the four image capture modules of the product detection device 100 (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) are respectively set at different and fixed positions in the product detection device 100, after the product to be detected placed in the accommodation space of the detection platform 110 is rotated as the detection platform 110 rotates, the four image capture modules of the product detection device 100 (i.e., the first image capture module 130A, the second image capture module 130B, the third image capture module 130C, and the fourth image capture module 130D) can further generate images of the other two faces of the product to be detected (for example, a front view and a left side view).
[0112] Thus, by performing as Figure 9The various steps shown enable the system 50 for automatically detecting products to generate six - view images of the product to be detected with only four imaging modules, thereby reducing the components and configuration space required by the product detection device 100 and lowering the manufacturing cost of the product detection device 100. In addition, by performing the steps as Figure 9 shown, the system 50 for automatically detecting products can also generate six - view images of the product to be detected with only one horizontal rotation of the product to be detected, thereby reducing the equipment complexity and / or operation complexity of the product detection device 100. That is, by performing the steps as Figure 9 shown, the system 50 for automatically detecting products can generate six - view images of the product to be detected while taking both cost and complexity into account.
[0113] Please refer to Figure 10 , Figure 10 which is an image schematic diagram illustrating the detection and analysis results of two different products to be detected. Taking the image 1010 as an example, after the system for automatically detecting products provided in the present application performs image shooting and image analysis on the product to be detected, the system can generate a detection and analysis result of the image 1010 as qualified. Taking the image 1020 as an example, after the system for automatically detecting products provided in the present application performs image shooting and image analysis on the product to be detected, since there are appearance defects in the connector in the image 1020, the system can generate a detection and analysis result of the image 1020 as unqualified.
[0114] Through the system for automatically detecting products and / or the method for automatically detecting products provided in the present application, the appearance of the product to be detected can be automatically detected, and better detection effects and / or detection efficiency can be achieved. More specifically, after training an image analysis model (e.g., YOLOv4) with 384 pre - labeled images (i.e., six - view images of 64 groups of products) and completing the training, the system for automatically detecting products provided in the present application actually performs image analysis on 1080 images (i.e., six - view images of 180 groups of products to be detected) and generates detection and analysis results, and its recognition accuracy (i.e., the accuracy of the detection and analysis results) can reach 99.93%.
[0115] In some embodiments, each image analysis step performed by a processing module in a computer device can be compiled into a computer program product, which can then be stored in a computer-readable storage medium. That is to say, the present application can also separately provide a computer-readable storage medium, and the computer-readable storage medium can store a computer program product corresponding to the image analysis steps described in the present application. In some embodiments, the computer-readable storage medium can be, for example, a hard disk, an optical disc, a magnetic disk, a USB flash drive, or a database accessible via a network, but is not limited thereto. After the computer device loads the computer program product stored in the computer-readable storage medium and executes the computer program product, the computer device can execute the corresponding image analysis steps to automatically detect the appearance of the product to be detected, and can have better detection effects and / or detection efficiency.
[0116] In some embodiments, each image analysis step performed by a processing module in a computer device can be compiled into a computer program product. That is to say, the present application can also separately provide a computer program product, and the computer program product can include the image analysis steps described in the present application, so that after the computer device loads the computer program product and executes the computer program product, the computer device can execute the corresponding image analysis steps to automatically detect the appearance of the product to be detected, and can have better detection effects and / or detection efficiency.
[0117] The present application has been further described through the above embodiments and the accompanying drawings, but those with ordinary knowledge in the technical field to which the present application belongs can still make many modifications and changes without departing from the scope and spirit set forth in the patent scope of the present application. Therefore, the protection scope of the present application should still be defined by the patent scope and should not be limited by the content disclosed in the specification.
Claims
1. A system for automatically detecting products, characterized in that, Applicable to detecting a product to be detected to generate a detection analysis result, the system includes: A computer device; and A product detection device configured to be coupled to the computer device; Wherein, the product detection device includes: A detection platform configured to provide an accommodation space, A rotation module configured to operably rotate the detection platform, and Four image capture modules respectively disposed at different positions in the product detection device and respectively configured to capture images of the product to be detected placed in the accommodation space to generate a product image set, wherein the product image set includes six views of the product to be detected, Wherein, the computer device includes: A storage module configured to store a computer program product, and A processing module configured to be coupled to the storage module and configured to, after loading and executing the computer program product, be able to perform at least one image analysis step to generate the detection analysis result based on the product image set.
2. The system according to claim 1, characterized in that, The at least one image analysis step includes: Receiving the product image set; Inputting the product image set into an image analysis model; and Analyzing the content of the product image set through the image analysis model and generating the detection analysis result.
3. The system according to claim 2, wherein The at least one image analysis step further includes: When the detection analysis result is unqualified, further determining whether the detection analysis result is a misjudgment, Wherein, when the detection analysis result is a misjudgment, correcting the detection analysis result.
4. The system according to claim 3, wherein The at least one image analysis step further includes: When the detection analysis result is a misjudgment, using the corrected detection analysis result and the product image set as a training data to further train the image analysis model with the training data.
5. The system according to claim 2, wherein The at least one image analysis step further includes: Storing a product identification code of the product to be detected, the product image set, and the detection analysis result in a product detection database.
6. A method for automatically detecting products, characterized in that, Applicable to be executed by the system for automatically detecting products according to claim 1, the method includes: Generating the product image set through the product detection device; Receiving the product image set through the computer device; Inputting the product image set into an image analysis model through the computer device; and Analyzing the content of the product image set through the image analysis model and generating the detection analysis result; Wherein, the product image set includes six views of the product to be detected.
7. The method according to claim 6, wherein Further includes: When the detection analysis result is unqualified, determining whether the detection analysis result is a misjudgment through the computer device, Wherein, when the detection analysis result is a misjudgment, correcting the detection analysis result through the computer device.
8. The method according to claim 7, characterized in that Further includes: When the detection analysis result is a misjudgment, using the corrected detection analysis result and the product image set as a training data through the computer device to further train the image analysis model with the training data.
9. The method according to claim 6, wherein Further includes: Storing a product identification code of the product to be detected, the product image set, and the detection analysis result in a product detection database through the computer device.
10. The method according to claim 6, characterized in that, Generating the product image set through the product detection device includes: The image capture modules respectively capture images of the product to be detected placed in the accommodation space; The rotation module rotates the detection platform; and The image capture modules respectively capture images of the product to be detected placed in the accommodation space again.