Flexible and intuitive alignment system for automated visual inspection systems

Abstract

To provide a flexible and intuitive arrangement system for automated visual inspection system.SOLUTION: In a flexible and intuitive arrangement system 1, an electronic device 11 is electrically-connected to a main control electronic device 23 of an automated visual inspection system, and is information-connected to N imagers and N robot arms 21 of the automated visual inspection system. First, the arm installation operation is performed for each robot arm, the imager installation operation is performed for the imager 22 grasped by the robot arm, and then control is performed so as to image a designated article by the imager. The system records arrangement parameters of the imager and the robot arm in association with each other and uploads multiple article images of the article to a remote electronic device. The remote electronic device generates an article defect identification model by using the multiple article images. By installing the article defect identification model in the main control electronic device, automated defect inspection is performed to the article.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the technical field of machine vision, and more particularly to a flexible and intuitive alignment system for automated visual inspection systems. [Background technology]

[0002] Generally speaking, semi-finished products or finished products manufactured using industrial technology require quality inspection to determine whether they conform to industrial standards. Quality inspection includes cosmetic inspection and functional testing. Practical experience has shown that quality inspectors often overlook or erroneously inspect industrial products when performing visual inspections. In response to this, automated visual inspection systems using machine vision technology have been developed and implemented in automated production lines to perform visual inspections of semi-finished or finished industrial products instead of human labor. For example, Patent Document 1 (Patent Document 1) discloses an automated visual inspection system used to inspect nuts, gears, etc. for defects. The automated visual inspection system mainly includes a light source, at least one camera, and a processing device (also referred to as an electronic device).

[0003] Generally speaking, before an industrial product leaves the factory, it must be inspected for defects from multiple angles, covering the entire product itself. At the same time, the market trend for industrial products is toward small-lot diversification and shorter product lifecycles. Therefore, the appearance of different models of industrial products changes rapidly. Therefore, the automatic visual inspection system on the production line may need to be installed frequently to rapidly change the appearance of the product. As a result, frequent changes may require manual intervention. For example, when inspecting gears for defects, quality inspectors will inspect several surfaces of the gear in accordance with relevant inspection regulations to check for foreign matter, dents, scratches, discoloration, and missing threads (i.e., missing internal threads in screw holes). To enable the automated visual inspection system to accurately inspect a lot of gears transported on a conveyor belt for defects, system manufacturers typically install a database and defect identification software in the electronic device. It is worth noting that the database pre-stores a number of labeled images of defective gears and images of normal gears (golden samples), and these pre-classified gear images are processed into a number of reference defect feature images, such as those with foreign matter, dents, scratches, different colors, and missing taps. In this configuration, the camera captures and acquires at least one gear image from a single gear, and the defect identification software then first performs feature extraction processing on the gear image to acquire a gear feature image. The defect identification software then performs a feature comparison between the gear feature image and at least one reference defect feature image stored in the database to determine whether the current test gear has a defect as defined by the associated inspection specification.

[0004] Engineers experienced in the development and manufacture of automated visual inspection systems for defect inspection will understand that, if the automated visual inspection system is to be used to accurately inspect and identify defects on Model A gears, the engineer must tune the imaging parameters of each camera, such as the installation height, installation angle, imaging distance, and aperture value, as well as the light source's illumination range and light intensity. To make matters more complicated, when a gear manufacturer's quality inspector discovers a new type of defect during the production process, the engineer may not be able to capture images using the vision system's parameters, which have already been tuned, due to differences in the defect's location or desired optical conditions. Therefore, the quality inspector cannot promptly provide multiple gear images with the new defect characteristics as soon as possible, and the engineer must create pre-classified gear images. This assumes that the camera, with its parameters already configured, can smoothly capture images of any defect. Next, the engineer can use machine learning technology to build a defect identification model included in the defect identification software. The defect identification model is generated using the following steps a to e. In step a, a plurality of sample images are input into a machine learning model to obtain predicted type information for each of the sample images, where the plurality of sample images include a plurality of gear images and a plurality of gear images with defect features, and each of the sample images is labeled (i.e., labeled). In step b, model parameters of the machine learning model are adjusted based on the difference between the predicted type information of the sample images and the type labels, thereby obtaining an adjusted machine learning model. In step c, the adjusted machine learning model is used to classify a plurality of test images to obtain a classification accuracy, where the plurality of test images include a plurality of gear images and a plurality of gear images with defect features. In step d, if the classification accuracy does not reach an accuracy threshold, all of the above steps are repeated.In step e, if the classification accuracy reaches the accuracy threshold, the adjusted machine learning model is set as the defect identification model to be integrated into the defect identification software.

[0005] In other words, if the gear manufacturer requests that its engineers retune the automated visual inspection system so that it can be used to accurately inspect and identify gears of model B, the engineers must tune the imaging parameters of each camera, such as the installation height, installation angle, imaging distance, and aperture value, as well as the light source's illumination range and illumination intensity. More importantly, if the gear manufacturer's quality inspectors do not have specialized knowledge of optical equipment, they will not be able to continuously provide multiple gear images with defect features during the production process by adjusting the optical equipment, and even the engineers will be unable to create corresponding labeled gear images and defect identification models. As a result, the optical setup work required to support the inspection of multiple gear models is not easy to complete, and it may require considerable specialized effort and time to complete. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Taiwan Registered Utility Model No. M355371U Summary of the Invention [Problem to be solved by the invention]

[0007] In view of this, the original automatic visual inspection system was developed to allow quality inspectors to set the relevant parameters of the light source and camera themselves, and to allow the quality inspectors to learn the quality inspection standards of different gear models through flexible and intuitive operation, thereby training a defect identification model to accurately inspect and identify the appearance defects of different gear models. For this reason, the inventors of the present invention have researched and devised as much as possible, and have finally developed and perfected a flexible and intuitive configuration system for the automatic visual inspection system of the present invention.

[0008] A primary objective of the present invention is to provide a flexible and intuitive configuration system implemented using electronic devices, which are electrically connected to a main control electronics of an automated visual inspection system and thereby information-linked with N cameras and N robotic arms of the automated visual inspection system. When performing a setup procedure for the automated visual inspection system, an arm installation operation is first performed for each robotic arm, and an imager installation operation is performed for each camera held by the robotic arm, and then the camera is controlled to photograph a designated item. In this process, the flexible and intuitive configuration system stores corresponding configuration parameters for the cameras and robotic arms, and uploads multiple item images to a remote electronics device, which then uses the multiple item images to generate an item defect identification model. Finally, by installing the item defect identification model in the main control electronics, the automated visual inspection system is suitable for use in automated defect inspection of the item.

[0009] For example, the flexible and intuitive configuration system of the present invention may be an application program installed on a tablet computer (i.e., the aforementioned electronic device). A quality inspector at a manufacturing plant can use the tablet computer to perform the above-described configuration procedure on any one of the automated visual inspection systems in the plant. After the automated visual inspection system is configured and configured, it can be used to perform automated defect inspections on designated products. In particular, the entire process of receiving the configuration procedure on the automated visual inspection system does not require the intervention of a technician. The quality inspector can intuitively adjust the angle of the robot arm by relying on their experience in quality inspections, and can complete the imaging of defect samples by taking photos via a smart handheld device. The technician can simply operate the remote electronic device to generate an article defect identification model based on multiple article images provided by the quality inspector. [Means for solving the problem]

[0010] In order to achieve the above object, in one embodiment of the flexible and intuitive configuration system provided by the present invention, it is used to perform setting operations on an automatic visual inspection system, so that the automatic visual inspection system is suitable for application to perform automated defect inspection on articles, wherein the automatic visual inspection system comprises a conveying device, N robot arms, N camera units held by the N robot arms respectively, and a main control electronic unit, where N is a positive integer of at least 1; the flexible and intuitive configuration system comprises an electronic unit electrically connected to the main control electronic unit and information-linked with the N camera units and the N robot arms through the main control electronic unit, wherein the electronic unit comprises a first processor and a first memory; and storing a first application program in the first memory, the first processor accessing the first memory to execute the first application program, thereby enabling the following functions: when the robot arm is operated to move the image capture device K times, the image capture device has an image capture height, an image capture angle, and an image capture distance after each movement, the image capture device records one image capture height, one image capture angle, and one image capture distance as a set of external image capture parameters after each movement, and obtains the K sets of external image capture parameters by summing them up, where K is a positive integer of at least 1; and the image capture device continues to accept an image capture device adjustment operation after each movement to obtain an aperture, a depth of field, and a shutter speed.In the case where the main control electronic device has a plurality of aperture values, a plurality of depths of field, a plurality of shutter speeds, a plurality of photosensitivities, and a plurality of focal lengths, one aperture value, one depth of field, one shutter speed, one photosensitivity, and one focal length are recorded as one set of internal imaging parameters, and the camera is moved K times to acquire a total of K sets of the internal imaging parameters. The camera takes pictures of articles transported by the transport equipment to acquire a plurality of first article images having steady features and a plurality of second article images having defect features, and then uploads the plurality of first article images and the plurality of second article images to a remote electronic device. The remote electronic device generates at least one update module using the plurality of first article images and the plurality of second article images, and then uses the update module to perform a model update operation on the first article defect identification model installed in the main control electronic device, or uses the update module to install a second article defect identification model in the main control electronic device.

[0011] In one embodiment, the electronic device is any one selected from the group consisting of a smartphone, a tablet computer, a desktop computer, an all-in-one computer, and a notebook computer.

[0012] In one possible embodiment, the imager is included in a portable electronic device, which is any one selected from the group consisting of a smartphone and a tablet computer, and the portable electronic device includes an inertial measurement unit (IMU) and a laser imaging detection and ranging (LiDAR) unit, and the imager is used to obtain three-dimensional movement data and three-dimensional measurement data after each movement and transmit the three-dimensional movement data and the three-dimensional measurement data to the electronic device via the portable electronic device, so that the electronic device aligns the three-dimensional movement data and the three-dimensional measurement data in the external imaging parameters.

[0013] In another possible embodiment, the imaging device is included in a portable electronic device, wherein the portable electronic device includes an inertial measurement unit (IMU) and is mounted on a fixture connected to the robot arm, and a laser imaging detection and ranging (LiDAR) unit is mounted on the fixture and electrically connected to the portable electronic device, and after each movement of the imaging device, the inertial measurement unit and the laser imaging detection and ranging unit respectively obtain three-dimensional movement data and three-dimensional measurement data, and the portable electronic device transmits the three-dimensional movement data and the three-dimensional measurement data to the electronic device, so that the electronic device aligns the three-dimensional movement data and the three-dimensional measurement data with the external imaging parameters.

[0014] In one embodiment, the first processor records and stores in the first memory N sets of the external imaging parameters corresponding to the N robot arms, and records and stores in the first memory N sets of the internal imaging parameters corresponding to the N imagers.

[0015] In one embodiment, after the first processor executes the first application program, an operation interface is displayed on the display of the electronic device, and a user can perform label processing on the plurality of first item images and the plurality of second item images by operating the operation interface.

[0016] In one embodiment, the conveying equipment includes a conveyor belt mechanism, a motor used to drive the conveyor belt mechanism, and a distance sensor, and the first processor, when executing the first application program, further enables the following functions: when the motor is set to operate at a rotation rate, when the conveyor belt mechanism is driven, the rotation rate is recorded as a motor control parameter; when the item is transported by the conveyor belt mechanism and it is within the imaging range of the jth imaging device, the jth movement distance is recorded as an item positioning parameter, where j∈N is satisfied; and, based on the motor control parameter and the N item positioning parameters, a parameter update operation is performed on the equipment control software installed in the main control electronic device.

[0017] In one embodiment, the automatic visual inspection system further includes N light sources, and the first processor, when executing the first application program, enables the following functions: if the light source has a light irradiation range, a light intensity, a color temperature, and a light color through a light source adjustment operation, record the light irradiation range, the light intensity, the color temperature, and the light color as a set of lighting parameters, and perform an update operation of the parameters on the device control software installed in the main control electronic device according to the lighting parameters.

[0018] In one embodiment, the main control electronic device includes a second processor and a second memory, and stores a second application program in the second memory. The second processor executes the second application program through access to the second memory, thereby enabling the following functions: control the robot arm to perform an operation based on the corresponding external imaging parameters, thereby correspondingly moving the imager; control the light source to emit inspection light provided based on the corresponding lighting parameters toward the article, thereby controlling the imager to take an article image from the article based on the corresponding internal imaging parameters; and perform a defect identification operation on the article image using the article defect identification model, thereby determining whether the article has at least one defect.

[0019] In one embodiment, the automatic visual inspection system further includes at least one material discharge device installed near the material outlet of the conveying equipment, and the first processor, when executing the first application program, further enables the following function: move one of the items N travel distances on the conveyor belt mechanism, and then move it an N+1th travel distance, and then record the N+1th travel distance as a material discharge parameter; when the material discharge device is operated to discharge the material of one of the items into a collection box, the item has an initial height, a placement angle, and a final height, and record the initial height, the placement angle, and the final height as a set of material discharge device parameters; and perform an update operation on the device control software installed in the main control electronic device based on the material discharge parameter and the N material discharge device parameters.

[0020] In one embodiment, the automatic visual inspection system further includes at least one article placement device installed near the material supply port of the conveying equipment, and the first processor, according to executing the first application program, further enables the following function: when the article placement device is operated to place one of the articles on the conveying equipment, the article has an initial height, a placement angle, and a final height, and records the initial height, the placement angle, and the final height as article placement device parameters, and performs an update operation on the parameters of the equipment control software installed in the main control electronic device based on the article placement device parameters. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a block diagram of an automated vision inspection system including a flexible and intuitive placement system according to the present invention; [Figure 2A] FIG. 1 is a first perspective view of an automated visual inspection system. [Figure 2B] FIG. 2 is a second perspective view of the automated visual inspection system. [Figure 3A] FIG. 1 is a first perspective view of the j-th robot arm and the j-th imaging device. [Figure 3B] FIG. 2 is a second perspective view of the j-th robot arm and the j-th imaging device. [Figure 4A] FIG. 1 is a first perspective view of the jth robot arm, the jth image capture device, and the jth light source. [Figure 4B] FIG. 10 is a second perspective view of the jth robot arm, the jth image capture device, and the jth light source. DETAILED DESCRIPTION OF THE INVENTION

[0022] To more clearly describe the flexible and intuitive configuration system for an automated visual inspection system proposed by the present invention, a preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Figure 1 is a block diagram of an automated visual inspection system including a flexible and intuitive configuration system according to the present invention. Figures 2A and 2B are first and second perspective views of the automated visual inspection system. As shown in Figures 1, 2A, and 2B, the flexible and intuitive configuration system 1 proposed by the present invention is used to configure the automated visual inspection system 2, making the automated visual inspection system 2 (hereinafter referred to as "AVI system 2") suitable for automated defect inspection of articles. For example, the AVI system 2 includes a conveying equipment 20, an article loading device 26, N robotic arms 21, N camera units 22, a main control electronic device 23, N light sources 24, and a material discharge device 25, where N is a positive integer equal to or greater than 1, and the main control electronic device 23 is electrically connected to the conveying equipment 20, the article loading device 26, the N robotic arms 21, the N camera units 22, the main control electronic device 23, the N light sources 24, and the material discharge device 25. More specifically, device control software and product defect identification software are installed in the main control electronic device 23, and the device control software and the product defect identification software are integrated into an automatic vision inspection application program.

[0024] Therefore, when a defect inspection is performed on a lot of articles (e.g., gears) using the AVI system 2, the main control electronic device 23 executes the device control software to control the conveying equipment 20, the article placement device 26, the N robot arms 21, the N imagers 22, the N light sources 24, and the material discharge device 25 to operate at appropriate times. More specifically, the article placement device 26 (i.e., the robot arm with a jig) is controlled to grip an article positioned on the conveying equipment 20 and then place the article on the conveying equipment 20, thereby positioning the article on the conveying equipment 20 along a placement angle. Continuously, the first robot arm 21 grasps the first imager 22 and, as it operates under control, moves the first imager 22 to have a set of external imaging parameters including an imaging height, an imaging angle, and an imaging distance. Next, the first camera 22 is controlled to photograph the article within its one imaging range, thereby obtaining an article image. To further explain, the first robotic arm 21 and the first camera 22 constitute a first inspection station. In the first inspection station, the first camera 22 is arranged to have a first set of external imaging parameters, and then photographs the article to obtain a first article image. In practical applications, in the first inspection station, the first camera 22 is also arranged to have a second set of external imaging parameters (i.e., a different imaging height, imaging angle, and imaging distance), and then photographs the article to obtain a second article image. Based on this, when the article moves to the first inspection station, a third and a fourth article image may also be photographed.

[0025] For the same reason, when the article is continuously transported by the conveying equipment 20 to the first inspection station, which is composed of a second robot arm 21 and a second imager 22, the second robot arm 21 is controlled to operate, and the second imager 22 is moved so that it has a set of external imaging parameters, including an imaging height, an imaging angle, and an imaging distance. The second imager 22 is then controlled to photograph the article within its imaging range, thereby obtaining an article image. To elaborate further, during the second inspection station, the second imager 22 is positioned to have the first set of external imaging parameters, and then photographs the article to obtain a first article image. In practical applications, during the second inspection station, the second imager 22 is also positioned to have a second set of external imaging parameters (i.e., a different imaging height, imaging angle, and imaging distance), and then photographs the article to obtain a second article image. Based on this analogy, when the item moves to the second inspection station, a third and a fourth image of the item may be taken.

[0026] To further explain, in the jth inspection station (j∈N), the jth imager 22 is configured to have a set of internal imaging parameters including aperture, depth of field, shutter speed, ISO, and focus, and the imager 22 will take an image of the article if it accepts the specified parameter configuration. Thus, after the article has passed through N inspection stations, the main control electronic device 23 successfully acquires multiple article images, and then executes the article defect identification software to perform a defect identification operation on each of the article images, thereby determining whether the article has at least one defect. Specifically, the product defect identification software includes an image processing unit (e.g., spatial filtering), a feature extraction unit, and an product defect identification unit, wherein the feature extraction unit is used to perform feature extraction processing on the product image to obtain an product feature image, and the product defect identification unit has a pre-trained product defect identification model and uses the product defect identification model to perform feature comparison between the product feature image and at least one reference defect feature image stored in the database, thereby determining whether the currently inspected product has a defect defined by a related inspection specification.

[0027] After completing the defect inspection and identification of the items, the main control electronic device 23 controls the material discharge device 25 (i.e., the robot arm with a jig) to remove the clamped items belonging to NG (not good) items from the conveying equipment 20 and place them in the NG item accumulation box 5. On the other hand, items that have passed the defect inspection (i.e., items belonging to normal products) are sent into the normal product accumulation box 4.

[0028] As can be seen from the above description, after the equipment control software and the article defect identification software are installed in the main control electronic device 23, the AVI system 2 including the main control electronic device 23, the conveying equipment 20, the article loading device 26, the N robot arms 21, the N camera devices 22, the main control electronic device 23, the N light sources 24, and the material discharge device 25 may be used to perform defect inspection on one lot of article A (e.g., gears with model number A). However, based on practical experience, it is not necessarily true that the AVI system 2 is applicable to performing defect inspection on one lot of article B (e.g., gears with model number B). In this case, as shown in Figures 1, 2A, and 2B, a quality inspector at a manufacturing plant can perform setting procedures for the same AVI system 2 by operating the flexible and intuitive configuration system 1 of the present invention, and the AVI system 2 can be applied to perform defect inspections on one lot of B products after undergoing a series of setting operations.

[0029] As shown in FIGS. 1, 2A, and 2B, the flexible and intuitive configuration system 1 of the present invention mainly comprises an attached electronic device 11 and AVI system configuration software (referred to as a first application program) installed in the electronic device 11. A quality inspector at a manufacturing plant can use the tablet computer to perform a series of configuration operations for any one AVI system 2 in the plant. According to the design of the present invention, the electronic device 11 is electrically connected to the main control electronic device 23 and is information-linked to the N camera cameras 22 and the N robot arms 21 through the main control electronic device 23. The electronic device 11 includes a first processor 11P and a first memory 11M. The first memory 11M stores a first application program (i.e., the AVI system configuration software). The first processor 11P accesses the first memory 11M to execute the first application program, thereby enabling multiple functions and completing a series of configuration operations for the AVI system 2. In one embodiment, after the first processor 11P executes the first application program, it causes the display of the electronic device 11 to display an operation interface (user interface: UI), so that a quality inspector at the manufacturing plant can perform the setting procedure of the AVI system 2 by operating the operation interface.

[0030] First, a quality inspector can manually rotate the jth (e.g., first) robot arm 21, and the jth imager 22 can have an imaging height, imaging angle, and imaging distance (i.e., a set of external imaging parameters). Next, the quality inspector can set the aperture value, depth of field, shutter speed, photosensitivity, and focal length (i.e., a set of internal imaging parameters) of the jth imager 22, and then the jth imager 22 is controlled to capture an image of an article within its imaging range, thereby obtaining an article image. It should be understood that the jth robot arm 21 and the jth imager 22 constitute the jth inspection station. FIGS. 3A and 3B are first and second perspective views of the jth robot arm 21 and the jth imager 22. As shown in FIG. 3A, in the jth inspection station, the jth imager 22 is positioned to have a first set of external imaging parameters, and then captures an image of the article by capturing an image of the article. In practical application, as shown in Figure 3B, during the jth inspection station, the first imager 22 is also arranged to have a second set of external imaging parameters (i.e., different imaging height, imaging angle, and imaging distance), and then images the article to obtain a second article image. Based on this, when the article moves to the jth inspection station, a third article image and a fourth article image may also be captured.

[0031] Therefore, as the robot arm 21 is operated to move the imaging device 22 K times, the imaging device 22 has an imaging height, an imaging angle, and an imaging distance after each movement. In this process, the first processor 11P records one imaging height, one imaging angle, and one imaging distance as a set of external imaging parameters after each movement of the imaging device 22, and then adds up and obtains the K sets of external imaging parameters and stores them in the first memory 11M, where K is a positive integer of at least 1. Similarly, if the imaging device 22 continues to receive an imaging device adjustment operation after each movement and has an aperture value, a depth of field, a shutter speed, a light sensitivity, and a focal length, one aperture value, one depth of field, one shutter speed, one light sensitivity, and one focal length are recorded as a set of internal imaging parameters, and after the imaging device 22 is moved K times, K sets of the internal imaging parameters are summed up and stored in the first memory 11M.

[0032] After the quality inspector completes setting the internal and external imaging parameters for each inspection point, the first processor 11P records and stores in the memory 11M a plurality of sets of external imaging parameters corresponding to the N robot arms 21, and records and stores in the memory 11M a plurality of sets of internal imaging parameters corresponding to the N cameras 22. It should be understood that during the jth inspection station, the jth camera 22 captures at least one image of the item. Thus, after passing through the N inspection stations, the electronic device 11 acquires a plurality of image pieces, including a plurality of first image pieces having normal features and a plurality of second image pieces having defect features. The quality inspector can label the first and second image pieces through the operation interface. Then, the electronic device 11 can be used to upload the first and second item images to the remote electronic device 3, which then uses the first and second item images to generate at least one update module. Finally, the update module can operate the electronic device 11 to perform a model update operation on the item defect identification model installed in the main control electronic device 23, or use the update module to install another set of item defect identification models in the main control electronic device 23.

[0033] 4A and 4B are first and second perspective views of the jth robot arm 21, the jth imager 22, and the jth light source 24. As shown in FIGS. 4A and 4B, the imager 22 is included in a portable electronic device such as a smartphone or a tablet computer, and the portable electronic device further includes an inertial measurement unit (IMU) and a laser imaging detection and ranging (LiDAR) unit. For example, the iPad Pro (Apple Inc.'s trademark) and the iPhone 12 (Apple Inc.'s trademark) are both built-in with a LiDAR unit and an IMU unit. According to this design, after each movement of the imager 22, the IMU unit and the LiDAR unit acquire three-dimensional movement data and three-dimensional measurement data, and then transmit the three-dimensional movement data and the three-dimensional measurement data to the electronic device 11 via the portable electronic device, so that the electronic device 11 aligns the three-dimensional movement data and the three-dimensional measurement data in the external imaging parameters.

[0034] However, not all portable electronic devices necessarily include a LiDAR unit. Therefore, in another possible embodiment, the imager 22 is included in a portable electronic device, and the portable electronic device includes an IMU unit and is installed on a fixture connected to the robot arm 21, and a LiDAR unit is installed on the fixture and electrically connected to the portable electronic device. According to this installation, after the fixture and the imager 22 are moved by the robot arm 21, the IMU unit and the LiDAR unit obtain three-dimensional movement data and three-dimensional measurement data, respectively, and the portable electronic device transmits the three-dimensional movement data and the three-dimensional measurement data to the electronic device 11, so that the electronic device 11 matches the three-dimensional movement data and the three-dimensional measurement data with the external imaging parameters.

[0035] In summary, the flexible and intuitive configuration system 1 of the present invention may be an application program installed on a tablet computer (i.e., the aforementioned electronic device 11). A quality inspector at a manufacturing plant can use the tablet computer to perform the above-described configuration procedure for any one AVI system 2 in the plant. After the AVI system 2 is configured and configured, it can be used to perform automated defect inspections on a specified product (e.g., a gear model B). In particular, the entire process of receiving the configuration procedure for the AVI system 2 does not require the intervention of a technician. The quality inspector can manually tune the configuration parameters of the N cameras 22 and N robot arms 21 based on their own experience (i.e., manually rotating the robot arms 21 and manually adjusting the imaging parameters). In contrast, a technician can simply operate the remote electronic device 3 to generate a product defect identification model based on multiple product images provided by the quality inspector.

[0036] To further explain, the electronic device 11 may be a smartphone, a desktop computer, an all-in-one computer, or a notebook computer. Furthermore, in one possible embodiment, the image capture device 22 is included in a portable electronic device, and the portable electronic device is, for example, a smartphone or a tablet computer. However, in another possible embodiment, the image capture device 22 has an Internet connection function. In other words, the image capture device 22 has a first communication interface and is used to transmit data via a second communication interface of the electronic device 11 or a main communication interface of the main control electronic device 23.

[0037] In addition to implementing a model update operation for the article defect identification model installed in the main control electronic device 23 through a series of settings, the flexible and intuitive configuration system 1 of the present invention also implements a software parameter update operation for the equipment control software installed in the main control electronic device 23 through a series of settings. As shown in FIGS. 1, 2A, and 2B, the conveying equipment 20 includes a conveyor belt mechanism 200, a motor 201 for driving the conveyor belt mechanism 200, and a distance sensor 202. Therefore, the first processor 11P may enable the software parameter update operation after executing the first application program. Specifically, in order for the conveyor belt mechanism 200 to transport N articles to the first through Nth inspection stations, a quality inspector operates the main control electronic device 23 to set the rotation rate of the motor 201, and then drives the conveyor belt mechanism 200 according to the rotation rate. Therefore, when the motor 201 is set to operate at the rotation rate, the first processor 11P records the rotation rate as a motor control parameter and stores it in the first memory 11M when driving the conveyor belt mechanism 200. At the same time, when the article is transported by the conveyor belt mechanism 200 and the article is within the imaging range of the jth imager 22, the first processor 11P records the jth movement distance as an article positioning parameter and stores it in the first memory 11M.

[0038] 4A and 4B, in actual operation, a quality inspector may also be configured to adjust the light source 24 for each inspection station, and after adjusting the light source 24 of each inspection station, the light source 24 has a specified light irradiation range, light intensity, color temperature, and light color. Therefore, when the j-th light source 24 has a light irradiation range, light intensity, color temperature, and light color after the light source adjustment operation, the first processor 11P records the light irradiation range, light intensity, color temperature, and light color as a set of lighting parameters and stores them in the first memory 11M.

[0039] After obtaining the motor control parameters, the N item positioning parameters, and the N sets of lighting parameters, the electronic device 11 can be operated to perform a parameter update operation on the equipment control software installed in the main control electronic device 23 based on the motor control parameters, the N item positioning parameters, and the N sets of lighting parameters. Thus, after undergoing the model update operation and parameter update operation, the AVI system 2 can be used to perform automated defect inspection on a specified item. Specifically, as shown in FIGS. 1, 2A and 2B, the main control electronic device 23 includes a second processor 23P and a second memory 23M. The second memory 23M stores a second application program (including device control software and product defect identification software). The second processor 23P accesses the second memory 23M to execute the second application program, thereby performing automated defect inspection. The following steps are performed: N robot arms 21 are controlled to operate according to the corresponding external imaging parameters, and N cameras 22 are controlled to operate according to the corresponding external imaging parameters. the N light sources 24 are controlled to emit N inspection lights based on the corresponding illumination parameters toward the N articles within the imaging ranges of the N cameras 22, respectively; the N cameras 22 are controlled to photograph the articles within their imaging ranges based on the corresponding internal imaging parameters, thereby obtaining a plurality of article images; and the method includes the steps of: using the article defect identification model to perform a defect identification operation on each of the article images, thereby determining whether the article has at least one defect.

[0040] After completing the defect inspection and identification of the item, the main control electronic device 23 controls the material discharge device 25 (i.e., the robot arm with a jig) to remove the clamped item belonging to the NG (not good) items from the conveying equipment 20 and place it in the NG item collection box 5. For this purpose, the flexible and intuitive placement system 1 of the present invention can also execute the following procedure. Specifically, after moving one of the items N distances on the conveyor belt mechanism 200 and then moving it N+1 distances, the item is now within the clamping range of the material discharge device 25. Therefore, the first processor 11P records the N+1 distance as a material discharge parameter and stores it in the first memory 11M. Next, the material discharge device 25 is operated to discharge the material of one item into the NG item collection box 5. It should be understood that in the process of discharging the material of an article, it necessarily has an initial height, a placing angle, and a final height, so the first processor 11P records the initial height, the placing angle, and the final height as a set of material discharge device parameters and stores them in the first memory 11M, so that the electronic device 11 can operate to update the parameters of the device control software installed in the main control electronic device 23 based on the material discharge parameters and the N material discharge device parameters.

[0041] On the other hand, when a lot of articles (e.g., gears) is to be placed on the conveying equipment 20 for defect inspection, the article placement device 26 (i.e., a robot arm with a jig) must first be operated to clamp the article located on the conveying equipment 20, and then the article is placed on the conveying equipment 20, thereby positioning the article on the conveying equipment 20 along the placement angle. When the article placement device 26 is operated to place one article on the conveying equipment 20, the article has an initial height, a placement angle, and a final height. It should be understood that the first processor 11P records the initial height, the placement angle, and the final height as article placement device parameters and stores them in the first memory 11M. Finally, the electronic device 11 can be operated to update the parameters of the equipment control software installed in the main control electronic device 23 based on the article placement device parameters.

[0042] After completing the update of the control parameters of the item loading device 26 and the material discharging device 25, the AVI system 2 can perform fully automatic operations for one lot of items, such as item loading, item defect inspection and identification, and material discharge of defective items.

[0043] Thus, the flexible and intuitive configuration system for an automated visual inspection system according to the present invention has been fully and clearly described above. However, it should be emphasized that the above detailed description specifically describes the workable embodiments of the present invention, but the scope of the present invention is not limited to these embodiments. As long as it does not deviate from the technical spirit of the present invention, equivalent implementations or modifications thereof are still included within the scope of the claims of the present application. [Explanation of symbols]

[0044] 1: Flexible and intuitive placement system 11:Electronic equipment 11P: First processor 11M: First memory 2:Automatic visual inspection system 20:Transportation equipment 200: Conveyor belt mechanism 201: Motor 202: Distance sensor 21: Robot arm 22: Camera 23: Main control electronics 23P: Second processor 23M: Second memory 24:Light source 25: Material discharge device 26: Item placement device 3: Remote electronic device 4: Normal product collection box 5: NG product collection box

Claims

1. A flexible and intuitive configuration system adapted for use in configuring an automated visual inspection system to perform automated defect inspections of articles, the system comprising: The automated visual inspection system comprises a conveying equipment, N robot arms, N camera devices respectively held by the N robot arms, and a main control electronic device in which a first article defect identification model is installed, where N is a positive integer of at least 1. The flexible and intuitive placement system comprises an electronic device electrically connected to the main control electronic device and information-linked with the N camera devices and the N robot arms through the main control electronic device, the electronic device including a first processor and a first memory connected to the first processor, wherein a first application program is stored in the first memory, and the first processor executes the first application program through access to the first memory to enable the following functions: In a process in which the robot arm is operated to move the image capture device K times, and the image capture device has an image capture height, an image capture angle, and an image capture distance after each movement, the image capture device records one image capture height, one image capture angle, and one image capture distance as a set of external image capture parameters after each movement, and the K sets of external image capture parameters are obtained by summing them up, where K is a positive integer of at least 1; When the imaging device is moved by the robot arm and receives an imaging device adjustment operation after each movement, and has an aperture value (Aperture), a depth of field (Depth of field), a shutter speed (Shutter speed), a light sensitivity (ISO), and a focal length (Focus), one aperture value, one depth of field, one shutter speed, one light sensitivity, and one focal length are recorded as one set of internal imaging parameters, and after the imaging device is moved K times, K sets of the internal imaging parameters are acquired by adding them up; the imaging device photographs an article (article) transported by the transport facility to obtain a plurality of first article images having regular features and a plurality of second article images having defect features, and then uploads the plurality of first article images and the plurality of second article images to a remote electronic device, the first article images containing the regular features of the article, and the second article images containing the defect features of the article; A flexible and intuitive configuration system, characterized in that the remote electronic device generates at least one update module using the plurality of first item images and the plurality of second item images, and then uses the update module to perform a model update operation on the first item defect identification model installed in the main control electronic device, or uses the update module to install a second item defect identification model in the main control electronic device.

2. The flexible and intuitive configuration system of claim 1, wherein the electronic device is any one selected from the group consisting of a smartphone, a tablet computer, a desktop computer, an all-in-one computer, and a notebook computer.

3. 2. The flexible and intuitive positioning system of claim 1, wherein the imaging device is included in a portable electronic device, and the portable electronic device is any one selected from the group consisting of a smartphone and a tablet computer.

4. 4. The flexible and intuitive positioning system of claim 3, wherein the portable electronic device includes an inertial measurement unit (IMU) and a laser imaging detection and ranging (LiDAR) unit, and the imager is used to obtain three-dimensional movement data and three-dimensional measurement data after each movement and transmit the three-dimensional movement data and the three-dimensional measurement data to the electronic device via the portable electronic device, so that the electronic device aligns the three-dimensional movement data and the three-dimensional measurement data in the external imaging parameters.

5. 4. The flexible and intuitive positioning system of claim 3, wherein the portable electronic device includes an inertial measurement unit (IMU) and is mounted on a fixture connected to the robot arm, and a laser imaging detection and ranging (LiDAR) unit is mounted on the fixture and electrically connected to the portable electronic device.

6. The flexible and intuitive positioning system of claim 5, characterized in that after each movement of the imaging device, the inertial measurement unit and the laser image detection and ranging unit obtain three-dimensional movement data and three-dimensional measurement data, respectively, and the portable electronic device transmits the three-dimensional movement data and the three-dimensional measurement data to the electronic device, so that the electronic device matches the three-dimensional movement data and the three-dimensional measurement data in the external imaging parameters.

7. 2. The flexible and intuitive positioning system of claim 1, wherein the first processor records and stores in the first memory a plurality of sets of the external imaging parameters corresponding to the N robot arms, and records and stores in the first memory a plurality of sets of the internal imaging parameters corresponding to the N imagers.

8. 2. The flexible and intuitive placement system of claim 1, wherein, after the first processor executes the first application program, the first processor causes the display of the electronic device to display an operation interface, and a user can perform label processing on the plurality of first item images and the plurality of second item images by operating the operation interface.

9. The conveying equipment includes a conveying belt mechanism, a motor used to drive the conveying belt mechanism, and a distance sensor, and the first processor, according to executing the first application program, further enables the following functions: When the motor is set to operate at a rotation rate, the rotation rate is recorded as a motor control parameter when driving the conveyor belt mechanism; As the article is transported by the conveyor belt mechanism, if the article is within the imaging range of the j-th imager, the j-th moving distance is recorded as an article positioning parameter, satisfying j∈N; 2. The flexible and intuitive positioning system of claim 1, wherein a parameter update operation is performed on device control software installed in the main control electronic device based on the motor control parameters and the N article positioning parameters.

10. The automated visual inspection system further includes N light sources, and the first processor, according to executing the first application program, enables the following functions: If the light source has a light irradiation range, a light intensity, a color temperature, and a light color through a light source adjustment operation, record the light irradiation range, the light intensity, the color temperature, and the light color as a set of lighting parameters; The flexible and intuitive configuration system according to claim 9, characterized in that the device control software installed in the main control electronic device is updated based on the lighting parameters.

11. The main control electronic device includes a second processor and a second memory, and stores a second application program in the second memory, so that the second processor executes the second application program through access to the second memory, thereby enabling the following functions: the robot arm controls the imager to move according to the external imaging parameters corresponding thereto; The light source is controlled to emit inspection light provided based on the corresponding illumination parameter toward the article; controlling the image capture device to capture an image of the item based on the corresponding internal image capture parameters; The flexible and intuitive location system of claim 10, further comprising: determining whether the product has at least one defect by performing a defect identification operation on the product image using the product defect identification model.

12. The automated visual inspection system further includes at least one material discharge device installed near a material outlet of the conveying equipment, and the first processor, according to executing the first application program, further enables the following functions: moving one of the articles by the conveyor belt mechanism N travel distances, and then moving the article by an (N+1)th travel distance, and then recording the (N+1)th travel distance as a material discharge parameter; When the material discharge device is operated to discharge the material of one of the items into a collection box, the item has an initial height, a placement angle, and a final height, and the initial height, the placement angle, and the final height are recorded as a set of material discharge device parameters; The flexible and intuitive placement system according to claim 11, characterized in that an update operation of the parameters is performed on the device control software installed in the main control electronic device based on the material discharge parameters and the N material discharge device parameters.

13. The automated visual inspection system further includes at least one article placement device installed near a material supply port of the conveying equipment, and the first processor, according to executing the first application program, further enables the following functions: When the article placement device is operated to place one article on the conveying equipment, the article has an initial height, a placement angle, and a final height, and the initial height, the placement angle, and the final height are recorded as article placement device parameters; The flexible and intuitive positioning system according to claim 12, characterized in that, based on the article placement device parameters, the device control software installed in the main control electronic device is updated with the parameters.

Images