Defect detection method, device and equipment of automatic optical detection system and medium

Abstract

The invention relates to the technical field of optical detection, and discloses a defect detection method, device and equipment of an automatic optical detection system and a medium, the automatic optical detection system comprises an edge server, a mobile terminal and a production execution end, and the method is applied to the edge server. Comprising the following steps: acquiring a to-be-detected product selected by a mobile terminal and a detection template corresponding to the to-be-detected product; generating detection control information according to the detection template, so that the production execution end performs picture acquisition on the to-be-detected product according to the detection control information after importing the to-be-detected product; and obtaining a first product picture collected by the production execution end, and performing defect detection based on the first product picture and the detection template. Intensive management of the production execution ends can be achieved, the computing power resource utilization rate and the remote control operation and maintenance capacity of equipment are improved, the cost of the production line execution side is reduced, data intercommunication and sharing of different production execution ends are achieved, and the current increasingly complex detection requirements are met.

Description

Technical Field

[0001] This invention relates to the field of optical inspection technology, and more specifically to defect detection methods, devices, equipment, and media for automated optical inspection systems. Background Technology

[0002] In smart factories, an increasing number of Automated Optical Inspection (AOI) devices are replacing manual visual inspections and being deployed on the production execution side of factory production lines. Existing production line AOI equipment typically connects to upstream and downstream workstations in an all-in-one configuration, providing core functions such as product template management, product arrival inspection, multi-point image acquisition, defect detection, and product fabrication. Among these, defect detection relies on AI model inference. Driven by the increasing complexity of inspection items and the need for faster production line cycles, this places high demands on the data processing performance of GPUs (Graphics Processing Units) and CPUs (Central Processing Units). Furthermore, to achieve traceability throughout the entire product lifecycle, there is a significant need for persistent storage of images during the inspection process, leading to increasingly higher physical hard drive resource consumption. As the coverage of AOI across various production lines in the factory increases, the deployment and maintenance costs of this equipment are escalating dramatically.

[0003] However, current AOI systems are stand-alone, with computing power integrated locally and data stored locally. Furthermore, the programming of product templates, automatic detection and equipment adjustment, and remote system upgrades can only be completed locally. Product and template parameters are isolated on a single machine, resulting in data silos that cannot be shared or communicated. This leads to low data utilization during the detection process, high equipment costs, and an inability to meet current detection needs. Summary of the Invention

[0004] In view of this, the present invention provides a defect detection method, apparatus, equipment and medium for an automatic optical inspection system to solve the problems of low system data utilization and high equipment cost.

[0005] In a first aspect, the present invention provides a defect detection method for an automated optical inspection system. The automated optical inspection system includes an edge server, a mobile terminal, and a production execution terminal. The defect detection method is applied to the edge server and includes: Obtain the product to be tested selected by the mobile terminal and the corresponding test template; Based on the testing template, testing control information is generated so that after the production execution end imports the product to be tested, images of the product to be tested are collected according to the testing control information. The system acquires the first product image collected by the production execution end and performs defect detection based on the first product image and the detection template.

[0006] The defect detection method of the automated optical inspection system provided by this invention obtains the product to be inspected and the corresponding inspection template selected by the mobile terminal through an edge server. After the product to be inspected is imported into the production execution terminal, inspection control information is generated to enable the production execution terminal to collect images of the product to be inspected and perform defect detection based on the collected first product image and inspection template. This invention migrates the algorithm detection module originally deployed at the production execution terminal to the edge server, enabling centralized management of the production execution terminal, improving the utilization rate of computing resources and the remote control and maintenance capabilities of equipment, reducing the manufacturing costs brought by high-performance workstations distributed on the production line execution side, and the equipment maintenance costs of multiple devices and multiple production lines. Different production execution terminals can achieve data interoperability and sharing, meeting the increasingly complex inspection needs.

[0007] In one optional implementation, the detection control information includes image acquisition point information and configuration information. Before acquiring the product to be inspected selected by the mobile terminal and the detection template corresponding to the product to be inspected, the method further includes: determining the detection template of the product to be inspected. The process of determining the detection template includes: acquiring the configuration information set by the mobile terminal, performing point planning based on the configuration information to obtain initial image acquisition point information; after the product to be inspected is imported, acquiring a second product image collected by the production execution terminal based on the initial image acquisition point information and configuration information, and feeding the second product image back to the mobile terminal; acquiring the detection information set by the mobile terminal based on the second product image and the adjusted configuration information, and performing point planning based on the detection information and the adjusted configuration information to obtain target image acquisition point information; acquiring a third product image collected by the production execution terminal based on the target image acquisition point information and the adjusted configuration information, performing defect detection based on the third product image, and determining the detection template based on the defect detection results.

[0008] This invention determines the detection template through an edge server, enabling different production execution ends on the production line to share the detection template, thereby achieving centralized management of computing and storage resources within the factory.

[0009] In one optional implementation, determining the detection template based on the defect detection results includes: determining the defective structure of the product based on the defect detection results, and feeding back the defective structure and a third product image to the mobile terminal for detection result judgment; if the detection result fails, obtaining the adjusted detection information and / or adjusted configuration information from the mobile terminal, and returning to the step of planning the location based on the detection information and the adjusted configuration information to obtain the target image acquisition point information; if the detection result passes, determining the detection template based on the third product image, the adjusted configuration information, and the detection information.

[0010] This invention, by iteratively adjusting the determination process of the detection template, can plan the most reasonable point information, detection information, and configuration information for the product to be tested, thereby achieving accurate defect detection of the product to be tested based on the detection template in subsequent defect detection.

[0011] Secondly, the present invention provides a defect detection method for an automated optical inspection system. The automated optical inspection system includes an edge server, a mobile terminal, and a production execution terminal. The production execution terminal includes an embedded device and an execution device. The defect detection method is applied to the embedded device and includes: Obtain the detection and control information from the edge server, and generate control commands based on the detection and control information; The control execution device acquires images of the imported product to be tested according to the control instructions, and obtains the first product image acquired. The first product image is fed back to the edge server so that the edge server can perform defect detection based on the first product image.

[0012] The defect detection method of the automated optical inspection system provided by this invention acquires detection control information from an edge server via an embedded device, and generates corresponding control commands for the execution device. The execution device then captures images of the imported product to be inspected and sends the acquired first product image back to the edge server, enabling the edge server to perform defect detection based on the first product image. This invention enables remote interaction between the edge server and the execution device through communication between the embedded device, the edge server, and the execution device. This allows for centralized management of the production execution end, reducing manufacturing costs associated with high-performance workstations distributed across the production line execution side, as well as equipment maintenance costs across multiple devices and production lines.

[0013] In one optional implementation, the detection control information includes image acquisition point information and configuration information. The configuration information includes camera parameters and light source parameters. The control commands include point movement commands and light source switching commands. The execution devices include a mobile device, an image capture device, and a light source device. The control execution device performs image acquisition on the imported product to be detected according to the control commands, and obtains the acquired first product image. This includes: controlling the mobile device to move the image capture device according to the point movement commands, and controlling the light source device to switch the light source according to the light source switching commands; when the image capture device moves to a preset position, adjusting the parameters of the image capture device according to the camera parameters, and generating an image capture command to enable the image capture device to acquire the image; obtaining the first product image, and uploading the first product image to the end of a preset image queue.

[0014] In one optional implementation, feeding the first product image back to the edge server includes: receiving an image acquisition instruction from the edge server; acquiring the first product image from the head of a preset image queue based on the image acquisition instruction; and uploading the first product image to the edge server.

[0015] This invention controls the execution device to complete image acquisition through a mobile photography thread and completes image upload through an upload detection process. It can achieve a parallel business solution through two threads. The threads jointly maintain the queue of images to be uploaded and each maintains its own business logic, making full use of the time for point movement and photography, and solving the latency problem caused by the data transmission of high-definition images.

[0016] Thirdly, the present invention provides a defect detection device for an automatic optical inspection system. The automatic optical inspection system includes an edge server, a mobile terminal, and a production execution terminal. The device is applied to the edge server and includes: a template acquisition module for acquiring the product to be inspected selected by the mobile terminal and the inspection template corresponding to the product to be inspected; an image acquisition control module for generating inspection control information based on the inspection template, so that after the production execution terminal imports the product to be inspected, it can acquire images of the product to be inspected according to the inspection control information; and a defect detection module for acquiring a first product image acquired by the production execution terminal and performing defect detection based on the first product image and the inspection template.

[0017] Fourthly, the present invention provides a defect detection device for an automatic optical inspection system. The automatic optical inspection system includes an edge server, a mobile terminal, and a production execution end. The production execution end includes an embedded device and an execution device. The device is applied to the embedded device and includes: an instruction generation module for acquiring detection control information from the edge server and generating control instructions based on the detection control information; an image acquisition module for controlling the execution device to acquire images of the imported product to be inspected according to the control instructions, thereby acquiring a first product image; and an image feedback module for feeding back the first product image to the edge server so that the edge server can perform defect detection based on the first product image.

[0018] Fifthly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the defect detection method of the automatic optical inspection system of the first aspect, the second aspect, or any corresponding embodiment thereof.

[0019] In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the defect detection method of the automatic optical inspection system of the first aspect, the second aspect, or any corresponding embodiment thereof.

[0020] In a seventh aspect, the present invention provides a computer program product, including computer instructions for causing a computer to execute the defect detection method of the automatic optical inspection system of the first aspect, the second aspect, or any corresponding embodiment thereof. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the composition structure of a standalone AIO system in the existing technology; Figure 2 This is a schematic flowchart of a defect detection method for an automated optical inspection system according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the composition structure of a distributed AIO system according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the interactive process of a defect detection method in an automated optical inspection system according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating a defect detection method for another automated optical inspection system according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the product import process for a defect detection method of another automated optical inspection system according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating a defect detection method for another automatic optical inspection system according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the parallel process of mobile image acquisition and uploading detection in a defect detection method of another automatic optical inspection system according to an embodiment of the present invention. Figure 9 This is a structural block diagram of a defect detection device in an automatic optical inspection system according to an embodiment of the present invention; Figure 10 This is a structural block diagram of a defect detection device in an automatic optical inspection system according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] This invention is applicable to scenarios involving defect detection of products, taking the defect detection of PCBA boards (Printed Circuit Board + Assembly) as an example. Figure 1 The diagram shows the structural composition of a standalone AIO system in the prior art. In such systems, algorithm configuration management, product management, template management, inspection management, and AI Alogorithm (artificial intelligence algorithm) templates are integrated into the industrial control computer on the production line execution side. Therefore, all related data processing is completed on the production line execution side, resulting in isolation between different standalone machines and preventing data sharing. This invention provides a defect detection method for an automated optical inspection system. By migrating the detection algorithm to an edge server, it achieves centralized management of computing resources and reduces costs.

[0025] According to an embodiment of the present invention, a defect detection method embodiment of an automatic optical inspection system is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] This embodiment provides a defect detection method for an automated optical inspection system, which can be used in the aforementioned edge server. Figure 2 This is a flowchart of a defect detection method for an automated optical inspection system according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the product to be tested selected by the mobile terminal and the test template corresponding to the product to be tested.

[0027] Specifically, in embodiments of the present invention, such as Figure 3The schematic diagram of the distributed AIO system illustrates the structural composition of the system. It migrates the computationally intensive algorithm detection module, and the storage-intensive configuration management, product management, template management, and detection management module (including detection history records) from the device side to a high-performance edge server. This centralized management of resource scheduling and allocation allows for more efficient resource management. The lightweight production execution end retains only the execution devices (production lines, cameras, camera mobile platforms, light sources, etc.) and low-cost embedded devices responsible for network data interaction. Combined with flexibly accessible mobile terminals, this constructs an edge-device collaborative distributed AOI system, centralizing the main product defect detection process on the edge server. Mobile devices include smartphones, tablets, and computers, while the edge server provides a web interface to support any mobile device accessing the unified management of the factory's execution-end AOI devices via the network. This provides users with remote deployment, remote debugging, and remote maintenance capabilities.

[0028] In some alternative implementations, such as Figure 4 The diagram illustrates the interactive flow of the defect detection method in the automated optical inspection system. After pre-completing import and modeling, when defect detection is required on a product on the production line, the user selects the imported product on their mobile device, and the edge server retrieves the corresponding detection template based on its built-in template management module. The detection template is pre-determined through the product import process after the device to be inspected is selected, providing a basis for subsequent defect detection.

[0029] Step S202: Generate detection control information based on the detection template so that after the production execution terminal imports the product to be detected, it can collect images of the product to be detected based on the detection control information.

[0030] Specifically, in embodiments of the present invention, such as Figure 4 As shown, the edge server of the AOI system generates an automatic detection task based on the loaded detection template and notifies the embedded device. This automatic detection task includes detection control information, such as image acquisition point information and configuration information. The image acquisition point information consists of coordinate parameters, and the configuration information includes light source parameters and camera parameters. This is just an example and not a limitation. After receiving the automatic detection task, the embedded device interacts with the production line to request images from upstream. When the product to be inspected arrives, the embedded device performs multi-point moving image acquisition and image upload detection in parallel based on the coordinate parameters, light source parameters, and camera parameters, uploading the first product image acquired by the device to the edge server.

[0031] Step S203: Obtain the first product image collected by the production execution terminal, and perform defect detection based on the first product image and the detection template.

[0032] Specifically, in this embodiment of the invention, the edge server performs defect detection on the first product image collected by the production execution terminal based on a built-in detection algorithm and detection template. The execution process of the specific detection algorithm is a conventional technique in the field and will not be described in detail here. Figure 4 As shown, the edge server notifies the user of the product inspection results and simultaneously generates a product inspection record in the inspection task, which is managed by the inspection management module. After the edge server completes defect detection, the embedded device triggers the downstream tape-out logic based on the inspection results. After tape-out is completed, it continues to request wafers from upstream, thus achieving automated defect detection of products to be inspected on the production line through a cyclical process, replacing manual defect detection of products on the production line. Meanwhile, while waiting for products to flow in, the user can also stop the automatic inspection task; after stopping, the automatic inspection task is completed.

[0033] The defect detection method of the automated optical inspection system provided in this invention obtains the product to be inspected and the corresponding inspection template selected by the mobile terminal through an edge server. After the product to be inspected is imported into the production execution terminal, inspection control information is generated to enable the production execution terminal to collect images of the product to be inspected and perform defect detection based on the collected first product image and inspection template. This invention migrates the algorithm detection module originally deployed on the production execution terminal to the edge server, enabling centralized management of the production execution terminal, improving the utilization rate of computing resources and the remote control and maintenance capabilities of equipment, reducing the manufacturing costs brought by high-performance workstations distributed on the production line execution side, and the equipment maintenance costs of multiple devices and multiple production lines. Different production execution terminals can achieve data interoperability and sharing, meeting the increasingly complex inspection needs.

[0034] This embodiment provides a defect detection method for an automated optical inspection system, which can be used in the aforementioned edge server. Figure 5 This is a flowchart of a defect detection method for an automated optical inspection system according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps: Step S501: Obtain the configuration information set by the mobile terminal, perform point planning based on the configuration information, and obtain the initial image acquisition point information.

[0035] Specifically, in this embodiment of the invention, before performing automated defect detection on the product to be inspected, a corresponding inspection template needs to be determined in advance. For example... Figure 6The product import process diagram shows that the user first selects the production execution device to be controlled on the edge server via a mobile terminal, and simultaneously inputs the product ID, size information, modeling light source information, camera parameter information, and other configuration information of the product to be imported for testing. The edge server then plans the points based on the product size information and the camera frame size to obtain the initial image acquisition point information, specifically represented by the camera's image acquisition point list, and at the same time, the embedded device on the production execution terminal begins to import the product.

[0036] Step S502: After the product to be tested is imported, the second product image collected by the production execution terminal based on the initial image acquisition point information and configuration information is obtained, and the second product image is fed back to the mobile terminal.

[0037] Specifically, in embodiments of the present invention, such as Figure 6 As shown, after receiving a product import request, the embedded device notifies the production line to request upstream tape-out. Once the product is in place, according to the location, light source, and camera parameters information sent by the edge server, it controls the mobile device and camera to complete the location movement, light source control, camera parameter adjustment, and image capture and storage. Simultaneously, it uploads the acquired second product image to the edge server. After receiving the complete second product image, the edge server presents it to the user on a web page or mobile terminal. The user then draws the detection area, configures the light source and camera parameters, and configures the detection algorithm and algorithm parameters, etc., which are determined based on the image imaging effect and are not limited here.

[0038] Step S503: Obtain the detection information and adjusted configuration information set by the mobile terminal based on the second product image, and perform point planning based on the detection information and adjusted configuration information to obtain the target image acquisition point information.

[0039] Specifically, in this embodiment of the invention, the edge server re-plans the minimum number of points and the optimal trajectory that need to be photographed during the detection process based on the detection area, detection parameters and configuration parameters drawn by the user, thereby obtaining the target image point information and sending a template debugging request to the embedded device.

[0040] Step S504: Obtain the third product image collected by the production execution end based on the target image acquisition point information and the adjusted configuration information, perform defect detection based on the third product image, and determine the detection template based on the defect detection results.

[0041] Specifically, in this embodiment of the invention, after receiving a debugging request, the embedded device controls the mobile device and the camera and other execution devices to complete the point movement, light source control, camera parameter adjustment and image storage according to the detection point, light source and camera parameters issued by the edge server. At the same time, the image is uploaded to the edge server in parallel for defect detection.

[0042] In some alternative implementations, such as Figure 6 As shown, the edge server determines the defective structure of the product based on the defect detection results and feeds back the defective structure and a third-party product image to the mobile terminal for user evaluation. If the user is not satisfied with the results, indicating a failed test, the user can flexibly adjust the image acquisition points, detection area, and detection parameters. The edge server obtains the adjusted detection information and / or configuration information from the mobile terminal and re-plans the image acquisition points based on the adjusted information to obtain the revised target image acquisition point information until the user confirms that everything is correct and the test results are passed. At this point, the edge server determines the detection template based on the third-party product image, the adjusted configuration information, and the detection information, saves the detection template, and completes the product import process.

[0043] Step S505: Obtain the product to be tested selected by the mobile terminal and the corresponding test template. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0044] Step S506: Generate inspection control information based on the inspection template, so that after the production execution terminal imports the product to be inspected, it can collect images of the product to be inspected according to the inspection control information. For details, please refer to [link to relevant documentation]. Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0045] Step S507: Obtain the first product image collected by the production execution terminal, and perform defect detection based on the first product image and the inspection template. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0046] The defect detection method of the automatic optical inspection system provided in this invention obtains the product to be inspected and the corresponding inspection template selected by the mobile terminal through an edge server. After the product to be inspected is imported into the production execution terminal, inspection control information is generated to enable the production execution terminal to collect images of the product to be inspected and perform defect detection based on the collected first product image and inspection template. This invention migrates the algorithm detection module originally deployed on the production execution terminal to the edge server, enabling centralized management of the production execution terminal, improving the utilization rate of computing resources and the remote control and maintenance capabilities of equipment, reducing the manufacturing costs brought by high-performance workstations distributed on the production line execution side, and the equipment maintenance costs of multiple devices and multiple production lines. Different production execution terminals can achieve interoperability and sharing of template data, realize accurate defect detection of the product to be inspected, and meet the increasingly complex inspection needs.

[0047] This embodiment provides a defect detection method for an automated optical inspection system, which can be used in the aforementioned embedded devices. Figure 7This is a flowchart of a defect detection method for an automated optical inspection system according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps: Step S701: Obtain the detection control information of the edge server and generate control instructions based on the detection control information.

[0048] Specifically, in this embodiment of the invention, after migrating the algorithm detection module (which consumes computing power), configuration management, product management, template management, and detection management module (including detection history records) from the device side to the factory edge server, a low-cost embedded device is deployed at the production execution end to realize communication between the edge server and the execution device. The embedded device is responsible for real-time interaction with the execution device at the production execution end, including: real-time interaction with the production line system to complete functions such as conveyor belt control, upstream wafer ordering, downstream wafer fabrication, and production line anomaly alarms; real-time interaction with the camera motion module to complete movement at different points; real-time interaction with the light source to complete light source control at different points; and interaction with the camera to set exposure / gain parameters for different points to meet the photography requirements of each point. Simultaneously, the embedded device also interacts with the edge server in real-time to obtain detection control information from the edge server, including image acquisition point information and configuration information, thereby interacting with the execution device in real-time based on the detection control information.

[0049] Step S702: The control execution device acquires images of the imported product to be tested according to the control command, and obtains the first product image.

[0050] Specifically, in this embodiment of the invention, compared with the single-machine AOI system solution, the distributed AOI system can solve the utilization of computing power and storage resources and improve the remote control and maintenance capabilities of the equipment. However, since image acquisition and detection are performed at the production execution end and the edge server respectively, the latency problem caused by the data transmission of high-definition images needs to be focused on. Based on this, this embodiment of the invention provides a solution for parallel mobile photography and upload detection, that is, deploying two independent threads in the embedded device: a mobile photography thread and an upload detection thread, so as to utilize the time for point movement and photography to upload images for detection in parallel, so as to ensure the cycle efficiency of automatic detection.

[0051] In an optional implementation, step S702 includes: Step S7021: Generate a point movement command based on the image acquisition point information, and generate a light source switching command based on the light source parameters.

[0052] Step S7022: Control the mobile device to move the camera according to the point movement command, and control the light source device to switch the light source according to the light source switching command.

[0053] Step S7023: After the camera device moves to the preset position, the camera device parameters are adjusted according to the camera parameters, and a shooting command is generated so that the camera device can capture images.

[0054] Step S7024: Obtain the first product image and upload it to the end of the preset image queue.

[0055] Specifically, in embodiments of the present invention, such as Figure 8 The diagram illustrates the parallel workflow of mobile image acquisition and upload detection in the defect detection method shown. In the mobile image-taking thread of the embedded device, the device generates point movement commands (i.e., coordinate parameters) based on the image point information sent by the edge server, and generates light source switching commands based on the light source parameters, thereby controlling point movement and light source switching in parallel. After movement and light source switching are completed, camera parameters are set and the camera is triggered to capture an image. Once a single image is acquired, its local path is written to the end of the image queue to be uploaded. With repeated operations from upstream chip fabrication and downstream chip fabrication, the acquired product images are sequentially uploaded to the end of the image queue until all images are acquired, without being blocked by delays in image upload and image detection.

[0056] Step S703: The first product image is fed back to the edge server so that the edge server can perform defect detection based on the first product image.

[0057] Specifically, in this embodiment of the invention, step S703 includes: Step S7031: Receive the image acquisition instruction from the edge server.

[0058] Step S7032: Obtain the first product image from the head of the preset image queue based on the image acquisition instruction.

[0059] Step S7033: Upload the first product image to the edge server.

[0060] Specifically, in this embodiment of the invention, the queue is a data structure that follows the "First In First Out" (FIFO) principle. The head of the queue is where the first element to enter the queue is located, and it is also the end where deletion (dequeue) operations are performed. The tail of the queue is where the last element to enter the queue is located, and it is the end where insertion (enqueue) operations are performed. Therefore, during the process of the embedded device controlling the execution device to move the point and acquire images based on the mobile photography thread, the upload detection thread can share this image queue.

[0061] In some alternative implementations, such as Figure 8As shown, after the edge server begins defect detection, it requests images of corresponding locations from the embedded devices in an orderly manner according to the list of product images to be inspected. If there are product images to be uploaded in the image queue, they are dequeued from the head of the queue and uploaded. After the upload is complete, the image is inspected on the edge server. If not all images have been inspected, they are retrieved from the image queue in an orderly manner until all images have been inspected. In the above process, the upload and inspection of images that have been collected are performed in parallel with the collection of images that have not been collected, making the most of the time spent on location movement and taking pictures to complete the upload and inspection of images that have been collected, thus solving the latency caused by image upload and avoiding the impact on production cycle.

[0062] In some optional implementations, embodiments of the present invention provide an edge-to-edge collaborative distributed AOI system and its implementation scheme. This system centrally manages computing and storage resources within the factory, reducing manufacturing costs associated with high-performance workstations distributed across the production line execution side, as well as equipment maintenance costs across multiple devices and production lines. It supports the following functions: remote product import and modeling programming; remote automatic detection and equipment debugging; centralized management of computing and storage resources, saving equipment costs; shared product and template parameters between the factory / workshop, supporting flexible production line changes; establishing a process data upload link for closed-loop optimization of detection capabilities; and remote diagnostics and software upgrade capabilities, saving on operation and maintenance costs. Simultaneously, through a parallel scheme of mobile image acquisition and upload detection using embedded devices, the impact of network latency on production cycle time is minimized, significantly reducing equipment costs and subsequent production and maintenance costs. This provides a feasible solution for the unmanned and intelligent transformation of smart factories, supporting better integration of the software platform into the future ecosystem.

[0063] The defect detection method of the automated optical inspection system provided by this invention acquires detection control information from an edge server via an embedded device, and generates corresponding control commands for the execution device. The execution device then captures images of the imported product to be inspected and sends the acquired first product image back to the edge server, enabling the edge server to perform defect detection based on the first product image. This invention enables remote interaction between the edge server and the execution device through communication between the embedded device, the edge server, and the execution device. This allows for centralized management of the production execution end, reducing manufacturing costs associated with high-performance workstations distributed across the production line execution side, as well as equipment maintenance costs across multiple devices and production lines.

[0064] This embodiment also provides a defect detection device for an automated optical inspection system. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0065] This embodiment provides a defect detection device for an automated optical inspection system, applied to an edge server, such as... Figure 9 As shown, it includes: The template acquisition module 901 is used to acquire the product to be tested selected by the mobile terminal and the test template corresponding to the product to be tested.

[0066] The image acquisition control module 902 is used to generate inspection control information based on the inspection template, so that after the production execution end imports the product to be inspected, it can acquire images of the product to be inspected based on the inspection control information.

[0067] The defect detection module 903 is used to acquire the first product image collected by the production execution terminal and perform defect detection based on the first product image and the detection template.

[0068] In some optional embodiments, the apparatus further includes: a template import module, which is used to acquire configuration information set by the mobile terminal, perform point planning based on the configuration information, and obtain initial image acquisition point information; after the product to be inspected is imported, acquire a second product image collected by the production execution terminal based on the initial image acquisition point information and configuration information, and feed the second product image back to the mobile terminal; acquire the inspection information and adjusted configuration information set by the mobile terminal based on the second product image, and perform point planning based on the inspection information and adjusted configuration information to obtain target image acquisition point information; acquire a third product image collected by the production execution terminal based on the target image acquisition point information and adjusted configuration information, perform defect detection based on the third product image, and determine the inspection template based on the defect detection results.

[0069] In some alternative implementations, the template import module includes: The detection result judgment unit is used to determine the defect structure of the product based on the defect detection results, and to feed back the defect structure and a third product image to the mobile terminal for detection result judgment.

[0070] The parameter adjustment unit is used to obtain the adjusted detection information and / or adjusted configuration information of the mobile terminal if the detection result fails, and return to the step of planning the point location based on the detection information and the adjusted configuration information to obtain the target image acquisition point location information.

[0071] The template determination module is used to determine the test template based on the third product image, the adjusted configuration information, and the test information if the test result passes.

[0072] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0073] In this embodiment, the defect detection device of the automatic optical inspection system is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0074] This embodiment provides a defect detection device for an automated optical inspection system, applied to embedded devices, such as... Figure 10 As shown, it includes: The instruction generation module 1001 is used to obtain the detection and control information of the edge server and generate control instructions based on the detection and control information.

[0075] The image acquisition module 1002 is used to control the execution device to acquire images of the imported product to be tested according to the control instructions, and to obtain the first product image acquired.

[0076] The image feedback module 1003 is used to feed back the first product image to the edge server so that the edge server can perform defect detection based on the first product image.

[0077] In some optional implementations, the detection control information includes image acquisition point information and configuration information. The configuration information includes camera parameters and light source parameters. The corresponding control commands include point movement commands and light source switching commands. The execution devices include mobile devices, imaging devices, and light source devices. The image acquisition module 1002 includes: The control unit is used to control the mobile device to move the camera according to the point movement command, and to control the light source device to switch the light source according to the light source switching command.

[0078] The image acquisition unit is used to adjust the parameters of the camera device according to the camera parameters after the camera device moves to a preset position, and generate a shooting command so that the camera device can acquire images.

[0079] The image storage unit is used to acquire the first product image and upload it to the end of the preset image queue.

[0080] In some alternative implementations, the image feedback module 1003 includes: The instruction acquisition unit is used to receive image acquisition instructions from the edge server.

[0081] The image acquisition unit is used to acquire the first product image from the head of a preset image queue based on the image acquisition instruction.

[0082] The image upload unit is used to upload the first product image to the edge server.

[0083] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0084] In this embodiment, the defect detection device of the automatic optical inspection system is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0085] This invention also provides a computer device having the above-described features. Figure 9 or Figure 10 The defect detection device of the automated optical inspection system shown.

[0086] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 11 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 11 Take a processor 10 as an example.

[0087] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0088] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0089] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0090] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0091] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.

[0092] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0093] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0094] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0095] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A defect detection method for an automated optical inspection system, characterized in that, The automated optical inspection system includes an edge server, a mobile terminal, and a production execution terminal, wherein the defect detection method is applied to the edge server and includes: Obtain the product to be tested selected by the mobile terminal and the test template corresponding to the product to be tested; Based on the detection template, detection control information is generated so that after the production execution terminal imports the product to be detected, it can collect images of the product to be detected based on the detection control information. The system acquires a first product image collected by the production execution terminal and performs defect detection based on the first product image and the detection template.

2. The method according to claim 1, characterized in that, The detection control information includes image acquisition point information and configuration information. Before obtaining the product to be detected selected by the mobile terminal and the detection template corresponding to the product to be detected, it also includes: The configuration information set by the mobile terminal is obtained, and the point planning is performed according to the configuration information to obtain the initial image acquisition point information; After the product to be tested is imported, the second product image collected by the production execution terminal based on the initial image acquisition point information and the configuration information is obtained, and the second product image is fed back to the mobile terminal. The detection information and adjusted configuration information set by the mobile terminal based on the second product image are obtained, and the point planning is performed based on the detection information and adjusted configuration information to obtain the target image acquisition point information; The production execution terminal acquires a third product image based on the target image acquisition point information and the adjusted configuration information, performs defect detection based on the third product image, and determines the detection template based on the defect detection results.

3. The method according to claim 2, characterized in that, The step of determining the detection template based on the defect detection results includes: Based on the defect detection results, the defect structure of the product is determined, and the defect structure and the third product image are fed back to the mobile terminal for detection result judgment. If the detection result fails, the adjusted detection information and / or adjusted configuration information of the mobile terminal are obtained, and the process is returned to the step of planning the location based on the detection information and the adjusted configuration information to obtain the target image acquisition point information. If the test result is satisfactory, the test template is determined based on the third product image, the adjusted configuration information, and the test information.

4. A defect detection method for an automated optical inspection system, characterized in that, The automated optical inspection system includes an edge server, a mobile terminal, and a production execution terminal. The production execution terminal includes an embedded device and an execution device. The defect detection method is applied to the embedded device and includes: Obtain the detection and control information of the edge server, and generate control instructions based on the detection and control information; The execution device is controlled to acquire images of the imported product to be tested according to the control command, thereby obtaining the first product image acquired. The first product image is fed back to the edge server so that the edge server can perform defect detection based on the first product image.

5. The method according to claim 4, characterized in that, The detection and control information includes image acquisition point information and configuration information. The configuration information includes camera parameters and light source parameters. The control commands include point movement commands and light source switching commands. The execution devices include mobile devices, imaging devices, and light source devices. The control device performs image acquisition on the imported product to be tested according to the control command, and obtains the acquired first product image, including: The mobile device is controlled to move the camera according to the point movement command, and the light source device is controlled to switch the light source according to the light source switching command; After the camera moves to the preset position, the camera parameters are adjusted according to the camera parameters, and a shooting command is generated so that the camera can capture an image. Obtain the first product image and upload it to the end of a preset image queue.

6. The method according to claim 5, characterized in that, The step of sending the first product image to the edge server includes: Receive the image acquisition instruction from the edge server; The first product image is obtained from the head of the preset image queue based on the image acquisition instruction; The first product image is uploaded to the edge server.

7. A defect detection device for an automatic optical inspection system, characterized in that, The automated optical inspection system includes: an edge server, a mobile terminal, and a production execution terminal. The device is applied to the edge server and includes: The template acquisition module is used to acquire the product to be tested selected by the mobile terminal and the detection template corresponding to the product to be tested; The image acquisition control module is used to generate detection control information based on the detection template, so that after the production execution terminal imports the product to be inspected, it can acquire images of the product to be inspected based on the detection control information. The defect detection module is used to acquire the first product image collected by the production execution terminal, and to perform defect detection based on the first product image and the detection template.

8. A defect detection device for an automatic optical inspection system, characterized in that, The automated optical inspection system includes: an edge server, a mobile terminal, and a production execution terminal. The production execution terminal includes an embedded device and an execution device. The device is applied to the embedded device and includes: The instruction generation module is used to obtain the detection and control information of the edge server and generate control instructions based on the detection and control information; The image acquisition module is used to control the execution device to acquire images of the imported product to be tested according to the control instructions, and to obtain the acquired first product image; The image feedback module is used to feed back the first product image to the edge server so that the edge server can perform defect detection based on the first product image.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the defect detection method of the automatic optical inspection system according to any one of claims 1 to 3 or claims 4 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform a defect detection method of the automated optical inspection system according to any one of claims 1 to 3 or claims 4 to 6.

Images