AOI intelligent image discrimination system based on multi-device video centralized control
Through the AOI intelligent graph judgment system with multi-device video control, the video signal data of the AOI machine is uniformly collected and processed, and combined with AI and manual judgment modules, the problems of low manual graph judgment efficiency and high cost of independent deployment of AI are solved, and the problem of low manual graph judgment efficiency and high cost of independent AI deployment is realized. Efficient and accurate defect detection and rapid configuration reduce labor consumption and cost.
Patent Information
- Application Number
- CN202411160181.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-29
AI Technical Summary
In the field of industrial automation quality inspection, especially in the PCBA field, manual mapping efficiency and low accuracy, and the existing AI mapping system needs to be deployed and debugged separately for each AOI machine, resulting in high cost and low efficiency. The configuration is cumbersome and the effect is not ideal.
Adopt AOI intelligent image determination system based on multi-device video centralized control, and collects video signal data from multiple AOI machines through the core server, uses AI image algorithm to uniformly determine defects, and sends the results to each AOI machine, combined with Manual judgment module and flowing manual judgment module build neural network algorithm models, providing pure AI, pure artificial and AI+ manual assistance modes, reducing manual consumption and independent deployment problems of AI models.
This greatly reduces the manual consumption of map judges, realizes the rapid configuration of standard map detection strategies, improves the accuracy and efficiency of AI map judges, reduces cost waste, and improves the accuracy of AI models through manual intervention.
Smart Images

Figure CN119048470A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial quality inspection, specifically, to the field of industrial automation quality inspection based on image detection algorithms, and more specifically, to an AOI intelligent image judgment system based on multi-device video centralized control. Background Art
[0002] In the field of industrial automation quality inspection based on image detection algorithms, especially in the field of PCBA, defects are usually detected by AOI machines. On the detected images, defects are identified manually to select whether they are good products. Defect identification is usually given after comparing the captured image with the standard image. Due to the characteristics of the AOI machine, this method leads to an excessive number of manual image judgments, low efficiency and low accuracy. Whether using manual detection or traditional image processing methods, when detecting defects in such images, it is often difficult to identify defects, inaccurate judgments, and too many false alarms. Moreover, the AOI machines commonly used in the industry are all operated locally, and each machine needs to be equipped with a separate image judge to operate. This method has caused a huge waste of labor costs.
[0003] Another method is to use image algorithms for detection, and use algorithms and calculate defect results for each image separately. Finally, the defect results of the image are summarized, and the only defect result is obtained according to the formula or priority logic. This method is usually implemented using AI technology. However, the principle of AOI machines is to give results after comparing the captured image with the standard image. Before creating a new version each time, the standard image needs to be configured and the judgment parameters of each defect need to be adjusted to meet the needs of high pass rate and high detection rate. A high pass rate means that good products have fewer false alarms and save manual image judgment work. A high detection rate means that defects are less missed and the factory yield is improved. However, the work of configuring standard images is usually cumbersome and difficult to see results, so the factory will spend a lot of time and personnel on standard image configuration, and the effect is often not ideal. In actual production, the time for configuring a new version is usually no more than 30 minutes, otherwise it will affect production capacity. Under this condition, in order to quickly go online, manufacturers usually choose more stringent judgment parameters to reduce the pass rate and ensure the detection rate. This creates a huge workload for the manual image judgment process.
[0004] With the development of artificial intelligence technology at this stage, there are already a variety of algorithm structures based on image AI that can initially solve this problem. The principle is to use neural network image processing technology to collect defective product images on a large scale and train them, so that the AI model has the ability to identify defects. In this way, before the product goes online, manufacturers can still quickly configure a more stringent AOI judgment parameter strategy and put most of the suspected defective images on the AOI alarm end. Use the trained AI model in the AOI back-end to perform more accurate secondary filtering on the defects reported by the machine. However, consistent with the dilemma of manual image judgment, the current AI image judgment is still performed independently on each AOI machine. Therefore, when deploying, each machine needs to be deployed and debugged separately. If the algorithm is deployed locally on the AOI machine, it will cause the machine to freeze, crash, and may lose the original factory warranty; if it is deployed in an independent computer, it will further cause cost waste. Summary of the invention
[0005] The present invention aims to overcome the above-mentioned defects and proposes an AOI intelligent image judgment system based on multi-device video centralized control. The invention uses the mode of video centralized control to centralize the video signal data of multiple AOI machines into a core server, and performs unified defect judgment through AI image algorithm, and sends the results to each AOI machine operation end to control the flow of the next process of the product. If a small number of AI cannot handle the parts that require manual intervention, the system will centrally send the pictures to the central control room or the on-site display end. After a small number of pictures are manually judged, the results are automatically sent to the AOI control end again to control the flow of the next process of the product.
[0006] Specifically, the present invention provides an AOI intelligent image judgment system based on multi-device video centralized control, which is characterized by: comprising a centralized control terminal and an AOI terminal;
[0007] The above-mentioned centralized control terminal collects the video signal data of each AOI terminal, makes a unified defect judgment, and sends the judgment result to each corresponding AOI terminal to control the flow of the product to the next process.
[0008] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0009] The above-mentioned centralized control terminal includes an AI determination module;
[0010] The above-mentioned AI determination module performs defect determination based on a preset AI image algorithm.
[0011] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0012] The above-mentioned centralized control terminal also includes a manual determination module;
[0013] The above-mentioned manual judgment module receives input results from the front end of the device and incorporates the results into the training set of the AI image algorithm to build a neural network algorithm model or improve the accuracy of the AI image algorithm.
[0014] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0015] The above-mentioned centralized control terminal also includes a flow manual determination module;
[0016] The data that cannot be judged by the AI judgment module is transmitted to the manual judgment module;
[0017] The above-mentioned manual judgment module transmits the data that cannot be judged to a third-party judgment terminal (such as an on-site monitor) or displays it on the front end of the device, and sends the feedback results to each corresponding AOI terminal and incorporates them into the training set of the AI image algorithm.
[0018] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0019] It also includes a mode selection module;
[0020] The above-mentioned mode selection module includes a plurality of operation modes that can be selected or switched;
[0021] The above-mentioned operation modes include at least two of a pure AI mode, a pure manual mode, and an AI+manual auxiliary mode.
[0022] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0023] In the above AI mode, only the AI judgment module is called to perform defect judgment on the video signal data of each AOI end.
[0024] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0025] In the above-mentioned purely manual mode, only the manual judgment module is called to perform defect judgment on the video signal data of each AOI end.
[0026] Furthermore, the present invention provides an AOI intelligent image recognition system based on multi-device video centralized control, which is also characterized by:
[0027] In the above-mentioned AI+manual auxiliary mode, the AI judgment module is first called to perform defect judgment on the video signal data of each AOI end, and then the video signal data of each AOI end containing the defect judgment result is transmitted to the manual judgment module for secondary judgment. When the results are inconsistent, the results are included in the training set of the AI image algorithm.
[0028] Functions and effects of the present invention:
[0029] The system of the present invention greatly reduces the labor consumption of image judges. At the same time, due to the use of AI image judgment technology, the standard image detection strategy configuration link can still be quickly configured.
[0030] The on-site monitor proposed by the present invention can be completed on a handheld tablet computer, which is convenient for operators to move and operate between multiple machines. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 , the operation flow chart of the first mode;
[0032] Figure 2 , the operation flow chart of the second mode. DETAILED DESCRIPTION
[0033] The present invention is capable of various modifications and various embodiments, and therefore specific embodiments are illustrated and described in the accompanying drawings. However, this is not intended to limit the present invention to specific implementations, but should be understood to include all modifications, equivalents, and even substitutes that fall within the concept and technical scope of the present invention.
[0034] The present embodiment provides an AOI intelligent image recognition system based on multi-device video centralized control, comprising a centralized control terminal and an AOI terminal;
[0035] The centralized control end collects the video signal data of each AOI end, makes a unified defect judgment, and sends the judgment result to each corresponding AOI end to control the flow of the product to the next process.
[0036] Specifically: the centralized control end includes an AI judgment module, which performs defect judgment based on a preset AI image algorithm (the existing AI algorithm can be used; the data after manual judgment during system operation can also be used as training data, and a model can be established based on a neural network algorithm);
[0037] The centralized control terminal also includes a manual judgment module, which is suitable for situations where data is lacking in the initial stage of system operation. When the module is activated, the results of manual judgment are realized by displaying them on the front end of the device. It can receive input results from the front end of the device and incorporate the results into the training set of the AI image algorithm to build a neural network algorithm model or improve the accuracy of the AI image algorithm.
[0038] The centralized control terminal also includes a circulation manual judgment module. The data that cannot be judged by the AI judgment module is transmitted to the circulation manual judgment module, which transmits the data that cannot be judged to a third-party judgment terminal (such as other mobile devices) or displays it on the front end of the device (the current system device or the mobile terminal that is currently loaded with the system and authorized), and sends the feedback results to each corresponding AOI terminal and the training set included in the AI image algorithm.
[0039] The centralized control terminal also includes a mode selection module, which includes a variety of operating modes that can be selected or switched, such as: pure AI mode, pure manual mode, and AI + manual auxiliary mode.
[0040] In pure AI mode, only the AI judgment module is called to perform defect judgment on the video signal data of each AOI end.
[0041] In pure manual mode, only the manual judgment module is called to perform defect judgment on the video signal data of each AOI end.
[0042] In the AI+manual auxiliary mode, the AI judgment module is first called to perform defect judgment on the video signal data of each AOI end, and then the video signal data of each AOI end containing the defect judgment result is transmitted to the manual judgment module for secondary judgment. When the results are inconsistent, the results are included in the training set of the AI image algorithm.
[0043] The specific operation process is as follows:
[0044] The core re-judgment interface can load the real-time operation interface of the AOI machine. The background is centrally controlled in real time through video capture, and the data of each AOI machine is aggregated to the main server and selectively displayed on the front-end page. When the operator uses the system proposed by the present invention, its interface is completely consistent with the original AOI machine software interface, and the operation will not change at all.
[0045] AI identifies defect images and standard images displayed on the software interface of the AOI machine in the centralized control data. It automatically determines the type of defect and whether it can be considered a good product through the algorithm model.
[0046] In the initial launch stage of the AI model, if its accuracy rate does not meet the standard, this method (the interface displays the current bad judgment results in AI in real time as an auxiliary judgment) can be used to only display the AI results on the interface as an aid to manual judgment. The final decision still needs to rely on the results of manual judgment. The auxiliary mode can effectively provide manual judgment references and reduce manual inattention due to fatigue. Judgment errors caused by inconsistent judgment standards and inconsistencies between manual judgment results and algorithm judgment results will be automatically recorded by the system and included in the training set to further improve the judgment accuracy of the AI algorithm model.
[0047] The current screenshot is saved according to the corresponding re-judgment result classification. The judgment results can be saved according to different strategy classifications such as original type, OKNG situation, doubtful situation, etc., which is convenient for secondary re-judgment, manual screening, removal of redundant information, removal of erroneous information and other data cleaning operations before incorporating them into the training set.
[0048] The login interface verifies the user's identity and uses a verification code to ensure the accuracy of the verification. Different jump pages are selected according to the identity of the logon user. When a user forgets his password, he can choose to submit a request to the administrator and get his password back.
[0049] In this embodiment, the main system is preferably set to two modes:
[0050] Mode 1. Fully manual online review mode: When the AI algorithm is not online, the full manual mode is used. The workload of manual review remains unchanged, but the system will automatically record the results of manual review and include the results in the training set to build a neural network algorithm model.
[0051] like Figure 1 As shown, in the full manual online review mode, the operator selects the centralized control screen to conduct remote operation review, as follows:
[0052] S1. When clicking on the next picture, press the right arrow key on the control device (the key can be defined), wait for 500 milliseconds (there is a certain delay when the customer reviews the software to switch pictures), take a screenshot of the bad picture area, and then call the AI interface to pass the picture to the AI. The AI interface will return a result and display the result in the AI result column.
[0053] S2. When you click OK, first call the current image to take a screenshot of the control device, save the screenshot to the OK folder under the FTP path, and then trigger the right key of the keyboard direction of the control device after the screenshot is saved (the key can be customized)
[0054] S3. When you click NG, first call the current image to take a screenshot of the control device, save the screenshot to the NG folder under the FTP path, and then trigger the keyboard delete keyboard + right arrow key of the control device after the screenshot is saved (the key can be customized).
[0055] Mode 2. AI operation + manual online re-judgment: After the AI algorithm is launched, most images are automatically judged by the algorithm and the operation information is automatically issued through the centralized control hardware. A small number of images still require manual intervention due to the imperfect AI capabilities. These images are handed over to manual judgment through the software interface. Similarly, these manual results will be automatically recorded by the system and included in the training set to further improve the accuracy of the AI algorithm.
[0056] like Figure 2 As shown, in the AI proxy operation + manual online review mode:
[0057] If the confidence level is greater than a certain value, AI will automatically review and operate it. If the confidence level is low, an alarm will be issued, and the personnel of the control center will manually review it on the review interface and classify and save the reviewed images:
[0058] S1. When clicking on the next picture, press the right arrow key on the control device (the key can be defined), wait for 500 milliseconds (there is a certain delay when the customer reviews the software to switch pictures), take a screenshot of the bad picture area, and then call the AI interface to pass the picture to the AI. The AI interface will return a result and display the result in the AI result column.
[0059] S2. When you click OK, first call the current image to take a screenshot of the control device, save the screenshot to the OK folder under the FTP path, and then trigger the right key of the keyboard direction of the control device (the key can be customized) after the screenshot is saved, and then call the RPA trigger to trigger the corresponding RPA proxy operation process, and continue the subsequent proxy operation review
[0060] S3. When you click NG, first call the current image to take a screenshot of the control device, save the screenshot to the NG folder under the FTP path, and then trigger the keyboard delete key + right arrow key of the control device (the key can be customized) after the screenshot is saved, and then call the RPA trigger to trigger the corresponding RPA proxy operation process, and continue the subsequent proxy operation review
[0061] S4. After clicking OK and NG, if there is a trigger for the next image to be reviewed, the review interface for the next image will be automatically loaded; if not, the interface will jump back to the main interface.
[0062] Although the above description is centered on the embodiment, it is only an example and does not limit the present invention. It is clear to those skilled in the art that various modifications and applications not illustrated above can be made within the scope of the essential characteristics of the present embodiment. For example, each component specifically shown in the embodiment can be implemented after being modified. Moreover, various differences related to such modifications and applications should be interpreted as being included in the scope of the present invention as defined in the attached claims.
Claims
1. An AOI intelligent image recognition system based on multi-device video centralized control, Features: Contains centralized control end and AOI end; The centralized control terminal collects the video signal data of each AOI terminal, performs defect determination in a unified manner, and sends the determination results to each corresponding AOI terminal to control the flow of the product to the next process.
2. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 1, Features: The centralized control terminal includes an AI determination module; The AI determination module performs defect determination according to a preset AI image algorithm.
3. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 1, Features: The centralized control terminal further includes a manual determination module; The manual determination module receives input results from the device front end and incorporates the results into the training set of the AI image algorithm to build a neural network algorithm model or improve the accuracy of the AI image algorithm.
4. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 2, Features: The centralized control terminal further includes a flow manual determination module; The data that cannot be judged by the AI judgment module is transmitted to the manual judgment module; The manual judgment module transmits the data that cannot be judged to a third-party judgment terminal or displays it on the front end of the device, and sends the feedback results to each corresponding AOI terminal and incorporates them into the training set of the AI image algorithm.
5. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 3, Features: It also includes a mode selection module; The mode selection module includes a plurality of operation modes that can be selected or switched; The operation modes include at least two of a pure AI mode, a pure manual mode, and an AI+manual auxiliary mode.
6. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 5, Features: In the AI mode, only the AI determination module is called to perform defect determination on the video signal data of each AOI end.
7. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 5, Features: The purely manual mode only calls the manual judgment module to perform defect judgment on the video signal data of each AOI end.
8. The AOI intelligent image recognition system based on multi-device video centralized control as claimed in claim 5, Features: The AI+manual auxiliary mode first calls the AI judgment module to perform defect judgment on the video signal data of each AOI end, and then transmits the video signal data of each AOI end containing the defect judgment result to the manual judgment module for secondary judgment. When the results are inconsistent, the results are included in the training set of the AI image algorithm.
Citation Information
Patent Citations
Centralized management and effective data display system of AOI detection equipment
CN107832454A
Automatic equality method for different panel defects based on AOI system and system thereof
CN108827970A
Manual redetermination device, method and system based on AOI detection
CN112827857A
Defect automatic detection method and system based on AI and storage medium
CN115311222A
AOI online detection system and method and electronic equipment
CN116468661A
