An ai rejudication platform system for electronic manufacturing industry and quality management method

Abstract

The application belongs to the technical field of industrial automation quality detection, and provides an AI re-judgment platform system for electronic manufacturing and a quality management method.The system comprises: an image acquisition module that receives detection images associated with board card identifiers and device position numbers; an image inference module that distributes images to multiple inference servers for deep learning inference through a load balancing strategy, and generates qualified confidence and unqualified confidence; a dynamic shunting module that shunts images into three categories of automatic qualification, automatic disqualification and manual re-judgment based on a preset double threshold; a man-machine interaction re-judgment module that supports batch display, digital shortcut key negation and color border synchronous switching; and a quality data modeling and analysis module that establishes a multi-level data model from board cards to device position numbers, and performs defect trend analysis and early warning.The application can reduce the amount of manual re-judgment through image inference and dynamic shunting, and realizes the balance between detection accuracy and efficiency by combining man-machine interaction and position number level quality backtracking.

Description

Technical Field

[0001] This invention relates to the field of industrial automation quality inspection technology, specifically to an AI-based review platform system and quality management method for the electronics manufacturing industry. Background Technology

[0002] In the SMT (Surface Mount Technology) assembly process of the electronics manufacturing industry, Automated Optical Inspection (AOI) is a core step in ensuring product quality. AOI equipment uses optical imaging and algorithms to identify soldering, positional, and appearance defects in components. However, due to the complex production environment, differences in component reflectivity, and algorithm limitations, AOI equipment generally suffers from a high false call rate, meaning that a large number of actually qualified components are mistakenly identified as potentially defective. Therefore, the industry typically requires manual verification of anomalies detected by AOI; this process is called re-verification. Currently, the existing technical solutions in the industry mainly include the following: (i) Traditional offline manual review workstations consist of AOI inspection equipment and independent manual review software. After the AOI equipment completes scanning, it sends all images and coordinates of suspected defects to the review station. Operators click on the images one by one with the mouse and compare them with standard sample images to determine OK / NG. The system only records the final judgment result and uploads it to the Manufacturing Execution System (MES). The shortcomings of this solution are: false alarm filtering relies entirely on manual labor, and operators are prone to missing detections due to fatigue when faced with a high false alarm rate; different operators have different judgment criteria for minor defects, lacking a unified standard; the data dimension is single, and it is impossible to statistically analyze the distribution trend of specific positioning numbers or specific defect types in real time, making it difficult to guide front-end process improvement.

[0003] (ii) Embedded standalone AI filtering plugin: This solution integrates a simple AI inference module within the AOI inspection machine. Before the image is uploaded to the review station, it performs preliminary screening using a preset fixed model. Images judged by AI to be "extremely likely OK" are directly filtered and no longer pushed to the human end. Its shortcomings are: lack of flexible confidence threshold configuration, inability to dynamically adjust the filtering intensity according to the accuracy requirements of different product models; standalone processing mode is prone to inference stacking and latency under high concurrency and large-batch inspection requirements; lack of load balancing capability, and poor system scalability.

[0004] (III) General Industrial MES Quality Reporting System. As a sub-module of MES, this system is mainly responsible for collecting statistical results after AOI and manual review, and generating macro reports such as first pass rate and defect rate through database queries. Its shortcomings are: it focuses on result recording and lacks an interactive interface specifically optimized for review operations; the data granularity is insufficient, usually only recording results at the board level, and it cannot achieve deep correlation analysis from board to specific component reference number; it lacks a real-time dashboard, and managers cannot intuitively see the real-time load of the current production machine and sudden quality fluctuations.

[0005] Regarding prior patents, for example, US5544256A discloses an automated defect classification system, but it does not disclose a dual-threshold confidence level splitting mechanism based on pass and fail confidence thresholds, thus failing to divide the detection results into three intervals: automatically pass, automatically fail, and awaiting manual review, thereby achieving precise allocation of detection resources. US20240330671A1 discloses an automatic wafer defect detection and classification method for semiconductor hybrid bonding processes. This method uses deep learning models such as VisionTransformer and convolutional neural networks to identify defects in wafer surface images. While classifying and using mix-up data augmentation technology to improve the model's generalization ability, this patent does not disclose a correlation analysis mechanism for defect images and multi-source data such as process parameters and equipment status during the re-judgment process. It cannot achieve in-depth tracing of defect causes by integrating data from the entire production chain, making it difficult to fundamentally optimize the production process to reduce the defect rate. US20240248463A1 discloses an AI-enhanced closed-loop control system, but its application scenarios are mainly concentrated in the field of process control. It does not disclose a multi-level quality data modeling and tag number-level backtracking analysis mechanism for the AOI re-judgment process in the electronics manufacturing industry.

[0006] However, none of the aforementioned patents disclose the comprehensive AI review platform proposed in this invention, which integrates high-concurrency distributed inference, dynamic confidence level splitting, efficient human-computer interaction, and multi-level quality data modeling. In particular, none of them solve the technical problems of how to achieve dynamic scheduling of inference resources in high-concurrency scenarios on large-scale production lines, how to minimize the workload of manual review while ensuring the detection rate through a dual-threshold mechanism, and how to establish a granular quality backtracking model from board to specific device reference number.

[0007] Therefore, there is a need for an AI-based review platform system and quality management method for the electronics manufacturing industry. Summary of the Invention

[0008] This invention provides an AI-powered re-judgment platform system and quality management method that integrates image reasoning, dynamic confidence level splitting, efficient human-computer interaction, and multi-dimensional quality closed-loop analysis. It aims to address the problems of low efficiency and easy omissions in traditional manual re-judgment in industrial automated inspection, lack of flexibility and scalability of embedded AI plugins, and insufficient granularity of data analysis in general quality systems.

[0009] This invention provides an AI-powered review platform system for the electronics manufacturing industry, comprising: The image acquisition module is used to receive image data of the device under inspection from at least one automated optical inspection device in real time; the image data is associated with a corresponding board identifier and the device reference number on the board. The image inference module is used to receive image data based on multiple inference servers through a load balancing strategy, and to use a pre-trained deep learning model to infer the image data to generate inference results. The inference results include a first confidence level that the image in the image data belongs to a qualified category and a second confidence level that it belongs to an unqualified category. The dynamic image splitting module is used to obtain preset pass / fail confidence thresholds and fail / fail confidence thresholds, and perform the following image splitting judgment: when the first confidence level is greater than or equal to the pass / fail confidence threshold, the corresponding image is marked as finally pass; when the second confidence level is greater than or equal to the fail / fail confidence threshold, the corresponding image is marked as finally fail; when the first confidence level is less than the pass / fail confidence threshold and the second confidence level is less than the fail / fail confidence threshold, the corresponding image and its inference result are pushed to the manual review queue. The human-computer interaction review module is used to display the images in the manual review queue on the manual review workstation and provide a result inversion interaction interface to respond to the inversion command of the manual review operator and correct the final judgment result of the corresponding image. The quality data modeling and analysis module is used to establish a multi-level data model including board identifiers and component reference numbers. It associates the final judgment result with the corresponding board identifier and the component reference number under that board identifier, and generates defect trend analysis data based on the aggregated statistics of specific component reference numbers.

[0010] Furthermore, the load balancing strategy includes: retrieving the corresponding inference server address list based on the product model, distributing inference requests using a weighted round-robin algorithm, and using an in-memory database to cache and record the current number of connections and response time of each inference server in order to dynamically adjust the distribution weight.

[0011] Furthermore, the image inference module is deployed using a container orchestration platform and configured with an elastic scaling component. The elastic scaling component is used to monitor the inference queue depth in the in-memory database. When the inference queue depth exceeds a first threshold, an inference server instance is added. When the inference queue depth is below a second threshold and remains below a predetermined time, an inference server instance is reduced.

[0012] Furthermore, in the weighted round-robin algorithm of the image inference module, the weight of each inference server is preset with an initial weight based on its hardware configuration, and dynamically adjusted according to the following formula based on the real-time response time recorded in the memory database cache: New weight = Initial weight × (Base response time / Current average response time); where the base response time is the standard response time preset according to the hardware configuration of each inference server, and the current average response time is the value obtained by averaging multiple real-time response times of each inference server in the most recent statistical period, read from the memory database.

[0013] Furthermore, the dynamic traffic diversion module is also used to implement mandatory review of high-risk defect types: One or more high-risk defect types are pre-stored, including but not limited to wrong parts, reversed polarity, and missing parts; Using a deep learning model, the input image data contains the image to be classified, and the output is the defect type to which the image belongs. When a defect type matches a high-risk defect type, it is not subject to the limitations of image diversion judgment and the image to be diverted and its inference results are forcibly pushed to the manual review queue.

[0014] Furthermore, the human-computer interaction review module is also used for: The manual review workstation's interface simultaneously displays multiple images from the manual review queue for batch review. In response to the operator pressing a numeric shortcut key, the judgment result of the image to be reviewed is inverted; the numeric shortcut key is pre-configured and corresponds to a specific position in the image to be reviewed.

[0015] Furthermore, batch re-judgment is performed, including: pre-setting color borders for the images to be re-judged based on the reasoning results, wherein qualified categories correspond to the first color border and unqualified categories correspond to the second color border; The process of inverting the judgment result of the image to be reviewed includes: in response to the operator's operation instruction, switching the judgment result of the image to be reviewed between the qualified and unqualified categories, and simultaneously switching the color borders corresponding to the qualified and unqualified categories respectively.

[0016] Furthermore, the human-computer interaction review module also integrates a text-to-speech unit, which is configured as follows: When switching to a different image for review, the identification information of that image is broadcast; the identification information includes the board identifier and the device reference number. When a high-risk defect type image is received for reassessment, the corresponding warning voice is broadcast according to the high-risk defect type, and the broadcast priority of the warning voice is higher than the broadcast priority of the image identification information.

[0017] Furthermore, the quality data modeling and analysis module is also used for: Calculate the defect rate of each device reference number within a predetermined time window, compare the defect rate with a preset reference number warning threshold, and when the defect rate exceeds the reference number warning threshold, mark the device reference number as a high-frequency fault point and generate a warning message for that device reference number. In response to a query request, it retrieves and displays the historical final judgment results associated with a specific device reference number and the corresponding device image, and supports the export of reports containing data from at least one level of a multi-level data model and adverse trend analysis data.

[0018] This invention proposes a quality management method for an AI-based review platform system applied in the electronics manufacturing industry, comprising the following steps: Receive image data of the device to be inspected from the automated optical inspection equipment; the image data is associated with the corresponding board identifier and the device reference number on the board. Based on multiple inference servers, image data is distributed through a load balancing strategy, and deep learning models are used to infer the images to generate a first confidence level and a second confidence level. Obtain the pass / fail confidence threshold and the fail / fail confidence threshold. If the first confidence level is greater than or equal to the pass / fail threshold, it is judged as pass / fail. If the second confidence level is greater than or equal to the fail / fail threshold, it is judged as fail / fail. Otherwise, it is pushed to the manual review queue. Images with defects belonging to a preset high-risk defect type are forcibly pushed to the manual review queue; The manual review workstation displays images in the manual review queue, provides a result inversion interface, responds to inversion commands to correct the final judgment result of the corresponding image, performs inversion operation, and simultaneously switches the color border. Establish a multi-level data model that includes board identifiers and component reference numbers, associate the final judgment results with the corresponding board identifiers and component reference numbers, aggregate and statistically analyze specific component reference numbers, and generate defect trend analysis data.

[0019] Compared with the prior art, the present invention has the following beneficial effects: By setting pass / fail confidence thresholds and fail / fail confidence thresholds, three intervals are formed: the high-confidence pass / fail interval is automatically judged as pass, the high-confidence fail / fail interval is automatically judged as fail, and only suspected images with confidence levels in the middle interval are pushed to manual review. This significantly reduces the workload of manual review and achieves precise allocation of testing resources. Batch review mode can significantly improve the efficiency and consistency of single-point operations in manual review.

[0020] An image inference module is employed, which distributes inference requests to multiple inference servers using a weighted round-robin algorithm. An in-memory database is used to record the number of connections and response time of each server in real time, and the distribution weight is dynamically adjusted. At the same time, a container orchestration platform and elastic scaling components are used to automatically increase or decrease inference instances based on the depth of the inference queue, which can eliminate the inference stack in high-concurrency environments and achieve linear expansion of the system's processing capacity.

[0021] A multi-level data model containing board identifiers and component reference numbers was established, linking the final judgment result to each component on each board. Through aggregated statistics, the defect rate of each component reference number within a predetermined time window can be calculated. When the defect rate exceeds the warning threshold, it is automatically marked as a high-frequency fault point and a warning message is generated. This enables accurate quality backtracking from the board to the component reference number, assisting in closed-loop process optimization.

[0022] The integrated text-to-speech unit broadcasts the board identifier and device reference number when switching images to be reviewed, and broadcasts the corresponding warning voice when a high-risk defect image is received. The warning voice has a higher priority than the ordinary broadcast, ensuring that operators pay attention to key defects in a timely manner, reducing operator fatigue and providing real-time abnormal warnings. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the AI-based review platform system for the electronics manufacturing industry according to the present invention.

[0024] Figure 2 This is a schematic diagram of the multi-level quality data model and closed-loop analysis of the present invention.

[0025] Figure 3 This is a schematic diagram illustrating the quality management method steps of the AI ​​review platform system applied to the electronics manufacturing industry according to the present invention. Detailed Implementation

[0026] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. This section describes the working principle and technical effects of the present invention by way of embodiments, but the scope of protection of the present invention is not limited to these embodiments.

[0027] Example 1: System Overall Architecture and Working Principle. (For example...) Figure 1 As shown, the AI ​​review platform system provided by this invention includes the following components: The image acquisition module is deployed on the surface mount technology production line and connects to multiple AOI inspection devices via a network. After the AOI device completes the scanning of a board, it packages and sends each device's partial image along with the board's unique identifier (board ID) and the device's reference number (RefDes, such as R102, C204, etc.) to the image acquisition module. The image acquisition module unifies and caches the received image data to form a queue for inference. The board identifier is used to uniquely identify a board being inspected, and the device reference number is used to identify a component on the board and is subordinate to the board identifier.

[0028] The image inference module contains multiple inference servers, each loaded with a pre-trained deep learning model. This module receives image data from the image acquisition module through a load balancer and distributes the images to idle inference servers. The inference servers perform forward computation on the images and output the probability value of the image belonging to the qualified category (first confidence level) and the probability value of belonging to the unqualified category (second confidence level). For example, for an image, the model may output a 95% first confidence level and a 5% second confidence level, indicating that the device is highly likely to be qualified.

[0029] The dynamic sorting module receives the inference results and performs image sorting judgment based on pre-configured pass / fail confidence thresholds (default 0.7) and fail / fail confidence thresholds (default 0.5). If the first confidence level is greater than or equal to the pass / fail confidence threshold, the image is directly judged as ultimately passable, and no manual review is required. If the second confidence level is greater than or equal to the fail / fail confidence threshold, the image is directly judged as ultimately failable, and again, no manual review is required. If the first confidence level is less than the pass / fail confidence threshold and the second confidence level is less than the fail / fail confidence threshold, meaning the model is not confident enough about either pass or fail, it is judged as a "suspected defect," and the image and inference results are pushed to the manual review queue. In addition, the dynamic sorting module also stores a list of high-risk defect types (such as wrong material, reversed polarity, missing parts). For defect types output by the model simultaneously, if they belong to high-risk types, they are forcibly pushed to the manual review queue regardless of the confidence level.

[0030] The human-computer interaction review module is deployed on the manual review workstation. This module pulls images to be reviewed from the manual review queue and displays them on the operation interface in batch mode (e.g., 10 images per page). Each image has a preset color border based on the AI ​​reasoning result: images judged as qualified by the AI ​​have a green border, and images judged as unqualified by the AI ​​have a red border. The operator can press the number keys (1-9, 0) to invert the result of the corresponding image. For example, if an image originally had a green border (the AI ​​judged it as qualified), but the operator believes that there is actually a defect, pressing the corresponding number key will change the judgment result of the image from qualified to unqualified, and the border will change to red. The module also integrates a text-to-speech unit, which announces the board ID and component reference number when loading a new batch of images to be reviewed, and announces warning voice such as "Please note, risk of incorrect components" when receiving high-risk defect images.

[0031] The quality data modeling and analysis module receives the final judgment result output by the human-computer interaction review module, associates it with the corresponding board identifier and component tag number, and writes it into a relational database. The database organizes data according to the hierarchical structure of "board ID-component tag number". This module periodically performs aggregate statistics, such as calculating the defect rate (number of NGs / total number of inspections) of each component tag number in the past hour. When the defect rate of a certain tag number exceeds a preset threshold (such as 5%), it automatically marks the tag number as a "high-frequency failure point" and highlights it in the defect trend dashboard. At the same time, it sends an alert to the process engineer via WeChat or email. Engineers can also use the query interface to input a specific component tag number (such as C1024) and time range to retrieve all historical judgment results and corresponding original images for in-depth analysis and traceability.

[0032] The working principle of this embodiment is as follows: high-concurrency image data is processed through the image inference set module, and the results confirmed by AI are automatically judged using a dual threshold diversion mechanism. Only suspected images are handed over to manual review. The manual review adopts batch display and shortcut key inversion to improve efficiency. Finally, all judgment results are stored at the board-tag level for quality trend analysis and process improvement. This architecture achieves a balance between detection accuracy and manual efficiency.

[0033] Example 2: Specific Implementation of Image Inference and Load Balancing. This example details the load balancing strategy of the image inference module and its technical effects.

[0034] Multiple AOI devices simultaneously send image data to the system, forming a high-concurrency request queue. The load balancer (which can use Nginx or a self-developed component) maintains a list of inference server addresses. Each server is assigned an initial weight based on its hardware configuration (such as GPU model). For example, server A is configured with a high-performance graphics card and has an initial weight of 3; server B is configured with another high-performance graphics card and has an initial weight of 2.

[0035] The load balancer uses a weighted round-robin algorithm to distribute requests: requests are allocated cyclically according to weight ratios, with the request sequence being A, A, B, A, B… Simultaneously, the system utilizes an in-memory database (such as Redis, but not limited to this) to record the current number of connections and the response time of each inference session for each inference server in real time. Every 500ms, the load balancer reads the real-time response time of each server from the in-memory database, calculates the current average response time, and then dynamically adjusts the weights according to the formula: New Weight = Initial Weight × (Base Response Time / Current Average Response Time), where the base response time is a preset standard based on the server hardware configuration. Response time (e.g., the baseline response time for a high-performance graphics card is 50ms, and for another high-performance graphics card it is 80ms). The current average response time is the average response time of all requests within the most recent statistical period (e.g., 10 seconds). If a server slows down due to high load (e.g., from 50ms to 150ms), the current average response time increases, the ratio in the formula becomes less than 1, and the new weight is reduced to 1 / 3 of the initial weight. Subsequent requests will be distributed to this server less frequently. Conversely, the weight of an idle server (e.g., the response time decreases from 80ms to 40ms) will be increased to twice the initial weight, allowing it to receive more requests.

[0036] In addition, the image inference module is deployed using a Kubernetes container orchestration platform. The system is configured with a horizontal elastic scaling component (HPA), which continuously monitors the inference queue depth (i.e., the number of images to be processed) in the in-memory database. When the queue depth exceeds the first threshold (e.g., 1000 images), the number of inference server instances is automatically increased (e.g., from 2 to 3). When the queue depth falls below the second threshold (e.g., 100 images) for a predetermined period of time (e.g., 5 minutes), the number of instances is automatically reduced to conserve resources.

[0037] The working principle of this embodiment is as follows: After the AOI device collects image data, it sends the image data and corresponding device number, timestamp, and other metadata to the load balancer through a message queue. The load balancer first parses the priority identifier in the request header. For product images of urgent work orders or high-value batches on the production line, they are marked as high-priority requests and enter the front end of the inference queue first. Subsequently, the load balancer distributes the image data to the target server according to the dynamic weights of each inference server. After receiving the image, the inference server calls the pre-installed deep learning model (such as a defect detection model trained based on YOLOv8 or ResNet architecture) to perform forward computation. The model outputs a JSON format result containing the defect category, confidence score, and defect region coordinate array. This result, along with the hash value of the original image, is returned to the load balancer, which routes the request back to the corresponding AOI device or writes it to the result database according to the request source.

[0038] During dynamic load adjustment, if a certain inference server experiences GPU memory overflow or abnormal process exit, the instance will be marked as unavailable. The load balancer will immediately remove it from the available list and redirect the requests being processed to other healthy instances to ensure that a single point of failure does not affect the overall inference throughput. At the same time, the abnormal event will be recorded in the log system, triggering an alarm to notify the operations and maintenance personnel to intervene and investigate.

[0039] The horizontal elastic scaling component (HPA) works in conjunction with the load balancer to form a closed-loop resource management system. During production line shift changes or peak order periods, the inference queue depth may rapidly increase. Once the HPA component detects that the queue depth exceeds the first threshold, it completes the image retrieval, container startup, and service registration process for the new inference instance within 30 seconds. The new instance is automatically added to the load balancer's address list and participates in request distribution according to its initial weight. When the production line enters a low-load operation state, and the queue depth remains below the second threshold for 5 minutes, the HPA component terminates redundant instances one by one according to the first-in, first-out principle. The released GPU resources are returned to the resource pool for other computing task scheduling. This elastic mechanism enables the system to meet the timeliness requirements of real-time inference (average response time controlled within 100ms) while optimizing infrastructure costs to a level close to the actual computing power requirements, even when facing the typical order fluctuations in the electronics manufacturing industry.

[0040] The technical effects of the above solution are as follows: the combination of weighted round-robin algorithm and dynamic weight adjustment avoids inference stack overflow caused by single machine overload, and the system throughput is effectively improved with the number of servers; the elastic scaling mechanism enables the system to adapt to production line peaks and troughs, automatically expanding capacity during peak periods to ensure real-time performance, and shrinking capacity during trough periods to reduce costs.

[0041] Example 3: Specific Implementation of Confidence-Based Triage and Mandatory Reassessment for High-Risk Cases. This example details the working principle of the dynamic triage module and its impact on the workload of manual reassessment.

[0042] Assuming a surface mount technology production line manufactures a smartphone motherboard, the AOI equipment generates approximately 100,000 device images daily. According to industry data, the average manual review time for a single image is approximately 1.0-1.5 seconds (including image recognition, defect assessment, and result recording). This embodiment uses 1.2 seconds as a typical estimate. Therefore, all 100,000 images require approximately 7 people per day (calculation formula: 100,000 images × 1.2 seconds / image ÷ 3600 seconds / hour ÷ 8 hours / day ≈ 4.2 people. Considering actual factors such as image retrieval delays, defect assessment discrepancies, system response waiting times, staff rotation, and operational fatigue during the review process, a 1.5 times management redundancy coefficient is introduced, ultimately determining the manual review workload to be 6.3 people per day, or 7 people per day based on a full-scale shift schedule). This invention significantly reduces the amount of manual review through dual-threshold diversion. The system administrator configures the first and second confidence levels based on the product's accuracy requirements. For ordinary consumer electronics products, the first confidence level can be set to 0.85 and the second confidence level to 0.70. This means that products with a model's probability of being qualified ≥85% are automatically judged as qualified, and products with a probability of being unqualified ≥70% are automatically judged as unqualified. Only products that do not meet either requirement require manual review. For high-reliability products (such as automotive products), the first confidence level can be set to 0.98 and the second confidence level to 0.92. The requirements are more stringent, and the amount of manual review increases accordingly, but it ensures zero-defect shipment.

[0043] In addition, the dynamic routing module maintains a list of high-risk defect types. For example, "wrong material" is listed as high-risk. If an image model outputs a defect type of "wrong material," then regardless of the pass / fail probability values, the image will still be forcibly pushed to the manual review queue and marked as "high-risk - forced review." This is because wrong materials can cause product functionality to fail and must be confirmed manually.

[0044] The working principle of this embodiment is as follows: After the AOI device acquires an image of a device, the image preprocessing module first performs noise reduction, normalization and size standardization on it to ensure that the input data meets the requirements of model inference; then, the multi-task defect detection model performs forward inference on the image and outputs three key pieces of information: the probability of passing, the probability of failing, and the defect type classification result.

[0045] After receiving the above output, the dynamic sorting module makes sorting decisions according to preset logic. First, it checks whether the defect type belongs to the high-risk defect list. If it does, it directly triggers the forced re-judgment mechanism, and the image enters the manual re-judgment queue and is marked as high-risk. This process does not refer to any probability values. For non-high-risk defect types, the module further compares the relationship between the pass probability and the first confidence level, and the fail probability and the second confidence level. When the pass probability is not lower than the first confidence level, the system determines that the device is qualified, automatically releases it, and records the inspection log. When the fail probability is not lower than the second confidence level, the system determines that the device is unqualified, automatically marks it, and triggers the downstream rejection or rework process. Only when the pass probability is lower than the first confidence level and the fail probability is lower than the second confidence level is the image judged as a blurry sample and enters the manual re-judgment queue to wait for manual confirmation. The technical effects of the above solution are as follows: by adjusting the first confidence level and the second confidence level, a balance can be achieved between the detection rate and the amount of manual review. For ordinary products, a more lenient threshold can be set, and the automatic judgment rate can reach more than 90%, while the amount of manual review is reduced by 90%. For high-reliability products, a more stringent threshold can be set, and the automatic judgment rate is about 70%, but the mandatory review mechanism for high-risk defects ensures zero missed detection of critical defects.

[0046] Example 4: Specific Implementation and Efficiency Improvement of Human-Computer Interaction Review. This example details the working principles of batch review, shortcut key inversion, and synchronous switching of color borders in the human-computer interaction review module.

[0047] The manual review workstation runs client software. The interface is divided into an image display area and an operation area. The image display area displays 10 images to be reviewed simultaneously in a grid format (e.g., 2 rows and 5 columns). Below each image, the device reference number (e.g., R102, C204, etc.) and the AI ​​prediction result (OK or NG) are displayed. The system presets the border color for each image based on the AI ​​reasoning result: green for AI-judged OK and red for AI-judged NG.

[0048] The operator scans these images one by one. If the AI ​​makes a mistake, the operator simply presses the corresponding number key to reverse the judgment. For example, if the AI ​​judges the third image as OK (green border), but the operator sees a cold solder joint defect, the operator presses the "3" key on the keyboard. The system responds to this key, changing the final judgment of the image from OK to NG, and simultaneously changing the border color from green to red. The entire operation does not require moving the mouse or taking the fingers off the keyboard, and the judgment time for a single image can be shortened to about 0.38 seconds.

[0049] After the batch re-evaluation is completed, the operator clicks the "Submit" button, and all the corrected judgment results are sent to the quality data modeling and analysis module. The system automatically loads the next batch of 10 images and broadcasts a voice prompt: "New batch of images to be re-evaluated, board ID: PCB2024001".

[0050] For images with high-risk defects that require mandatory re-evaluation, the system will play a specific warning voice when loading, such as "Attention: Risk of incorrect material, tag number R102". The playback priority of this warning voice is higher than that of ordinary board switching reminders, ensuring that the operator pays attention immediately.

[0051] The working principle of the above technical solution is as follows: By presenting the AI ​​prediction result intuitively with a colored border and combining it with numeric shortcut keys to achieve quick inversion, it can realize the interaction of visual recognition, keyboard response, and status synchronization. When the AI ​​outputs OK / NG result, the system's underlying layer renders the image border in real time according to the preset color mapping rules (OK corresponds to green RGB value 0,255,0, and NG corresponds to red RGB value 255,0,0). The operator can quickly locate the item to be corrected based on visual perception. A one-to-one mapping relationship is established between the numeric shortcut keys and the image grid positions (1-10). After pressing the corresponding numeric key, the client software triggers the event handling function, which simultaneously executes three core functions. The system performs the following operations: First, it updates the determination result field of the device reference number in the database (changing is_ok=1 to is_ok=0 or vice versa); second, it calls the UI rendering interface to switch the border color value; third, it records the operation timestamp and operator ID to the operation log table. For high-risk defects, the system manages voice broadcast tasks through a priority queue. Warning voices (such as the risk of incorrect materials) are marked as the highest priority (priority=1), while ordinary board switching prompts are given the default priority (priority=3). When processing the queue, the audio playback engine prioritizes high-priority audio files. At the process level, the risk classification mechanism improves the efficiency of anomaly identification.

[0052] The technical advantages of this embodiment are as follows: batch display reduces page switching time, inverting the number shortcut keys eliminates the delay of mouse movement and clicking, and the synchronous switching of color borders provides intuitive visual feedback and avoids misoperation; at the same time, the linkage between the inversion operation and the color borders reduces the missed judgment rate caused by fatigue, and the voice broadcast mechanism allows operators to know that new images have arrived in a timely manner without having to stare at the screen all the time when switching tasks.

[0053] Example 5: Specific Implementation of Multi-Level Quality Data Modeling and Trend Analysis. This example details how the quality data modeling and analysis module achieves accurate quality backtracking from board to component reference number. The system establishes a relational database (such as PostgreSQL) to store quality data. The core data table design is as follows: Once all components on a circuit board have been tested and re-evaluated, the system will summarize the final evaluation results of all components on that board and calculate the board's pass rate (i.e., number of qualified components / total number of components). Simultaneously, the system will perform aggregated statistics by component tag number. After executing the above query, the system will obtain the defect rate for each component tag number over the past hour. The preset tag number warning threshold is 5%. If the calculated defect rate for a certain tag number (e.g., C1024) is 8.5%, the system will automatically mark that tag number as a "high-frequency fault point" and highlight it in the defect trend dashboard, while simultaneously generating a warning message and sending it to the process engineer.

[0054] After receiving the warning, the engineer entered the system's query interface, input the tag number "C1024" and the time range, and the system retrieved all historical inspection results and corresponding original images for that tag number. The engineer reviewed these images one by one and found that the defects were mainly concentrated in the "tombstone" type and appeared in a certain time period. Combining the production records, the engineer found that a nozzle of the pick-and-place machine was worn during that time period, which caused unstable placement pressure. After replacing the nozzle, the defect rate of that tag number returned to normal.

[0055] Combination Figure 2 As shown, the multi-level quality data modeling and closed-loop analysis process of the present invention includes the following sequential steps: (1) Board-level summary data: The system uses the board identifier as the basic unit to summarize the test results of all devices on each board and calculates the first pass rate (number of qualified devices / total number of devices), number of defective devices and distribution of major defect types of the board; the board-level data is used for macro-monitoring of production line yield and quick location of abnormal batches.

[0056] (2) Component reference number level details: The system drills down to decompose the test results of each board to specific component reference numbers (such as R102, C204). Record the judgment results (OK / NG), defect type, AI confidence level and manual review and correction information for each reference number at different boards and at different time points; this level of data is the key to locating high-frequency fault points.

[0057] (3) Defect type feature extraction: For devices judged as unqualified, the system automatically extracts their defect type (such as tombstoning, insufficient solder, bridging, wrong material, reverse polarity, etc.) and quantifies the image features of each defect type (such as offset distance, solder paste area ratio, etc.). Through cluster analysis, the system identifies the correlation between specific defect types and tag numbers, equipment, and shifts.

[0058] (4) Defect quantity trend chart over time: The system uses time window (such as hour, shift, day) as the horizontal axis and the number of defects of each defect type or digit as the vertical axis to generate a trend curve. This trend chart can intuitively show the start and end time, peak value and periodicity of quality fluctuations, and help engineers judge whether the problem is sudden (such as equipment failure) or gradual (such as nozzle wear).

[0059] (5) Process optimization suggestions / trend backtracking: Based on the above analysis, the system automatically generates or assists engineers in formulating process optimization suggestions. For example, for frequent tombstoning defects of a certain reference number, it is recommended to adjust the placement pressure of the pick and place machine or modify the stencil opening; for a surge in incorrect materials during a certain period, it is recommended to check the material loading records; at the same time, the system supports backtracking from the abnormal points on the trend chart to the corresponding board, component reference number and original inspection image, forming a closed-loop management of "discovering problems → locating causes → taking measures → verifying effects".

[0060] The working principle of the above technical solution is as follows: By establishing a data table linking boards, components, and test results, the system achieves structured storage and hierarchical association of quality data. After the board completes the test and re-judgment, the system automatically summarizes and calculates the board pass rate based on the foreign key relationship between the data tables, and aggregates and statistically analyzes the defect rate by component tag number. When the defect rate of a specific tag number exceeds a preset threshold, an early warning mechanism is triggered, highlighting high-frequency fault points and notifying engineers. Engineers can accurately retrieve historical test data and original images by tag number and time range, and locate the root cause of process problems by combining production records. Then, targeted measures (such as replacing nozzles) are taken to improve production quality, forming a closed-loop management process from data acquisition, modeling and analysis, anomaly early warning to problem solving.

[0061] The technical effect of this embodiment is that by associating the detection results with specific device reference numbers, the system can accurately locate high-frequency fault points in the production process, providing data support for process improvement, enabling engineers to quickly and accurately find the root cause of the problem, significantly shortening the troubleshooting time and improving product yield.

[0062] Example 6: Model Training Method. The deep learning model of this invention uses a ResNet-50 residual network enhanced with Feature Pyramid Network (FPN) as the backbone feature extractor, connected to a fully connected classification layer at the back end. The model input is a 128×128×3 RGB local image tensor of the device, and the output is a 20-dimensional classification probability vector, corresponding to 20 predefined specific defect types (such as cracks, scratches, pores, etc.). The number of neurons in the fully connected classification layer is set to 20, and a softmax activation function is used to convert the output into a probability distribution. Finally, the category with the highest probability is taken as the defect type predicted by the model.

[0063] The training dataset contains at least 50,000 labeled images, covering more than 20 typical defects such as insufficient tin, bridging, tombstone erection, incorrect material, and polarity reversal. The ratio of positive to negative samples is controlled within 3:1, and data augmentation is performed using brightness and contrast enhancement. The Adam optimizer is used for training, with an initial learning rate of 1e-4. Cosine annealing is used to adjust the learning rate. The training lasts for 100 epochs, with a batch size of 64. After training, the model's false positive rate on the validation set is less than 5%, and the false negative rate on the non-validation set is less than 0.1% (after manual review).

[0064] The working principle and beneficial effects of the above technical solution are as follows: By combining FPN and ResNet-50 as the backbone feature extractor, feature information from different levels can be effectively fused. This preserves the detailed features at the lower level to capture minor defects, while utilizing the semantic features at the higher level to accurately determine the defect type, thus improving the model's ability to identify complex defects. The input uses a 128×128×3 RGB device local image tensor, which ensures the detail information of the image while controlling the amount of input data, facilitating efficient model processing. The output is a two-dimensional classification probability vector, intuitively providing the probabilities of OK and NG, providing a clear basis for subsequent judgments. The training dataset has at least 50,000 labeled images and covers more than 20 typical defects, ensuring that the model can learn effectively in diverse defect scenarios. The ratio of positive to negative samples is controlled within 3:1 to avoid the adverse effects of sample imbalance on model training. Data augmentation methods that enhance brightness and contrast further enrich the diversity of training samples and enhance the model's generalization ability. By using the Adam optimizer in combination with a cosine annealing strategy to adjust the learning rate, the model can converge better during training and avoid getting trapped in local optima. After 100 epochs of training, the batch size was set to 64 to ensure the stability and efficiency of training.

[0065] Example 7: Parameter Definition and Usage Instructions of the Weighting Formula. In some embodiments of the present invention, a dynamic weight adjustment formula is adopted: New Weight = Initial Weight × (Baseline Response Time / Current Average Response Time). The parameters in the formula are defined as follows: The initial weight is a preset baseline weight based on the hardware configuration of each inference server (such as GPU model, number of CPU cores, and memory size). The stronger the hardware performance, the larger the initial weight. The baseline response time corresponds to the initial weight and is a preset standard response time based on the server hardware configuration. For example, the standard time for a high-performance graphics card to process a 128×128 image is 50ms, and for another high-performance graphics card it is 80ms. The current average response time is the arithmetic mean of the actual response times of each inference server in processing all requests within the most recent statistical period (such as 10 seconds), read from the memory database. The physical meaning of the formula is: when the server's current actual response speed is faster than the baseline response speed (i.e., the current average response time is less than the baseline response time), the new weight is greater than the initial weight, and the server will receive more requests; conversely, the new weight is less than the initial weight, and fewer requests are received. This enables dynamic adjustment of inference resources. The above formula can be modified according to actual needs, such as adding a smoothing factor to avoid drastic fluctuations in weights, or using an exponentially weighted moving average instead of an arithmetic average. These modifications are all equivalent embodiments of the present invention.

[0066] The working principle and beneficial effects of the above technical solution are as follows: By dynamically adjusting the weights of each inference server, intelligent and flexible request allocation is achieved. When a server processes faster due to its hardware performance advantage or low current load, its new weight increases, and the system automatically directs more requests to that server to fully utilize its processing capacity. Conversely, when a server's response slows down due to increased load, its weight decreases, and the number of requests allocated decreases accordingly, avoiding the impact of individual server performance degradation on the overall system's response efficiency. This dynamic adjustment mechanism based on real-time performance data can significantly improve the utilization rate of inference resources, ensuring that the system maintains a highly efficient and stable operating state under different load conditions. At the same time, it avoids resource waste or overload problems that may occur under static weight allocation methods, thereby optimizing the overall processing performance and user experience of the AI ​​review platform for the electronics manufacturing industry.

[0067] Example 8: Figure 3 As shown, this invention proposes a quality management method for an AI-based review platform system applied in the electronics manufacturing industry, comprising the following steps: Receive image data of the device to be inspected from the automated optical inspection equipment; the image data is associated with the corresponding board identifier and the device reference number on the board. Based on multiple inference servers, image data is distributed through a load balancing strategy, and deep learning models are used to infer the images to generate a first confidence level and a second confidence level. Obtain the pass / fail confidence threshold and the fail / fail confidence threshold. If the first confidence level is greater than or equal to the pass / fail threshold, it is judged as pass / fail. If the second confidence level is greater than or equal to the fail / fail threshold, it is judged as fail / fail. Otherwise, it is pushed to the manual review queue. Images with defects belonging to a preset high-risk defect type are forcibly pushed to the manual review queue; The manual review workstation displays images in the manual review queue, provides a result inversion interface, responds to inversion commands to correct the final judgment result of the corresponding image, performs inversion operation, and simultaneously switches the color border. Establish a multi-level data model that includes board identifiers and component reference numbers, associate the final judgment results with the corresponding board identifiers and component reference numbers, aggregate and statistically analyze specific component reference numbers, and generate defect trend analysis data.

[0068] The working principle and beneficial effects of the above technical solution are as follows: First, by receiving image data of the device under test collected by an automated optical inspection device and associating it with the board identifier and device reference number, accurate data traceability is ensured. Next, the image data is efficiently distributed using multiple inference servers and a load balancing strategy. First and second confidence levels are generated through parallel inference using a deep learning model, enabling rapid processing of the inspection task and preliminary result determination. The setting of pass / fail thresholds provides clear standards for automatic screening, effectively reducing the scope of manual intervention. For automatically determined blurry images, they are pushed to a manual review queue. For images with defects belonging to a preset high-risk defect type, they are forcibly pushed to the manual review queue. The manual review workstation provides a result inversion interface, allowing... Operators corrected the judgment results to ensure accuracy. Finally, by establishing a multi-level data model, the final judgment results were deeply correlated with the board and component reference numbers, and specific component reference numbers were aggregated and statistically analyzed to generate defect trend analysis data. The benefits are twofold: firstly, the load-balanced multi-server inference architecture greatly improves image processing efficiency, meeting the needs of large-scale inspection in the electronics manufacturing industry; secondly, the mechanism combining automatic judgment with manual review reduces labor costs while ensuring inspection accuracy. The establishment of the multi-level data model and defect trend analysis can help companies promptly identify quality problems with specific component reference numbers, providing data support for production process optimization and quality improvement, and contributing to improved overall product yield and production management levels.

[0069] Experimental Example: The system of this invention was deployed on the SMT production line of a large electronics foundry, connecting 8 AOI devices and configuring 4 inference servers (2 high-performance graphics cards and 2 more high-performance graphics cards). T_ok was set to 0.85 and T_ng to 0.70. After 72 hours of continuous operation, approximately 860,000 images were processed. Statistical results showed that approximately 580,000 images were automatically identified as qualified, approximately 150,000 images were automatically identified as unqualified, and approximately 130,000 images were pushed for manual review, reducing the amount of manual review by approximately 85%. For manual review, the average time per image was 0.38 seconds, an improvement of approximately 68% compared to the traditional mode (1.2 seconds). The average response latency of the inference server was 95ms, with a maximum latency not exceeding 150ms, and no stacking or frame dropping occurred. The system successfully identified 17 high-frequency fault points, and after process adjustments, the overall defect rate of the production line decreased by 22%.

[0070] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention is also intended to include these modifications and variations.

Claims

1. An AI-powered review and judgment platform system for the electronics manufacturing industry, characterized in that, include: An image acquisition module is used to receive image data of the device under inspection from at least one automated optical inspection device in real time. The image data is associated with a corresponding board identifier and the component reference number on that board; The image inference module is used to receive image data based on multiple inference servers through a load balancing strategy, and to use a pre-trained deep learning model to infer the image data to generate inference results. The inference results include a first confidence level that the image in the image data belongs to a qualified category and a second confidence level that it belongs to an unqualified category. The dynamic image splitting module is used to obtain preset pass / fail confidence thresholds and fail / fail confidence thresholds, and perform the following image splitting judgment: when the first confidence level is greater than or equal to the pass / fail confidence threshold, the corresponding image is marked as finally pass; when the second confidence level is greater than or equal to the fail / fail confidence threshold, the corresponding image is marked as finally fail; when the first confidence level is less than the pass / fail confidence threshold and the second confidence level is less than the fail / fail confidence threshold, the corresponding image and its inference result are pushed to the manual review queue. The human-computer interaction review module is used to display the images in the manual review queue on the manual review workstation and provide a result inversion interaction interface to respond to the inversion command of the manual review operator and correct the final judgment result of the corresponding image. The quality data modeling and analysis module is used to establish a multi-level data model including board identifiers and component reference numbers. It associates the final judgment result with the corresponding board identifier and the component reference number under that board identifier, and generates defect trend analysis data based on the aggregated statistics of specific component reference numbers.

2. The AI ​​review platform system for the electronics manufacturing industry according to claim 1, characterized in that, The load balancing strategy includes: retrieving the corresponding inference server address list based on the product model, distributing inference requests using a weighted round-robin algorithm, and using an in-memory database to cache and record the current number of connections and response time of each inference server in order to dynamically adjust the distribution weight.

3. The AI ​​review platform system for the electronics manufacturing industry according to claim 2, characterized in that, The image inference module is deployed using a container orchestration platform and configured with an elastic scaling component. The elastic scaling component is used to monitor the inference queue depth in the in-memory database. When the inference queue depth exceeds a first threshold, an inference server instance is added. When the inference queue depth falls below a second threshold and remains below a predetermined time, an inference server instance is reduced.

4. The AI ​​review platform system for the electronics manufacturing industry according to claim 2, characterized in that, In the weighted round-robin algorithm of the image inference module, the weight of each inference server is preset with an initial weight based on its hardware configuration, and dynamically adjusted according to the real-time response time recorded in the memory database cache, according to the following formula: New weight = Initial weight × (Base response time / Current average response time); where the base response time is the standard response time preset according to the hardware configuration of each inference server, and the current average response time is the value obtained by averaging multiple real-time response times of each inference server in the most recent statistical period, read from the memory database.

5. The AI ​​review platform system for the electronics manufacturing industry according to claim 1, characterized in that, The dynamic traffic splitting module is also used to enforce mandatory review of high-risk defect types: One or more high-risk defect types are pre-stored, including but not limited to wrong parts, reversed polarity, and missing parts; Using a deep learning model, the input image data contains the image to be classified, and the output is the defect type to which the image belongs. When a defect type matches a high-risk defect type, it is not subject to the limitations of image diversion judgment and the image to be diverted and its inference results are forcibly pushed to the manual review queue.

6. The AI ​​review platform system for the electronics manufacturing industry according to claim 1, characterized in that, The human-computer interaction review module is also used for: The manual review workstation's interface simultaneously displays multiple images from the manual review queue for batch review. In response to the operator pressing a numeric shortcut key, the judgment result of the image to be reviewed is inverted; the numeric shortcut key is pre-configured and corresponds to a specific position in the image to be reviewed.

7. The AI ​​review platform system for the electronics manufacturing industry according to claim 6, characterized in that, Batch re-judgment includes: pre-setting color borders for the images to be re-judged based on the reasoning results, wherein qualified categories correspond to the first color border and unqualified categories correspond to the second color border; The process of inverting the judgment result of the image to be reviewed includes: in response to the operator's operation instruction, switching the judgment result of the image to be reviewed between the qualified and unqualified categories, and simultaneously switching the color borders corresponding to the qualified and unqualified categories respectively.

8. The AI ​​review platform system for the electronics manufacturing industry according to claim 1, characterized in that, The human-computer interaction review module also integrates a text-to-speech unit, which is configured as follows: When switching to a different image for review, the identification information of that image is broadcast; the identification information includes the board identifier and the device reference number. When a high-risk defect type image is received for reassessment, the corresponding warning voice is broadcast according to the high-risk defect type, and the broadcast priority of the warning voice is higher than the broadcast priority of the image identification information.

9. The AI ​​review platform system for the electronics manufacturing industry according to claim 1, characterized in that, The quality data modeling and analysis module is also used for: Calculate the defect rate of each device reference number within a predetermined time window, compare the defect rate with a preset reference number warning threshold, and when the defect rate exceeds the reference number warning threshold, mark the device reference number as a high-frequency fault point and generate a warning message for that device reference number. In response to a query request, it retrieves and displays the historical final judgment results associated with a specific device reference number and the corresponding device image, and supports the export of reports containing data from at least one level of a multi-level data model and adverse trend analysis data.

10. A quality management method applied to an AI review platform system for the electronics manufacturing industry as described in any one of claims 1 to 9, characterized in that, Includes the following steps: Receive image data of the device to be inspected from the automated optical inspection equipment; the image data is associated with the corresponding board identifier and the device reference number on the board. Based on multiple inference servers, image data is distributed through a load balancing strategy, and deep learning models are used to infer the images to generate a first confidence level and a second confidence level. Obtain the pass / fail confidence threshold and the fail / fail confidence threshold. If the first confidence level is greater than or equal to the pass / fail threshold, it is judged as pass / fail. If the second confidence level is greater than or equal to the fail / fail threshold, it is judged as fail / fail. Otherwise, it is pushed to the manual review queue. Images with defects belonging to a preset high-risk defect type are forcibly pushed to the manual review queue; The manual review workstation displays images in the manual review queue, provides a result inversion interface, responds to inversion commands to correct the final judgment result of the corresponding image, performs inversion operation, and simultaneously switches the color border. Establish a multi-level data model that includes board identifiers and component reference numbers, associate the final judgment results with the corresponding board identifiers and component reference numbers, aggregate and statistically analyze specific component reference numbers, and generate defect trend analysis data.

Images