Defect positioning and detecting method and system for DIP integrated circuit board and medium

Abstract

The invention discloses a DIP integrated circuit board defect positioning and detecting method and system and a medium. The method comprises the steps that an image of a DIP integrated circuit board is collected through edge end equipment; the collected image of the DIP integrated circuit board is preprocessed; inputting the preprocessed image into a deep learning model issued by a cloud algorithm platform for defect detection and element positioning to obtain defect detection and positioning data; and uploading defect detection and positioning data to a cloud algorithm platform, and updating the deep learning model by the cloud algorithm platform. According to the invention, the efficiency and the precision of defect detection are improved, the production cost is reduced, the requirements of industrial production for real-time performance, high precision and self-adaptability are met, and a solid technical support is provided for realizing the intelligentization and automation of production.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and industrial automation detection technology, and in particular to a DIP (dual in-line package) integrated circuit board defect positioning and detection method, system and medium based on AI (artificial intelligence) and cloud-edge-end collaborative technology. Background Art

[0002] At present, the solutions for locating and detecting defects on DIP integrated circuit boards are end-to-end visual solutions, which have the following defects:

[0003] 1. The computing cost is too high: The deep learning method is used to process DIP integrated circuit board images, which has the following characteristics:

[0004] First, in terms of spatial scale, the area of ​​the DIP integrated circuit board is very large, requiring a high-resolution industrial camera to complete image acquisition;

[0005] Secondly, there are many components on the circuit board, and a large number of samples are needed to build a data set for model training. Therefore, when designing the algorithm, a deep neural network with a large number of parameters must be used to meet the accuracy requirements for defect location and detection of DIP integrated circuit boards in actual production as much as possible. In this scenario, high-resolution industrial cameras and deep models with a large number of parameters have high requirements for hardware performance, resulting in a surge in computing power costs. In the traditional solution, each set of equipment runs and trains independently, cannot share computing power, and has low resource utilization.

[0006] 2. Data island problem: In a typical end-to-end industrial vision solution, the training data of the model comes from the preset application scenario and its known target objects. This method of using prior knowledge to process objects in real scenes in the real world is often ineffective. In order to obtain the best detection effect in a specific scene (such as a batch of integrated circuit boards on a production line in a factory), a large number of real samples are required to iteratively train the deep neural network to obtain a network model that is infinitely close to 100% correct. However, in the traditional solution, the model's processing of the image is completed locally, and the effective data in production cannot be shared among various systems in a timely manner, making it impossible for all systems to obtain "effective knowledge" and retrain the model in a timely manner, which in turn affects the overall production effect and progress. Summary of the invention

[0007] The main purpose of the present invention is to propose a DIP integrated circuit board defect positioning and detection method, system and medium, aiming to solve the efficiency and accuracy problems in DIP (dual in-line package) integrated circuit board component positioning and defect detection, improve positioning detection accuracy, effectively enhance flexible production capacity and reduce production costs.

[0008] To achieve the above object, the present invention provides a DIP integrated circuit board defect positioning and detection method, characterized in that the DIP integrated circuit board defect positioning and detection method is applied to a DIP integrated circuit board defect positioning and detection system, the DIP integrated circuit board defect positioning and detection system includes an edge device and a cloud algorithm platform, and the method includes the following steps:

[0009] Step S100, collecting an image of a DIP integrated circuit board through the edge device;

[0010] Step S200, preprocessing the collected image of the DIP integrated circuit board;

[0011] Step S300, inputting the preprocessed image into the deep learning model issued by the cloud algorithm platform to perform defect detection and component positioning, and obtaining defect detection and positioning data;

[0012] Step S400, uploading the defect detection and positioning data to the cloud algorithm platform, and the cloud algorithm platform updates the deep learning model.

[0013] A further technical solution of the present invention is that the cloud algorithm platform includes a data management unit, a model training and verification unit, and a computing resource scheduling unit, and the step S400 includes:

[0014] Step S410, after the cloud algorithm platform receives the defect detection and location data, a knowledge graph database is constructed and updated through a data management unit, wherein the knowledge graph database includes component features, defect types, and production parameter information;

[0015] Step S420, based on the updated knowledge graph database, the deep learning model is trained and verified by the model training and verification unit;

[0016] The step S410 includes: cleaning, deduplication and standardization of the received defect detection and location data through the data management system, and integrating the data into the knowledge graph database, and combining prior knowledge and expert experience to perform semantic association and logical reasoning on the data;

[0017] The step S420 includes: the model training and verification unit uses a distributed training framework and a deep learning algorithm to perform neural network model training and verification, wherein the model training strategy includes: adaptive model training, transfer learning, and continuous learning online update.

[0018] A further technical solution of the present invention is that step S420 includes:

[0019] Step S4201, the model training and verification unit calls the computing resource scheduling algorithm to dynamically select a resource configuration scheme based on the real-time requirements of the defect detection task and the heterogeneous resource characteristics of the system to achieve multi-task parallel scheduling;

[0020] The step S4201 includes: calculating the running time of the defect detection task under different resource configurations through the resource-time model, and analyzing the balance between computing and communication; evaluating the running efficiency of different resource schemes through the resource-performance model, and selecting the scheme with the best performance on the premise of meeting the real-time requirements of the task; reducing resource fragmentation and improving the overall utilization of the system through real-time task scheduling and resource migration models.

[0021] A further technical solution of the present invention is that the step of calculating the running time of the defect detection task under different resource configurations through the resource-time model and analyzing the balance relationship between calculation and communication includes:

[0022] Calculate the total task time, where the calculation formula is:

[0023] T run =T step ×N step ×N epoch

[0024]

[0025] Among them, T run T is the total task running time; step N is the running time of each processing step, including computation and communication time; step is the number of batches required for each round of processing; S dataset is the dataset size; S batch is the batch size; N epoch is the number of processing rounds; N device is the number of devices;

[0026] Calculate the running time of the step T step , where the calculation formula is:

[0027] T step =T cal +T comm

[0028] Among them, T cal is the computation time, which means the time required for the model to perform forward and backward propagation on the device; T comm is the communication time, which reflects the communication overhead of multiple devices during the data synchronization phase;

[0029] Calculate the communication time T comm , where the calculation formula is:

[0030]

[0031] Among them, N param is the total amount of model parameters; B is the communication bandwidth between devices; N device is the number of devices;

[0032] The real-time requirements of the calculation task are as follows:

[0033] T dl =T arr +T exp

[0034]

[0035] Among them, T dl is the task deadline; T arr is the task arrival time; T exp is the expected running time of the task; α∈{0.5,1.0,1.5}, represents the task priority, and high-priority tasks correspond to shorter deadlines; It is the running time of the task on a single device.

[0036] A further technical solution of the present invention is that the step of evaluating the operating efficiency of different resource solutions through the resource-performance model and selecting the solution with the best performance under the premise of meeting the real-time requirements of the task includes:

[0037] Resource solution performance evaluation, where the calculation formula is:

[0038]

[0039] in, is the resource solution performance indicator; T dl is the task deadline; T end is the actual end time of the task; N device The number of devices used in the resource plan.

[0040] A further technical solution of the present invention is that the step of reducing resource fragmentation and improving the overall utilization of the system through real-time task scheduling and resource migration model includes dynamic task adjustment and resource allocation, and resource migration mechanism;

[0041] The steps of dynamic task adjustment and resource allocation include:

[0042] Resource detection: obtain the number of idle devices of each node in the system;

[0043] Resource plan generation: Generate a single-node or cross-node resource plan for each task based on the current resource status;

[0044] Run time calculation: Calculate the task run time under each resource solution according to the total task time calculation formula, step run time calculation formula, and communication time calculation formula;

[0045] Solution screening and sorting: Screen out solutions that meet real-time requirements and sort them according to the resource solution performance evaluation formula;

[0046] Task allocation: select the resource solution with the best performance to schedule tasks and allocate computing resources;

[0047] The steps of the resource migration mechanism include:

[0048] When node resources are insufficient, some tasks are suspended and migrated to nodes with more resources to reallocate resources and reduce device idleness.

[0049] A further technical solution of the present invention is that after step S400, the following steps are further included:

[0050] The updated deep learning model is sent to the edge device through the cloud algorithm platform, and the edge device replaces the old deep learning model.

[0051] A further technical solution of the present invention is that after step S300, the following steps are further included:

[0052] A corresponding control signal is sent to a DIP integrated circuit board production line according to the defect detection and positioning data.

[0053] To achieve the above objectives, the present invention also proposes a DIP integrated circuit board defect location and detection system, the system comprising a memory, a processor, and a DIP integrated circuit board defect location and detection program stored on the processor, and the DIP integrated circuit board defect location and detection program executes the steps of the method described above when run by the processor.

[0054] To achieve the above objectives, the present invention also proposes a computer-readable storage medium, which stores a DIP integrated circuit board defect location and detection program, and the DIP integrated circuit board defect location and detection program executes the steps of the method described above when run by a processor.

[0055] The beneficial effects of the DIP integrated circuit board defect location and detection method, system and medium of the present invention are:

[0056] The present invention makes full use of the real-time nature of edge computing and the powerful computing power of cloud computing. The professional design of edge devices and high-performance hardware ensure fast and accurate defect detection of DIP circuit boards on the production line. The cloud algorithm platform achieves continuous optimization and updating of the model through the data management strategy of the knowledge graph and advanced model training technology. In particular, the computing resource scheduling scheme implemented by the present invention significantly improves the utilization efficiency of cloud computing resources and accelerates the model training and updating process by establishing accurate mathematical models and optimization algorithms. The present invention improves the efficiency and accuracy of defect detection, reduces production costs, meets the requirements of industrial production for real-time, high precision and adaptability, and provides solid technical support for realizing intelligent and automated production. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0058] Figure 1 It is a flow chart of a preferred embodiment of the DIP integrated circuit board defect location and detection method of the present invention;

[0059] Figure 2 It is a system overall architecture diagram of the DIP integrated circuit board defect location and detection system of the present invention;

[0060] Figure 3 This is a timing diagram of cloud-edge-end collaborative work.

[0061] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In order to solve the efficiency and accuracy problems in DIP (dual in-line package) integrated circuit board component positioning and defect detection, the present invention proposes a DIP integrated circuit board defect positioning and detection method based on artificial intelligence (AI) and cloud-edge collaboration. The technical solution adopted by the present invention is mainly to design a matching cloud computing resource scheduling solution, data management strategy and model update process through the powerful computing power of the cloud and the real-time response capability of the edge device, and optimize the defect detection process of the DIP integrated circuit board. Through the computing resource management mechanism, the dynamic data management mechanism, the timely response of the model update and the continuous improvement of performance, higher detection accuracy and lower production costs are achieved.

[0064] The present invention combines the cloud collaborative architecture to implement training on a cloud unified architecture platform, especially high-resolution DIP integrated circuit boards and their algorithm models. The training process has very high requirements on computing power scale and hardware resources (memory). The solution of the present invention improves the system computing power utilization and reduces hardware costs, and realizes rapid iterative updates of new models.

[0065] In order to meet the ultra-high requirement of model accuracy (generally above 99.5%) in component positioning and defect detection of DIP integrated circuit boards, the present invention adopts data intercommunication and closed-loop design to enable real-time sharing of production data, adds dynamic data management and model update mechanism, supports continuous optimization of component positioning and defect detection algorithms, and effectively improves flexible production capabilities.

[0066] For details, please refer to Figure 1 , Figure 1 It is a flow chart of a preferred embodiment of the DIP integrated circuit board defect location and detection method of the present invention.

[0067] like Figure 2 As shown, in this embodiment, the DIP integrated circuit board defect location and detection method is applied to the DIP integrated circuit board defect location and detection system. The DIP integrated circuit board defect location and detection system includes an edge device and a cloud algorithm platform. The cloud algorithm platform includes a data management unit, a model training and verification unit, and a computing resource scheduling unit.

[0068] The following first describes the DIP integrated circuit board defect location and detection system involved in the present invention.

[0069] First, the edge devices are highly professional and advanced, including image acquisition units, image processing units, and communication and data transmission units.

[0070] 1. Image acquisition unit: This unit consists of a high-resolution AI smart camera and a high-precision mechanical structure, and can operate stably in complex industrial environments. The camera uses a high-pixel sensor and advanced optical lens, combined with a programmable light source control system to ensure that high-quality images can be captured under different ambient light conditions. The mechanical structure design meets industrial standards, has high-precision positioning and synchronization control capabilities, and can be seamlessly integrated with the conveyor system of the production line to achieve high-speed and continuous image acquisition of DIP integrated circuit boards.

[0071] 2. Image processing unit: This is a hardware platform that integrates a variety of high-performance processors, including central processing units (CPUs), graphics processing units (GPUs), and neural network processing units (NPUs). The platform supports parallel computing and deep learning acceleration, and can meet the real-time processing requirements of complex algorithms. The software layer integrates self-developed and optimized traditional visual algorithm libraries and deep learning algorithm libraries. The traditional visual algorithm library provides basic image processing functions such as filtering, edge detection, and morphological processing; the deep learning algorithm library contains cutting-edge models such as convolutional neural networks (CNNs) and transformers, which are used to achieve component positioning, defect detection, and classification. Through the coordinated optimization of software and hardware, the image processing unit can complete high-precision image analysis within milliseconds, providing a reliable basis for subsequent defect determination.

[0072] 3. Communication and data transmission unit: It adopts 5G communication technology and combines common communication protocols such as TCP / IP and HTTP, with the characteristics of high speed, low latency and high reliability. The communication module supports multiple network formats to ensure that the device can maintain a stable connection in various network environments. The data transmission unit provides an efficient data compression and encryption mechanism to ensure the integrity and security of data during transmission. Through the high-speed 5G network, real-time data interaction and remote control can be achieved, providing a fast channel for model updates and parameter distribution of the cloud algorithm platform.

[0073] Secondly, the cloud algorithm platform includes data management unit, model training and verification unit, and computing resource scheduling unit.

[0074] 1. Data management unit: Using the database architecture of "knowledge graph", a multi-dimensional database covering DIP integrated circuit board component information, process flow, defect characteristics and detection experience is constructed. The database describes the relationship between different data entities in the form of nodes and edges, and realizes the semantic association and logical reasoning of data. Data management strategies include:

[0075] (1) Knowledge graph construction: The collected information such as component features, defect types, and production parameters is stored in the form of entity nodes. The nodes are connected through association relationships (such as "belongs to", "cause", and "affect") to form a rich knowledge graph.

[0076] (2) Integration of prior knowledge: Introducing industry standards, expert experience, and historical data to enrich the content of the knowledge graph and make it professional and practical.

[0077] (3) Model training and updating: Using the associations in the knowledge graph, we intelligently extract and filter the training data to generate high-quality training sets. By continuously feeding new test data and results back into the knowledge graph, we continuously update and improve the model to make it adaptive and generalizable.

[0078] (4) Data governance: Establish a data quality assessment and management mechanism to clean, deduplicate and standardize data to ensure data accuracy and consistency.

[0079] 2. Model training and verification unit: Based on the high-quality data provided by the data management unit, the distributed training framework and deep learning algorithm are used to train and verify the neural network model. The model training strategy includes:

[0080] (1) Adaptive model training: Based on the guidance of the knowledge graph database, the most relevant data is dynamically selected for training to improve the effectiveness of the model.

[0081] (2) Transfer learning: Utilize existing general models to adapt to new production environments and product types through transfer learning, thereby reducing training time and data requirements.

[0082] (3) Continuous learning and online updating: The model can learn new data in real time in the cloud and collaborate with the edge to achieve online updating and optimization of the model.

[0083] 3. Computing resource scheduling unit: This invention implements an efficient computing resource scheduling solution, uses mathematical models to deeply describe the relationship between model training and computing resources, and proposes specific scheduling and optimization strategies. Based on the theory of task scheduling and resource allocation, combined with the characteristics of deep learning models, this unit establishes an optimization model for computing resource scheduling.

[0084] Thirdly, the DIP integrated circuit board defect location and detection system is a cloud-edge-end collaborative system for DIP integrated circuit board defect detection, including the above-mentioned edge-end devices and cloud-end algorithm platform. Through the effective combination of edge computing and cloud computing, efficient data transmission, real-time model updating and rapid feedback of defect detection are achieved.

[0085] The cloud-edge-device collaboration mechanism is as follows: Figure 3 As shown:

[0086] 1. Model distribution: The cloud-trained and updated models are distributed to the edge devices through the communication unit. The edge devices can quickly apply the latest models to improve detection performance.

[0087] 2. Data feedback: Edge devices feed back new detection data and results to the cloud data management unit to support continuous learning and optimization of the model.

[0088] Please refer to Figure 1 The preferred embodiment of the DIP integrated circuit board defect location and detection method of the present invention comprises the following steps:

[0089] Step S100: collecting an image of a DIP integrated circuit board through an edge device.

[0090] In this embodiment, the edge device is installed next to the production line equipment, including a high-resolution AI smart camera, a high-precision mechanical structure, an image processing unit, and a communication unit. The production line equipment refers to the DIP integrated circuit board manufacturing equipment deployed on the factory production line.

[0091] This embodiment uses a high-resolution AI smart camera on the edge device to collect images of DIP integrated circuit boards passing through the production line in real time.

[0092] Step S200, pre-processing the collected image of the DIP integrated circuit board.

[0093] In this embodiment, the collected high-resolution images are preprocessed by an image processing unit, specifically including traditional image processing operations such as denoising, enhancement, and correction, to prepare for subsequent defect detection.

[0094] Step S300: Input the preprocessed image into the deep learning model issued by the cloud algorithm platform to perform defect detection and component positioning to obtain defect detection and positioning data.

[0095] The pre-processed images are used on edge devices to detect defects and locate components using deep learning models. Since edge devices integrate high-performance CPUs, GPUs, and NPUs, they can meet the real-time computing requirements of deep learning models.

[0096] During the inspection process, the model will identify each component in the image, locate its position on the circuit board, and determine whether there are defects. For example, it will detect whether there are leaks or continuous solder joints, and whether components are missing or shifted.

[0097] Step S400, uploading the defect detection and positioning data to the cloud algorithm platform, and the cloud algorithm platform updates the deep learning model.

[0098] The cloud algorithm platform receives the detection data from multiple edge devices and uses the data management unit to build and update the knowledge graph database, which covers information such as component characteristics, defect types, and production parameters.

[0099] In the data management unit, new data is cleaned, deduplicated, standardized, and integrated into the knowledge graph, and semantic association and logical reasoning are performed on the data using prior knowledge and expert experience.

[0100] Based on the updated knowledge graph, the model training and verification unit will start a new model training task, using technologies such as adaptive model training, transfer learning, and continuous learning to generate a more accurate deep learning model.

[0101] At the same time, the computing resource scheduling unit of the cloud algorithm platform will optimize task allocation, improve resource utilization, and accelerate the model training process based on the current computing resources and model training requirements.

[0102] After the new model training is completed, it is sent to the edge device through the 5G network. The edge device receives the new model, replaces the old model, and applies it in subsequent detection to improve detection accuracy.

[0103] Through the above technical solution, this embodiment achieves the following technical effects:

[0104] (1) Improved detection accuracy: By leveraging the powerful computing power and data resources of the cloud, we continuously optimize and update the deep learning model, bringing the accuracy of defect detection to over 99.5%.

[0105] (2) Reduced computing costs: With a cloud-edge-end collaborative architecture, edge devices only need to complete real-time detection tasks, and complex model training and updates are completed in the cloud, reducing the hardware cost of a single device.

[0106] (3) Data sharing and closed-loop optimization: Production data is uploaded to the cloud in real time to achieve data interoperability and sharing. The model can be iteratively optimized based on the latest data, improving the intelligence level of the entire system.

[0107] (4) Improved production efficiency: Real-time defect detection and feedback mechanisms enable the production line to quickly handle anomalies, reducing the time and cost of manual inspection and improving production efficiency.

[0108] In this embodiment, step S400 specifically includes:

[0109] Step S410, after the cloud algorithm platform receives the defect detection and positioning data, the data management unit constructs and updates the knowledge graph database, which includes component characteristics, defect types and production parameter information.

[0110] Wherein, step S410 specifically includes:

[0111] The received defect detection and location data are cleaned, deduplicated and standardized through the data management system and then integrated into the knowledge graph database. Prior knowledge and expert experience are combined to perform semantic association and logical reasoning on the data.

[0112] Step S420: Based on the updated knowledge graph database, the deep learning model is trained and verified through the model training and verification unit.

[0113] Step S420 specifically includes: the model training and verification unit uses a distributed training framework and a deep learning algorithm to perform neural network model training and verification, wherein the model training strategy includes: adaptive model training, transfer learning, and continuous learning online update.

[0114] Step S420 specifically includes:

[0115] In step S4201, the model training and verification unit calls the computing resource scheduling algorithm to dynamically select a resource configuration scheme based on the real-time requirements of the defect detection task and the heterogeneous resource characteristics of the system to achieve multi-task parallel scheduling.

[0116] This embodiment provides an efficient computing resource scheduling solution, which uses mathematical models to deeply describe the relationship between model training and computing resources, and proposes specific scheduling and optimization strategies. This solution is based on the theory of task scheduling and resource allocation, combined with the characteristics of deep learning models, and establishes an optimization model for computing resource scheduling.

[0117] The computing resource scheduling algorithm CRS (Computing Resource Scheduling) dynamically selects resource allocation schemes based on the real-time requirements of defect detection tasks and the heterogeneous resource characteristics of the system to achieve multi-task parallel scheduling. Its main ideas include:

[0118] 1. Resource-time model: Calculate the running time of the defect detection task under different resource configurations and analyze the balance between computing and communication.

[0119] 2. Resource-performance model: Evaluate the operating efficiency of different resource solutions and select the solution with the best performance while meeting the real-time requirements of the task.

[0120] 3. Dynamic scheduling and resource migration: Reduce resource fragmentation and improve the overall utilization of the system through real-time task scheduling and resource migration.

[0121] In this embodiment, step S4201 includes: calculating the running time of the defect detection task under different resource configurations through the resource-time model, and analyzing the balance between computing and communication; evaluating the running efficiency of different resource solutions through the resource-performance model, and selecting the solution with the best performance while meeting the real-time requirements of the task; reducing resource fragmentation and improving the overall utilization of the system through real-time task scheduling and resource migration models.

[0122] Specifically, in this embodiment, the goal of establishing the resource-time model is to measure the impact of different resource solutions on the running time of the defect detection task and balance computing and communication.

[0123] The resource-time model is used to calculate the running time of the defect detection task under different resource configurations. The steps to analyze the balance between computing and communication include:

[0124] (1) Calculation of total task time, where the calculation formula is:

[0125] T run =T step ×N step ×N epoch

[0126]

[0127] Among them, T run T is the total task running time; step N is the running time of each processing step, including computation and communication time; step is the number of batches required for each round of processing; S dataset is the dataset size; S batch is the batch size; N epoch is the number of processing rounds; N device is the number of devices.

[0128] It should be noted that: increase the number of devices N device The number of batches required for each round of processing can be reduced, thereby speeding up the defect detection process. However, the increase in the number of devices also brings additional communication overhead, and the actual running time may be limited by the communication bottleneck.

[0129] (2) Step running time T step Calculate, where the calculation formula is:

[0130] T step =T cal +T comm

[0131] Among them, T cal is the computation time, which means the time required for the model to perform forward and backward propagation on the device; T commis the communication time, which reflects the communication overhead of multiple devices during the data synchronization phase;

[0132] (3) Communication time T comm Calculate, where the calculation formula is:

[0133]

[0134] Among them, N param is the total amount of model parameters; B is the communication bandwidth between devices. The bandwidth of a single-node device is higher (such as PCIe), and the bandwidth across nodes is lower (such as Ethernet); N device is the number of devices.

[0135] (4) Calculation of task real-time requirements, where the calculation formula is:

[0136] T dl =T arr +T exp

[0137]

[0138] Among them, T dl is the task deadline; T arr is the task arrival time; T exp is the expected running time of the task; α∈{0.5,1.0,1.5}, represents the task priority, and high-priority tasks correspond to shorter deadlines; It is the running time of the task on a single device.

[0139] This embodiment constructs a real-time requirement model (Real-Time Requirement ModeL), the goal of which is to ensure that tasks are completed on time and to improve the real-time guarantee rate of tasks.

[0140] In this embodiment, a resource-performance model is constructed, and its goal is to select the optimal resource solution and improve system utilization.

[0141] The resource-performance model is used to evaluate the operating efficiency of different resource solutions. The steps to select the solution with the best performance while meeting the real-time requirements of the task include:

[0142] Resource solution performance evaluation, where the calculation formula is:

[0143]

[0144] in, is the resource solution performance indicator; T dl is the task deadline; T end is the actual end time of the task; N deviceThe number of devices used in the resource plan.

[0145] It should be noted that this embodiment gives priority to selecting a resource solution that uses the least number of devices and has the shortest running time, to ensure that the task is completed within the deadline and to reduce resource waste.

[0146] Further, in this embodiment, the steps of reducing resource fragmentation and improving the overall utilization of the system through real-time task scheduling and resource migration models include dynamic task adjustment and resource allocation, and resource migration mechanisms;

[0147] The steps of dynamic task adjustment and resource allocation include:

[0148] Resource detection: obtain the number of idle devices of each node in the system;

[0149] Resource plan generation: Generate a single-node or cross-node resource plan for each task based on the current resource status;

[0150] Run time calculation: Calculate the task run time under each resource solution according to the total task time calculation formula, step run time calculation formula, and communication time calculation formula;

[0151] Solution screening and sorting: Screen out solutions that meet real-time requirements and sort them according to the resource solution performance evaluation formula;

[0152] Task allocation: Select the resource solution with the best performance for task scheduling and allocation of computing resources.

[0153] The steps of resource migration mechanism include:

[0154] When node resources are insufficient, some tasks are suspended and migrated to nodes with more resources to reallocate resources and reduce device idleness.

[0155] The goal of the resource migration mechanism is to reduce fragmentation and improve computing resource utilization through resource migration.

[0156] Preferably, in this embodiment, after step S400, the following steps are further performed:

[0157] The updated deep learning model is sent to the edge device through the cloud algorithm platform, and the edge device replaces the old deep learning model.

[0158] Furthermore, in this embodiment, after step S300, the following steps are further included:

[0159] According to the defect detection and positioning data, the corresponding control signal is sent to the DIP integrated circuit board production line.

[0160] After obtaining the defect detection and location data, this embodiment will provide result feedback and processing. For the detected defects, the edge device will immediately send a control signal to the production line equipment, such as alarm, marking, defective product removal, etc. In this way, the problem can be handled in the shortest time possible to avoid further influx of defective products.

[0161] At the same time, the edge device will upload the detection results and related data to the cloud algorithm platform through the 5G communication network.

[0162] In order to better understand the technical solution of the present invention, the following Figures 1 to 3 The defect location and detection method of the DIP integrated circuit board of the present invention is further elaborated in detail.

[0163] In order to better illustrate the technical solution of the present invention, the present invention is described in detail below in conjunction with specific embodiments.

[0164] On the production line of an electronics manufacturing plant, a large number of DIP (dual in-line package) integrated circuit boards need to be defect-located and inspected to ensure product quality and production efficiency. Traditional inspection methods are difficult to meet the needs of modern industrial production due to high hardware costs, low computing resource utilization, and the inability to share data.

[0165] In order to solve the above problems, the present invention introduces a DIP integrated circuit board defect location and detection method based on AI (artificial intelligence) and cloud-edge collaboration. The entire system includes a cloud algorithm platform, edge devices, and production line equipment, which realizes real-time data sharing, rapid model update, and efficient defect detection.

[0166] 1. The system structure involved in the present invention is as follows:

[0167] Production line equipment: DIP integrated circuit board manufacturing equipment deployed on factory production lines.

[0168] Edge equipment: installed next to the production line, including high-resolution AI smart cameras, high-precision mechanical structures, image processing units and communication units.

[0169] Cloud algorithm platform: located in the cloud server, including data management unit, model training and verification unit, and computing resource scheduling unit.

[0170] 2. Workflow

[0171] (1) Data collection and preprocessing

[0172] The high-resolution AI smart camera on the edge device captures images of DIP integrated circuit boards passing through the production line in real time.

[0173] The collected high-resolution images are preprocessed by the image processing unit, including traditional image processing operations such as denoising, enhancement, and correction, to prepare for subsequent defect detection.

[0174] (2) Defect detection and location

[0175] The pre-processed images are used on edge devices to detect defects and locate components using deep learning models. Since edge devices integrate high-performance CPUs, GPUs, and NPUs, they can meet the real-time computing requirements of deep learning models.

[0176] During the inspection process, the model will identify each component in the image, locate its position on the circuit board, and determine whether there are defects. For example, it will detect whether there are leaks or continuous solder joints, and whether components are missing or shifted.

[0177] (3) Result feedback and processing

[0178] For detected defects, the edge device will immediately send control signals to the production line equipment, such as alarm, marking, defective product removal, etc. In this way, the problem can be handled in the shortest time possible to avoid the further influx of defective products.

[0179] At the same time, the edge device will upload the detection results and related data to the cloud algorithm platform through the 5G communication network.

[0180] (4) Data management and model updating

[0181] The cloud algorithm platform receives the detection data from multiple edge devices and uses the data management unit to build and update the knowledge graph database, which covers information such as component characteristics, defect types, and production parameters.

[0182] In the data management unit, new data is cleaned, deduplicated, and standardized before being integrated into the knowledge graph. Prior knowledge and expert experience are used to perform semantic association and logical reasoning on the data.

[0183] Based on the updated knowledge graph, the model training and verification unit will start a new model training task, using technologies such as adaptive model training, transfer learning, and continuous learning to generate a more accurate deep learning model.

[0184] (5) Computing resource scheduling and model delivery

[0185] The computing resource scheduling unit of the cloud algorithm platform will optimize task allocation, improve resource utilization, and accelerate the model training process based on the current computing resources and model training requirements.

[0186] After the new model training is completed, it is sent to the edge device through the 5G network. The edge device receives the new model, replaces the old model, and applies it in subsequent detection to improve detection accuracy.

[0187] In summary, the defect location and detection method of the DIP integrated circuit board of the present invention fully utilizes the real-time nature of edge computing and the powerful computing power of cloud computing. The professional design and high-performance hardware of the edge device ensure fast and accurate defect detection of the DIP circuit board on the production line. The cloud algorithm platform realizes continuous optimization and updating of the model through the data management strategy of the knowledge graph and advanced model training technology. In particular, the computing power resource scheduling scheme implemented by the present invention significantly improves the utilization efficiency of cloud computing power resources and accelerates the model training and updating process by establishing accurate mathematical models and optimization algorithms. The present invention improves the efficiency and accuracy of defect detection, reduces production costs, meets the requirements of industrial production for real-time, high precision and adaptability, and provides solid technical support for realizing intelligent and automated production.

[0188] To achieve the above-mentioned purpose, the present invention also proposes a DIP integrated circuit board defect location and detection system, characterized in that the system includes a memory, a processor, and a DIP integrated circuit board defect location and detection program stored on the processor. When the DIP integrated circuit board defect location and detection program is run by the processor, the steps of the method described in the above embodiment are executed, which will not be repeated here.

[0189] To achieve the above objectives, the present invention also proposes a computer-readable storage medium, which stores a DIP integrated circuit board defect location and detection program. When the DIP integrated circuit board defect location and detection program is run by a processor, the steps of the method described in the above embodiment are executed, which will not be repeated here.

[0190] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A DIP integrated circuit board defect location and detection method, characterized in that: The DIP integrated circuit board defect positioning and detection method is applied to a DIP integrated circuit board defect positioning and detection system, the DIP integrated circuit board defect positioning and detection system includes an edge device and a cloud algorithm platform, and the method includes the following steps: Step S100, collecting an image of a DIP integrated circuit board through the edge device; Step S200, preprocessing the collected image of the DIP integrated circuit board; Step S300, inputting the preprocessed image into the deep learning model issued by the cloud algorithm platform to perform defect detection and component positioning, and obtaining defect detection and positioning data; Step S400, uploading the defect detection and positioning data to the cloud algorithm platform, and the cloud algorithm platform updates the deep learning model.

2. The DIP integrated circuit board defect location and detection method according to claim 1, characterized in that: The cloud algorithm platform includes a data management unit, a model training and verification unit, and a computing resource scheduling unit. Step S400 includes: Step S410, after the cloud algorithm platform receives the defect detection and location data, a knowledge graph database is constructed and updated through a data management unit, wherein the knowledge graph database includes component features, defect types, and production parameter information; Step S420, based on the updated knowledge graph database, the deep learning model is trained and verified by the model training and verification unit; The step S410 includes: cleaning, deduplication and standardization of the received defect detection and location data through the data management system, and integrating the data into the knowledge graph database, and combining prior knowledge and expert experience to perform semantic association and logical reasoning on the data; The step S420 includes: the model training and verification unit uses a distributed training framework and a deep learning algorithm to perform neural network model training and verification, wherein the model training strategy includes: adaptive model training, transfer learning, and continuous learning online update.

3. The DIP integrated circuit board defect location and detection method according to claim 2, characterized in that: The step S420 includes: Step S4201, the model training and verification unit calls the computing resource scheduling algorithm to dynamically select a resource configuration scheme based on the real-time requirements of the defect detection task and the heterogeneous resource characteristics of the system to achieve multi-task parallel scheduling; The step S4201 includes: calculating the running time of the defect detection task under different resource configurations through the resource-time model, and analyzing the balance between computing and communication; evaluating the running efficiency of different resource schemes through the resource-performance model, and selecting the scheme with the best performance on the premise of meeting the real-time requirements of the task; reducing resource fragmentation and improving the overall utilization of the system through real-time task scheduling and resource migration models.

4. The DIP integrated circuit board defect location and detection method according to claim 3, characterized in that: The step of calculating the running time of the defect detection task under different resource configurations through the resource-time model and analyzing the balance relationship between computing and communication includes: Calculate the total task time, where the calculation formula is: T run =T step ×N step ×N epoch Among them, T run T is the total task running time; step N is the running time of each processing step, including computation and communication time; step is the number of batches required for each round of processing; S dataset is the dataset size; S batch is the batch size; N epoch N is the number of processing rounds; device is the number of devices; Calculate the running time of the step T step , where the calculation formula is: T step =T cal +T comm Among them, T cal is the computation time, which means the time required for the model to perform forward and backward propagation on the device; T comm is the communication time, which reflects the communication overhead of multiple devices during the data synchronization phase; Calculate the communication time T comm , where the calculation formula is: Among them, N param is the total amount of model parameters; B is the communication bandwidth between devices; N device is the number of devices; The real-time requirements of the calculation task are as follows: T dl =T arr +T exp Among them, T dl is the task deadline; T arr is the task arrival time; T exp is the expected running time of the task; α∈{0.5,1.0,1.5}, represents the task priority, and high-priority tasks correspond to shorter deadlines; It is the running time of the task on a single device.

5. The DIP integrated circuit board defect location and detection method according to claim 3, characterized in that: The steps of evaluating the operating efficiency of different resource solutions through the resource-performance model and selecting the solution with the best performance under the premise of meeting the real-time requirements of the task include: Resource solution performance evaluation, where the calculation formula is: in, is the resource solution performance indicator; T dl is the task deadline; T end is the actual end time of the task; N device The number of devices used in the resource plan.

6. The DIP integrated circuit board defect location and detection method according to claim 3, characterized in that: The steps of reducing resource fragmentation and improving the overall utilization of the system through real-time task scheduling and resource migration models include dynamic task adjustment and resource allocation, and resource migration mechanisms; The steps of dynamic task adjustment and resource allocation include: Resource detection: obtain the number of idle devices of each node in the system; Resource plan generation: Generate a single-node or cross-node resource plan for each task based on the current resource status; Run time calculation: Calculate the task run time under each resource solution according to the total task time calculation formula, step run time calculation formula, and communication time calculation formula; Solution screening and sorting: Screen out solutions that meet real-time requirements and sort them according to the resource solution performance evaluation formula; Task allocation: select the resource solution with the best performance to schedule tasks and allocate computing resources; The steps of the resource migration mechanism include: When node resources are insufficient, some tasks are suspended and migrated to nodes with more resources to reallocate resources and reduce device idleness.

7. The DIP integrated circuit board defect location and detection method according to claim 6, characterized in that: After step S400, the following steps are further included: The updated deep learning model is sent to the edge device through the cloud algorithm platform, and the edge device replaces the old deep learning model.

8. The DIP integrated circuit board defect location and detection method according to claim 7, characterized in that: After step S300, the following steps are further included: A corresponding control signal is sent to a DIP integrated circuit board production line according to the defect detection and positioning data.

9. A DIP integrated circuit board defect location and detection system, characterized in that: The system includes a memory, a processor, and a DIP integrated circuit board defect location and detection program stored on the processor. When the DIP integrated circuit board defect location and detection program is executed by the processor, the steps of the method described in any one of claims 1 to 8 are executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a DIP integrated circuit board defect location and detection program, and the DIP integrated circuit board defect location and detection program, when executed by a processor, performs the steps of the method described in any one of claims 1 to 8.

Images