LLM model-based defect detection system

Abstract

The invention relates to an LLM model-based defect detection system, which comprises a visual detection unit and an intelligent debugging unit, and is characterized in that the visual detection unit is provided with a plurality of APIs (Application Program Interface); the intelligent debugging unit comprises an interactive interface and a processor, and the processor comprises an intelligent debugging process arrangement module, a debugging workflow analysis module, a debugging workflow parameter optimization module and a debugging workflow deployment module; a user inputs a demand task through the interactive interface, and the intelligent debugging process arrangement module analyzes the demand task and generates corresponding debugging workflow information; the debugging workflow analysis module is used for analyzing the workflow data and generating an interface sequence; the debugging workflow parameter optimization module is used for carrying out optimization search on various parameters in the interface sequence; and the debugging workflow deployment module is used for generating a dynamic library and a configuration parameter document based on the optimized parameters so as to realize online operation of the debugging workflow. According to the invention, the defect detection process can be intelligently debugged, and manual participation in the debugging process is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a defect detection system based on an LLM model. Background Art

[0002] At present, the defect detection systems in the automation industry first use cameras to image defective objects, and then use a series of algorithm processing processes to detect and control the required defects. However, the current visual inspection systems are very complex, involving knowledge in optics, structure, algorithm, software, etc. The threshold for use is very high and it is difficult to learn. In particular, the card control changes for some defects involve imaging and algorithm processes, parameters, and card control logic adjustments. All of these require personnel familiar with multi-functional modules to participate in the adjustment, which results in a lot of personnel communication and debugging costs. The specific problems are as follows: (1) During the production process, manufacturers may need to adjust imaging and card control parameters due to changes in demand and process. During these adjustments, engineers from multiple trades, such as optical engineers and algorithm engineers, are required to participate in the process to complete the update from imaging to algorithm flow, which usually requires a large development cycle and development workload; (2) During normal production, manufacturers may need to add, control and delete various defects. They need to submit their requirements to the after-sales personnel of the corresponding software supplier, and the docking personnel will arrange resources to update the formula, thereby increasing the addition of defects, card control and deletion operations, etc.; (3) At present, the entire detection system is composed of multiple modules. The entire system is relatively complex. It is difficult for manufacturers' personnel to learn and master various card control adjustments, logical processing, etc. The learning cost of manufacturers' personnel to learn and master the corresponding adjustment processing logic will be greatly increased.

[0003] At present, the solutions to the above problems are mainly through: (1) arranging the corresponding after-sales support personnel to conduct development and debugging on site, and converting the development and debugging tasks into requirement documents, and feeding them back to the corresponding developers for development; (2) having the corresponding engineers conduct on-site docking, and when problems arise, the corresponding engineers will develop; (3) building a modular system, writing various debugging instructions, or SOPs, and using them after the developers have been trained and learned; (4) using AI learning to achieve card control, but currently AI training is usually achieved using classification, semantic and instance segmentation, target detection and other technologies. For extremely small defects, color anomalies, etc., which require the processing of multiple traditional algorithm operators, the corresponding algorithm engineers still need to adjust the card control logic and parameters, and the card control of some defects will involve imaging and algorithm joint debugging, which is currently a difficult point for AI to handle. In all of the above situations, a dedicated person is required to handle the debugging tasks and convert the corresponding tasks into development requirements, so as to coordinate the corresponding developers and resources for development. Overall, there is a problem of low efficiency. Summary of the invention

[0004] The present invention provides a defect detection system based on the LLM model to solve the above technical problems.

[0005] In order to solve the above technical problems, the present invention provides a defect detection system based on the LLM model, including a visual detection unit and an intelligent debugging unit.

[0006] The visual inspection unit has multiple API interfaces, which are connected to at least a camera, an alarm module, a database, a communication module, and a defect management module, and support parameter configuration corresponding to each module;

[0007] The intelligent debugging unit includes an interactive interface and a processor, wherein the processor includes an intelligent debugging process arrangement module, a debugging workflow analysis module, a debugging workflow parameter optimization module, and a debugging workflow deployment module; the intelligent debugging process arrangement module is obtained by training the LLM model;

[0008] The user inputs the required task through the interactive interface, and the intelligent debugging process arrangement module parses the required task and generates corresponding debugging workflow information, wherein the debugging workflow information includes a workflow description document and workflow data, wherein the workflow description document is used for display and interaction on the interactive interface, and the workflow data is the debugging instruction of the entire workflow;

[0009] The debugging workflow parsing module is used to parse the workflow data and generate an interface sequence consisting of interfaces of the visual inspection unit;

[0010] The debugging workflow parameter optimization module is used to optimize the search for various parameters in the interface sequence;

[0011] The debugging workflow deployment module is used to generate a dynamic library and configuration parameter documents based on the optimized parameters. The dynamic library and configuration parameter documents are imported through the visual detection unit to implement the online operation of the debugging workflow.

[0012] Preferably, the parameter configuration of the camera includes at least the camera serial number and identification number; the pixel size related to the imaging parameters; the camera trigger mode, dark field, gamma, frame rate related to the trigger; the trigger source, exposure time, trigger model related to the imaging data acquisition; the gain related to the white balance; and support the addition, deletion, modification and saving of the camera.

[0013] Preferably, the parameter configuration of the defect management module includes: adding, modifying and deleting defect detection rules, and supporting the configuration of defect detection signals, priorities and card control parameters.

[0014] Preferably, the training data of the LLM model at least includes: an introduction to each module of the visual inspection unit; the operating procedures of the software; general knowledge of the corresponding field; and software debugging log record documents.

[0015] Preferably, the required task is input in the form of voice or text.

[0016] Preferably, the debugging workflow information is divided into two types according to whether manual intervention is required: one is a debugging process that requires manual intervention, in which the workflow description document lists the content, operation steps, evaluation methods, and result storage format that require manual participation, and is interactively displayed on the interface. After manual intervention is processed, the debugging workflow analysis module is entered; the other is a debugging process that does not require manual intervention, in which the program directly enters the debugging workflow analysis module after user confirmation.

[0017] Preferably, for debugging processes that require manual intervention, the manual intervention results are also checked. If the check fails, the failure information is returned to the intelligent debugging process orchestration module, and corresponding prompt information is generated and displayed on the interactive interface for manual adjustment until the analysis requirements of the debugging workflow analysis module are met.

[0018] Preferably, whether to enter the debugging workflow parameter optimization module is determined according to whether parameter search is required. If necessary, the debugging workflow parameter optimization module is entered to perform parameter optimization; if not necessary, the debugging workflow deployment module is directly entered.

[0019] Preferably, the debugging workflow parameter optimization module uses any one of gradient descent, grid search, Bayesian optimization or particle optimization algorithm to perform parameter optimization.

[0020] Preferably, the debugging workflow parameter optimization module uses a grid method to search for optimal parameters grid by grid within each parameter range with a step size of 0.01 for the defects to be controlled.

[0021] Compared with the prior art, the defect detection system based on the LLM model provided by the present invention has the following advantages:

[0022] 1. The present invention uses various data document information in the visual inspection unit detection process to build an intelligent inspection system that can respond to user debugging requirements, which can significantly reduce the use threshold requirements and reduce the manpower and resource consumption of various debugging, optimization and other task processing processes during the use of the visual inspection unit;

[0023] 2. The present invention uses the LLM model learned from the debugging data to achieve intelligent answers to common demand questions on the one hand, and on the other hand, it can also achieve intelligent adjustment for imaging adjustment and algorithm formula adjustment of demand tasks, thereby solving the problems of high cost and low efficiency in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A block diagram of a defect detection system based on an LLM model in a specific embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a response of an intelligent debugging unit to an input in a specific embodiment of the present invention;

[0026] Figure 3 The figure is a schematic diagram of the processing flow of the intelligent debugging unit in a specific implementation manner of the present invention. DETAILED DESCRIPTION

[0027] In order to describe the technical solution of the above invention in more detail, specific embodiments are listed below to demonstrate the technical effects; it should be emphasized that these embodiments are used to illustrate the present invention but not to limit the scope of the present invention.

[0028] The defect detection system based on the LLM model provided by the present invention is as follows: Figures 1 to 3 As shown, it includes a visual inspection unit and an intelligent debugging unit, and the intelligent debugging unit is used to realize intelligent adjustment of the visual inspection unit, wherein:

[0029] The visual inspection unit has multiple API interfaces (Application Programming Interface), which are connected to at least a camera, an alarm module, a database, a communication module, and a defect management module, and support parameter configuration corresponding to each module.

[0030] The intelligent debugging unit includes an interactive interface (UI interface) and a processor, wherein the interactive interface serves as a front end and the processor serves as a back-end processing software. The processor includes an intelligent debugging process orchestration module, a debugging workflow parsing module, a debugging workflow parameter optimization module, and a debugging workflow deployment module. The intelligent debugging process orchestration module is the core of the entire processing flow. On the one hand, it is responsible for responding to customer needs and arranging corresponding debugging workflows. On the other hand, it is responsible for all error debugging during the entire program (or process) processing and providing corresponding prompt information. The intelligent debugging process orchestration module can be obtained by training the LLM model.

[0031] The user inputs the required task through the interactive interface, and the intelligent debugging process orchestration module parses the required task and generates corresponding debugging workflow information. The debugging workflow information includes a workflow description document and workflow data. The workflow description document is used to display and interact on the interactive interface, and the workflow data is the debugging instructions for the entire workflow.

[0032] The debugging workflow parsing module is used to parse the workflow data and generate an interface sequence consisting of interfaces of the visual inspection unit.

[0033] The debugging workflow parameter optimization module is used to perform an optimization search on various parameters in the interface sequence.

[0034] The debugging workflow deployment module is used to generate a dynamic library and configuration parameter documents based on the optimized parameters. The dynamic library and configuration parameter documents are imported through the visual detection unit to implement the online operation of the debugging workflow.

[0035] The present invention uses a large language model learned through debugging data to achieve intelligent answers to common demand questions, and can also achieve intelligent adjustment for imaging adjustment and algorithm formula adjustment of demand tasks, thereby solving the problems of high learning cost, resource and manpower consumption; the present invention also uses various data document information in the detection process of the visual detection unit to build a set of intelligent detection system that can respond to user debugging task requirements, which can significantly lower the usage threshold requirements and reduce the manpower and resource consumption of various debugging, optimization and other task processing processes during the use of the visual detection unit.

[0036] In some embodiments, the visual detection unit implements the following functions: (1) configuration of visual imaging parameters, including camera serial number, identification number, etc., imaging parameter-related pixel size, height, width, etc., trigger-related including camera trigger mode, dark field, gamma, frame rate, etc., imaging data acquisition-related trigger source, exposure time, trigger model, etc., white balance-related including various gains, etc., support for adding, deleting, modifying, saving, etc. of cameras; (2) support for setting alarm information, such as alarm mode, alarm rule, alarm type, etc., and also support for adding and modifying alarm information; (3) support for database configuration settings, including database type, address, user name, password, configuration parameters, etc.; (4) support for configuration of communication module, such as communication mode, data, command protocol, etc.; (5) support for configuration of defect management, including adding, modifying and deleting defect detection rules, support for configuration of defect detection signals, priorities and card control parameters, etc.; (6) support for interface configuration of setting requirements of other parameters, such as debugging settings, image storage settings, team settings, etc.

[0037] In addition, during the development process of the first step, for each module and interface in the first step, a detailed description and explanation document is written to clarify the effect, scope, usage parameters, parameter description and other documents of each interface. This document is used for the training of LLM (Large Language Model) in the subsequent intelligent debugging unit. Specifically, in one embodiment, the coating software is used as the basic software, and corresponding API interfaces are developed for all the above functions in the coating software, and the interfaces are encapsulated in a specified programming language for subsequent use.

[0038] In some embodiments, please refer to Figure 2 and Figure 3 The required tasks can be input in the form of voice or text, reducing the manual participation in the entire debugging process, shortening the development and debugging workload and the online cycle of debugging tasks.

[0039] The specific debugging process is as follows: The front-end UI interface is available for user input and output, and is used to input customer requirements and display the processed results. The back-end processing software is based on the intelligent debugging process arrangement module, which uses the large language model pre-trained by LLM. It uses a large amount of data learning to master the response to common demand tasks. Specific demand tasks include imaging parameter adjustment, card control algorithm operator combination and parameter adjustment, card control process change, defect increase and decrease, and other operations of automatic response and operation workflow, as well as program execution errors, debugging information, etc. The intelligent debugging process arrangement module generates corresponding debugging workflow information based on demand. The debugging workflow information includes workflow description documents and workflow data, wherein the workflow description documents are used for interface display and interaction, and the workflow data defines the entire processing workflow in the form of a combination of debugging instructions, including the corresponding parameter range and step size, etc.

[0040] The program is divided into two situations according to whether it needs manual intervention: one is the debugging process that needs manual intervention; the other is the debugging process that does not require manual intervention; for the debugging process that needs manual intervention, the workflow description document will list the content, operation steps, evaluation methods, and result storage forms that require manual participation, and display them interactively on the interface for manual intervention, and then enter the subsequent debugging workflow parsing module. The debugging workflow parsing module will check the results of manual intervention. If the check fails, it will enter the intelligent debugging process arrangement module with the failure information, and generate corresponding prompt information, which will be displayed on the UI interface for manual adjustment until the parsing requirements of the debugging workflow parsing module are met, and then determine whether to enter the debugging workflow parameter optimization module based on whether parameter search is required. For the debugging process that does not require manual intervention, the program will directly enter the debugging workflow parsing module after the user confirms. The module will parse the generated debugging workflow information. If an error occurs during the parsing process, the error information will be returned to the intelligent debugging process arrangement module for debugging. After passing, it will determine whether to enter the debugging workflow parameter optimization module based on whether parameter search is required. After the debugging workflow information is processed in the debugging workflow parsing module, if it is determined that parameter search is required, it will enter the debugging workflow parameter optimization module, which will search for parameters in the compiled workflow data. If there is any error during execution, the error information will be returned to the intelligent debugging process orchestration module for debugging. After the correct search, it will enter the subsequent debugging workflow deployment module for compilation and processing, generate the corresponding dynamic library and configuration file, and import the software to restart and go online; if parameter search is not required, it will directly enter the final debugging workflow deployment module, and the corresponding dynamic library and parameter configuration file will be generated after compilation by this module. Any errors that occur during the process will also be returned to the intelligent debugging process orchestration module for debugging with error information until the compilation is successful and the dynamic library and configuration file are generated.

[0041] In some embodiments, the training process of the LLM model can use a custom GPT model, such as GPT2, GPT3, or can use an open source commercial large language model fine-tuned, such as ChatGPT, ChatGPT2, ChatGPT3, LLAMA2, LLAMA3, LLAMA3.1, MOSHI, etc. During the entire custom training process, the training data of the LLM model at least includes: an introduction to each module of the visual inspection unit, including corresponding API interface calls, parameter settings, expected results and other information; the operating procedures of the software, including operation information such as imaging, defect card control algorithms, processes, parameter setting information, etc., and each process is explained with corresponding API calls, etc.; general knowledge in the corresponding field, taking the photovoltaic solar cell field as an example, can include books, papers and other document information on solar cell production and technology; and software debugging log record documents, which are documents that continuously collect customer needs and various processing and debugging processes during the actual visual inspection unit inspection in the past, and describe the response to each demand task, processing steps, parameter settings, final results and performance and other information. By using the training of the above documents, the intelligent debugging process arrangement module based on the LLM model can respond to various debugging demand tasks and generate corresponding debugging workflow documents for debugging demand tasks. The debugging instruction part of the debugging workflow document clearly defines the input of the debugging process; the input and output of each calling interface; the execution order; and the output of the entire debugging process. It is a workflow instruction file that specifies input and output, and can generate a complete workflow during the compilation process. The intelligent debugging process arrangement module can also analyze and debug all subsequent module execution error information.

[0042] In some embodiments, Python can be used as the debugging instruction code. The user proposes a requirement to "add chamfers and missing corners" task. After being processed by the intelligent debugging process orchestration module, a description document including the operation process and a python code are generated. The description document is displayed on the UI interface to prompt the customer for the operations and steps required, while the python code is used for the underlying process compilation and debugging. Each python function corresponds to a specific debugging operation and specifies the input and output of the entire Python program. The configuration parameters display the step size and range of each parameter in the form of a dictionary. The Python code is parsed by the subsequent workflow parsing module to generate the workflow for the parameters and interface.

[0043] In some embodiments, the debugging workflow parsing module is used to process the debugging workflow information generated by the intelligent debugging process arrangement module, wherein the workflow data can be compiled into an executable interface code. During the execution and parsing process, if the debugging workflow parsing module encounters an error, it will attach the error information and return to the intelligent debugging process arrangement module for error checking, and return the correct debugging workflow information content until an executable interface code is generated, which can be used for the subsequent debugging workflow parameter optimization module and debugging workflow deployment module work, and the workflow description document is attached as historical information to the corresponding generated interface code, and is input into the debugging work of the intelligent debugging process arrangement module after a subsequent error occurs. In one embodiment, this step is a Python language parser, which can parse the generated debugging workflow information and generate an interface sequence consisting of a visual inspection software interface, which is used for subsequent parameter search and compilation work.

[0044] In some embodiments, the debugging workflow parameter optimization module uses any one of gradient descent, grid search method, Bayesian optimization or particle optimization algorithm to optimize parameters. If the parsed debugging workflow information does not contain parameters, the program will not enter the module, and will directly enter the final debugging workflow deployment module for compilation after the debugging workflow information is parsed by the debugging workflow parsing module, generate corresponding dynamic libraries and configuration files, and load the debugging optimization work by the software; if the parsed debugging workflow information contains parameters, the optimization algorithm will be used in this module to optimize the parameters in the entire debugging workflow, and the optimization results will be passed to the final debugging workflow deployment module in the form of parameters and compiled, and the corresponding dynamic libraries and parameter files will be generated for the software to load and complete the debugging optimization work. In one embodiment, the debugging workflow parameter optimization module uses the grid method to search for the optimal parameters grid by grid within each parameter range with a step size of 0.01 for the defects to be controlled, and realizes the automatic setting of gain parameters, exposure parameters, threshold segmentation parameters, open operation parameters, closed operation, and intersection parameters, and realizes the detection workflow of the missing corner on the input solar cell.

[0045] In some embodiments, the debugging workflow deployment module is used to process the parsed debugging workflow information, such as the attached parameter search results, and automatically merge the optimized parameters to generate the corresponding dynamic library and configuration parameter documents; the dynamic library and configuration parameter documents can be imported through the visual detection unit to realize the online operation of the debugging workflow. In one embodiment, this step can be composed of a C++ compiler and a Python code. The C++ compiler will compile the parsed debugging workflow information into a dynamic library and a configuration file. The Python code is used to implement the entire processing process and information transmission, including dynamic library and configuration file updates, software restart and other operations. After the C++ compiler compiles successfully, the Python code will first close the software, use the latest generated dynamic library and parameter documents to overwrite the existing library and parameters of the software, and then import the new parameters and start the software. If this step fails, the original dynamic library and configuration file are retained, and the error information is returned to the intelligent debugging process arrangement module for debugging.

[0046] Below, the debugging process is introduced in detail using the task requirement of adding "chamfer and chipped corner defects" proposed by the customer as an example.

[0047] After the task requirement is input into the system, the intelligent debugging process arrangement module responds and processes it, and generates the following workflow description document:

[0048] 1. Respond and set the task number to 23416.

[0049] 2. The processing of this task includes five steps, of which the first step requires human participation. The following are the contents of each processing step:

[0050] Step 1: Defect image collection: This step requires manual assistance to collect images containing chamfer defects and normal chamfer images, and place them in the "workdirs / 23416 / images / chamfer and chipped corners" and "workdirs / 23416 / images / OK" directories. During the collection process, the following checks need to be carefully performed on the collected images: (1) Ensure that each chamfer and chipped corner cell image has a chipped corner defect and the defect is in the chamfer area, and that each OK cell has no defects; (2) Try to make the number of chamfer and chipped corner images in the four chamfer areas as equal as possible, and use images from different batches for OK images; (3) If there are different chamfer sizes, collect at least 100 images for each different chamfer size, and place them in "workdirs / 23416 / images / chamfer and chipped corners / size1", "workdirs / 23416 / images / chamfer and chipped corners / size2", ..., and create several directories for each chamfer size. Each directory should contain at least 100 chamfered and chipped images of the same size; (4) If there are other differences, create directories as shown in (3) and place them in corresponding folders, such as "workdirs / 23416 / images / chamfered and chipped / difference 1", "workdirs / 23416 / images / chamfered and chipped / difference 2", "difference 1" and "difference 2" can be set arbitrarily. If there are several difference images, create several difference type folders, but make sure that the directory must contain OK images, "workdirs / 23416 / images / chamfered and chipped / OK"; (5) If the number of images is less than 100, 50 of each type can also be used. Reducing the number of images may lead to worse results; (6) When collecting images, try to use bmp or png. Images in other formats such as jpeg and jpg can also be processed, but will result in longer edges due to image compression; (where workdirs is the working directory pre-set by the system and is automatically created after the system is started).

[0051] Step 2: Defective image inspection: This step will check the collected images to ensure that the collected images meet the requirements. If they do not meet the requirements, it may be necessary to collect the images again. However, when the re-collection process is required, matters that need attention will be given.

[0052] Step 3: Automated processing: This step uses the collected images to automatically adjust the recipe and parameters, and automatically updates them to the software. After the update is completed, the interface will give a prompt of "Software has updated 23416 tasks". After seeing this prompt, you can use the actual chamfered and chipped corner images to evaluate the effect after debugging.

[0053] Step 4: Effect evaluation: This step can choose offline test or online test. The default is online test. No operation is required, and the film can be run directly. For offline test mode, you need to check the offline mode in the system settings menu and restart manually to turn it on, or you can enter "Set offline test mode" in the interface, and the system will automatically switch the software to offline mode. Then you can evaluate the accuracy of chamfer and corner detection according to the normal evaluation process. If the effect is not good after the evaluation test, you can use one of the following two methods to optimize: one is to continue the current task and then optimize it. You can collect more defects and OKs to add to the task image directory, so as to optimize the algorithm (the added defect / OK images need to be placed in the corresponding defect folder). On the other hand, you can also not add defect images, so that only the parameters will be re-optimized. After processing the above content, directly enter "debug 23416" to execute automatically.

[0054] Step 5: Please confirm whether the defect image collection in the first step has been completed. After confirmation, you can enter 'Continue 23416' to enter the next step of the operation process.

[0055] The entire processing flow is as follows: After the user enters the task debugging requirements in the corresponding UI interactive interface, the intelligent debugging software automatically assigns the corresponding task number and creates a working folder corresponding to the task number in the working directory workdirs (the working directory preset by the system, automatically created after the system is started). The corresponding task requirement information will flow into the intelligent debugging process arrangement module to analyze the task requirements and arrange the process. When the arrangement is completed, the debugging workflow information will be generated, which includes the workflow description document and workflow data. The workflow description document is used for display, interaction and debugging in the interface, and the workflow data is used for processing in the background. When the user confirms the completion of the processing of the part that requires manual intervention through UI interaction, the program will automatically enter the debugging workflow analysis module. This module will execute the instructions defined in the workflow data one by one. If the execution error is returned to the intelligent debugging process arrangement module for error checking. After the entire analysis work is completed, it will be processed according to whether it contains the parameters to be searched. If parameter search is required, it will enter the debugging workflow parameter optimization module. This module will search for the parameters in the compiled debugging workflow information. After the search is completed, it will enter the subsequent debugging workflow deployment module for compilation and system online operation.

[0056] In summary, the defect detection system based on the LLM model provided by the present invention constructs an intelligent debugging system by using the LLM pre-trained large language model trained based on a large amount of industry knowledge and software debugging data as the core. The system can perform intelligent updates on defects and routine operations during detection, thereby simplifying the formula update and debugging process of the visual inspection system; the system uses the latest AI technology to realize intelligent adjustment methods, which is expected to improve the iteration cycle and resource expenditure during product updates and debugging.

[0057] Obviously, those skilled in the art can make various changes and modifications to the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A defect detection system based on the LLM model, characterized in that: Including visual inspection unit and intelligent debugging unit, The visual inspection unit has multiple API interfaces, which are connected to at least a camera, an alarm module, a database, a communication module, and a defect management module, and support parameter configuration corresponding to each module; The intelligent debugging unit includes an interactive interface and a processor, wherein the processor includes an intelligent debugging process arrangement module, a debugging workflow analysis module, a debugging workflow parameter optimization module, and a debugging workflow deployment module; the intelligent debugging process arrangement module is obtained by training the LLM model; The user inputs the required task through the interactive interface, and the intelligent debugging process arrangement module parses the required task and generates corresponding debugging workflow information, wherein the debugging workflow information includes a workflow description document and workflow data, wherein the workflow description document is used for display and interaction on the interactive interface, and the workflow data is the debugging instruction of the entire workflow; The debugging workflow parsing module is used to parse the workflow data and generate an interface sequence consisting of interfaces of the visual inspection unit; The debugging workflow parameter optimization module is used to optimize the search for each parameter in the interface sequence; The debugging workflow deployment module is used to generate a dynamic library and configuration parameter documents based on the optimized parameters. The dynamic library and configuration parameter documents are imported through the visual detection unit to implement the online operation of the debugging workflow.

2. The defect detection system based on the LLM model according to claim 1, characterized in that: The parameter configuration of the camera includes at least the camera serial number and identification number; the pixel size related to the imaging parameters; the camera trigger mode, dark field, gamma, frame rate related to the trigger; the trigger source, exposure time, trigger model related to the imaging data acquisition; the gain related to the white balance; and supports the addition, deletion, modification and saving of the camera.

3. The defect detection system based on the LLM model according to claim 1, characterized in that: The parameter configuration of the defect management module includes: adding, modifying and deleting defect detection rules, and supporting the configuration of defect detection signals, priorities and card control parameters.

4. The defect detection system based on the LLM model according to claim 1, characterized in that: The training data of the LLM model at least includes: an introduction to each module of the visual inspection unit; the operating procedures of the software; general knowledge of the corresponding field; and a software debugging log record document.

5. The defect detection system based on the LLM model according to claim 1, characterized in that: The required task is input in the form of voice or text.

6. The defect detection system based on the LLM model according to claim 1, characterized in that: According to whether manual intervention is required, the debugging workflow information is divided into two types: one is the debugging process that requires manual intervention, and the workflow description document lists the content, operation steps, evaluation method, and result storage form that require manual participation, and is interactively displayed on the interface. After manual intervention, the debugging workflow analysis module is entered; The other is a debugging process that does not require human intervention. After the user confirms, the program directly enters the debugging workflow analysis module.

7. The defect detection system based on the LLM model according to claim 6, characterized in that: For debugging processes that require manual intervention, the manual intervention results are also checked. If the check fails, the failure information is returned to the intelligent debugging process orchestration module, and corresponding prompt information is generated and displayed on the interactive interface for manual adjustment until the analysis requirements of the debugging workflow analysis module are met.

8. The defect detection system based on the LLM model according to claim 7, characterized in that: Determine whether to enter the debugging workflow parameter optimization module according to whether parameter search is required, and if necessary, enter the debugging workflow parameter optimization module to perform parameter optimization; If not required, directly enter the debugging workflow deployment module.

9. The defect detection system based on the LLM model as claimed in claim 8, characterized in that: The debugging workflow parameter optimization module uses any one of gradient descent, grid search, Bayesian optimization or particle optimization algorithms to optimize parameters.

10. The defect detection system based on the LLM model according to claim 9, characterized in that: The debugging workflow parameter optimization module uses a grid method to search for optimal parameters grid by grid within each parameter range with a step size of 0.01 for the defects to be controlled.

Images