A defect detection system based on a LLM model

Abstract

The application relates to a defect detection system based on an LLM model, which comprises a visual detection unit and an intelligent debugging unit, the visual detection unit being provided with a plurality of API interfaces; the intelligent debugging unit comprises an interactive interface and a processor, the processor comprising an intelligent debugging flow arrangement module, a debugging work flow analysis module, a debugging work flow parameter optimization module and a debugging work flow deployment module; a user inputs a demand task through the interactive interface, the intelligent debugging flow arrangement module analyzes the demand task and generates corresponding debugging work flow information; the debugging work flow analysis module is used for analyzing work flow data and generating an interface sequence; the debugging work flow parameter optimization module is used for optimally searching each parameter in the interface sequence; and the debugging work flow deployment module is used for generating a dynamic library and a configuration parameter document based on the optimized parameters, thereby realizing online operation of the debugging work flow. The application can intelligently debug a defect detection process and reduce manual participation in the debugging process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, and in particular to a defect detection system based on an LLM model. BACKGROUND

[0002] At present, the defect detection system of the automation industry is first to image the defective object through a camera, and then to realize the detection and control of the required defects through a series of algorithm processing procedures. However, the current visual detection system is very complex, involving knowledge of optics, structure, algorithm, software and other aspects, and the use threshold is very high, and it is also difficult to learn, especially for the control change of part of the defects, which involves the adjustment of imaging and algorithm process, parameters, and control logic, and requires the participation of multi-functional module personnel to adjust, thereby generating a large amount of personnel communication and debugging cost. The specific problems are as follows: (1) During the production process, the production manufacturer may adjust the imaging and control parameters due to changes in demand and process, and during these adjustment processes, engineers of multiple types such as optical engineers and algorithm engineers are required to participate in the processing to complete the update from imaging to algorithm process, which usually requires a large amount of development period and development workload; (2) During the normal production process, the production manufacturer may need to add, control and delete various defects, and needs to submit the demand to the corresponding software supplier after-sales personnel, and arrange resources for formula update to add, control and delete defects; (3) The current detection system is composed of multiple modules, and the whole system is complex, and it is difficult for the production manufacturer to learn and master various control adjustments and logic processing, and it will greatly increase the learning cost for the production manufacturer to learn and master the corresponding adjustment processing logic.

[0003] The current solutions to the above problems are mainly as follows: (1) Arrange corresponding after-sales support personnel to carry out on-site development and debugging, and convert the development and debugging tasks into demand documents and feed back to the corresponding developers for development; (2) The corresponding engineers are on-site to deal with the problems when they occur; (3) Build a modular system and write various debugging instruction documents or SOPs for developers to use after training; (4) Use AI learning to realize control, but currently AI training usually uses classification, semantic and instance segmentation, target detection and other technologies to realize, and the processing of extreme small defects, color abnormalities and other defects that require multiple traditional algorithm operators is still required to adjust the control logic and parameters by corresponding algorithm engineers, and the control of part of the defects will involve imaging and algorithm joint debugging, which is also a difficult point for AI to handle. The above situations all require special personnel to handle the debugging tasks and convert the corresponding tasks into development requirements, so as to coordinate the corresponding developers and resources to develop, and the overall efficiency is low. SUMMARY

[0004] The application provides a defect detection system based on an LLM model to solve the above technical problems.

[0005] To solve the above technical problems, the application provides a defect detection system based on an LLM model, which comprises a visual detection unit and an intelligent debugging unit,

[0006] The visual detection unit is provided with a plurality of API interfaces, which are connected with at least a camera, an alarm module, a database, a communication module and a defect management module, and support parameter configuration of the corresponding modules.

[0007] The intelligent debugging unit comprises an interactive interface and a processor, and the processor comprises an intelligent debugging flow programming module, a debugging workflow analysis module, a debugging workflow parameter optimization module and a debugging workflow deployment module. The intelligent debugging flow programming module is obtained by training an LLM model.

[0008] A user inputs a demand task through the interactive interface, the intelligent debugging flow programming module analyzes the demand task and generates corresponding debugging workflow information, the debugging workflow information comprises a workflow specification document and workflow data, the workflow specification document is used for display and interaction on the interactive interface, and the workflow data is a debugging instruction of the entire workflow.

[0009] The debugging workflow analysis module is used for analyzing the workflow data and generating an interface sequence composed of interfaces of the visual detection unit.

[0010] The debugging workflow parameter optimization module is used for optimizing search of each parameter in the interface sequence.

[0011] The debugging workflow deployment module is used for generating a dynamic library and a configuration parameter document based on the optimized parameters, and the dynamic library and the configuration parameter document are imported through the visual detection unit to realize online operation of the debugging workflow.

[0012] Preferably, the parameter configuration of the camera at least comprises a camera serial number, an identification number, an imaging parameter related pixel size, a camera trigger mode related trigger, a dark field, a gamma, a frame rate, a trigger source related to imaging data acquisition, an exposure time, a trigger model, a gain related to white balance, and support for adding, deleting, modifying and saving the camera.

[0013] Preferably, the parameter configuration of the defect management module comprises adding, modifying and deleting of defect detection rules, and supports configuration of defect detection signals, priorities and card control parameters.

[0014] Preferably, the training data of the LLM model at least includes: introduction of each module of the visual detection unit; operation flow of the software; general knowledge of the corresponding field; and software debugging log recording document.

[0015] Preferably, the requirement task is input in the form of voice or text.

[0016] Preferably, the debugging workflow information is divided into two types according to whether human intervention is needed: one is a debugging flow that needs human intervention, and the workflow description document lists the content, operation steps, evaluation method, result saving form that need human intervention, and is displayed on the interface for interaction, and after human intervention processing, enters the debugging workflow analysis module; the other is a debugging flow that does not need human intervention, and the program directly enters the debugging workflow analysis module after user confirmation.

[0017] Preferably, for the debugging flow that needs human intervention, the human intervention result is checked, and if the check fails, the failure information is returned to the intelligent debugging flow arrangement module, and corresponding prompt information is generated and displayed on the interactive interface for manual adjustment until the debugging workflow analysis module meets the analysis requirements.

[0018] Preferably, whether to enter the debugging workflow parameter optimization module is judged according to whether parameter search is needed, if needed, the debugging workflow parameter optimization module is entered for parameter optimization; if not needed, the debugging workflow deployment module is directly entered.

[0019] Preferably, the debugging workflow parameter optimization module uses any one of gradient descent, grid search method, Bayesian optimization or particle optimization algorithm for parameter optimization.

[0020] Preferably, the debugging workflow parameter optimization module uses the grid method to search the optimal parameters in the range of each parameter with a step of 0.01.

[0021] Compared with the prior art, the defect detection system based on the LLM model has the following advantages:

[0022] 1. The present application builds an intelligent detection system that can respond to user debugging requirement tasks by using various data document information in the detection process of the visual detection unit, which can significantly reduce the use threshold requirement and reduce the human and resource consumption in the use process of the visual detection unit.

[0023] 2、The LLM model learned by using the debugging data can realize intelligent answering for common demand problems, and can also realize intelligent adjustment by adjusting imaging and algorithm formula of the demand task, thereby solving the problems of high cost and low efficiency in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0024] Fig. 1 a block diagram of a defect detection system based on an LLM model in an embodiment of the present application;

[0025] Fig. 2 a response diagram of an intelligent debugging unit to input in an embodiment of the present application;

[0026] Fig. 3 a processing flow diagram of an intelligent debugging unit in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to more fully describe the technical solutions of the above-mentioned application, the following specific embodiments are listed to prove the technical effects; it should be emphasized that these embodiments are used to illustrate the present application and not to limit the scope of the present application.

[0028] The defect detection system based on the LLM model provided by the present application comprises a visual detection unit and an intelligent debugging unit, and the intelligent debugging unit is used to realize intelligent adjustment of the visual detection unit, wherein: Figs. 1 to 3

[0029] The visual detection unit is provided with a plurality of API interfaces (Application Programming Interface, i.e. application programming interface), and the API interfaces are connected with at least a camera, an alarm module, a database, a communication module and a defect management module, and support parameter configuration corresponding to each module.

[0030] The intelligent debugging unit comprises an interactive interface (UI interface) and a processor, the interactive interface is used as a front end, and the processor is used as a back end processing software, the processor comprises an intelligent debugging flow arrangement module, a debugging work flow analysis module, a debugging work flow parameter optimization module and a debugging work flow deployment module; the intelligent debugging flow arrangement module is the core of the whole processing flow, which is responsible for responding to customer demand and arranging corresponding debugging work flow on one hand, and is responsible for all error debugging in the whole program (or flow) processing process and giving corresponding prompt information on the other hand, the intelligent debugging flow arrangement module can be obtained by training the LLM model.

[0031] ​A user inputs a demand task through the interactive interface, the intelligent debugging flow arrangement module analyzes the demand task, and generates corresponding debugging workflow information, the debugging workflow information includes a workflow specification document and workflow data, the workflow specification document is used for display and interaction on the interactive interface, and the workflow data is a debugging instruction of the whole workflow.

[0032] The debugging workflow analysis module is used for analyzing the workflow data, and generating an interface sequence composed of interfaces of the visual detection unit.

[0033] The debugging workflow parameter optimization module is used for optimizing each parameter in the interface sequence.

[0034] The debugging workflow deployment module is used for generating a dynamic library and a configuration parameter document based on the optimized parameters, and the dynamic library and the configuration parameter document are imported through the visual detection unit to realize online operation of the debugging workflow.

[0035] By using the large language model learned by the debugging data, the present application can realize intelligent answering to common demand problems, and can realize intelligent adjustment by adjusting imaging and algorithm formula of the demand task, thereby solving the problems of high learning cost and resource and manpower consumption; the present application also uses the visual detection unit to detect various data document information in the process, and builds an intelligent detection system that can respond to the demand of the user debugging task, which can significantly reduce the use threshold demand and reduce the manpower and resource consumption in the process of various debugging, optimization and other task processing of the visual detection unit.

[0036] In some embodiments, the visual detection unit realizes the following functions: (1) configuration of visual imaging parameters, including camera serial number, identification number, pixel size, height, width, trigger related parameters including camera trigger mode, dark field, gamma, frame rate, imaging data acquisition related parameters including trigger source, exposure time, trigger model, white balance related parameters including various gains, and support for adding, deleting, modifying, saving and the like of the camera; (2) support for setting alarm information, such as alarm mode, alarm rule, alarm type, and support for adding and modifying alarm information; (3) support for configuration and setting of a database, including database type, address, username, password, configuration parameters and the like; (4) support for configuration of a communication module, such as communication mode, data, instruction protocol and the like; (5) support for configuration of defect management, including addition, modification and deletion of defect detection rules, and support for configuration of defect detection signals, priority and card control parameters; (6) support for interface configuration of other parameter setting requirements, such as debugging setting, image storage setting, team setting and the like.

[0037] In addition, during the development of the first step, a detailed description and instruction document is written for each module and interface in the first step, which clearly defines the effect, range, parameter, and parameter description of each interface. The document is used for the training of the LLM (Large Language Model) in the subsequent intelligent debugging unit. Specifically, in one embodiment, a coating software is used as the basic software, and corresponding API interfaces are developed for all the above functions in the coating software, and the interfaces are packaged in a specified programming language for subsequent calling.

[0038] In some embodiments, referring to Fig. 2 and Fig. 3 , the demand task can be input in the form of voice or text, reducing the manual participation in the entire debugging process, shortening the development and debugging workload and the online cycle of the debugging task.

[0039] The specific debugging process is as follows: the front-end UI interface can be used for user input and output, for inputting customer demand and displaying the processed results. The back-end processing software takes the intelligent debugging process arrangement module as the core, which uses the pre-trained large language model of the LLM to learn the response to common demand tasks through a large amount of data. The specific demand task includes automatic response and operation workflow of imaging parameter adjustment, card control algorithm operator combination and parameter adjustment, card control process change, defect increase and decrease, and program execution error, debugging information, etc. The intelligent debugging process arrangement module generates corresponding debugging workflow information based on the demand, which includes workflow description document and workflow data. The workflow description document is used for interface display and interaction, and the workflow data defines the entire processing workflow in the form of debugging instruction combination, including the corresponding parameter range and step size, etc.

[0040] The program is divided into two cases according to whether human intervention is needed: one is the debugging process that needs human intervention; the other is the debugging process that does not need human intervention; for the debugging process that needs human intervention, the workflow description document lists the content, operation steps, evaluation method, result saving form that need human intervention, and displays them on the interface for human intervention, after the human intervention, the subsequent debugging workflow analysis module is entered, the debugging workflow analysis module checks the human intervention result, if the check fails, the failure information is attached and entered into the intelligent debugging process arrangement module, and the corresponding prompt information is generated and displayed on the UI interface for manual adjustment until the debugging workflow analysis module meets the analysis requirements, then according to whether parameter search is needed to judge whether to enter the debugging workflow parameter optimization module. For the debugging process that does not need human intervention, the program directly enters the debugging workflow analysis module after the user confirms, the module analyzes the generated debugging workflow information, if an error occurs during the analysis process, the error information is returned to the intelligent debugging process arrangement module for debugging until it passes, then according to whether parameter search is needed to judge whether to enter the debugging workflow parameter optimization module. When the debugging workflow information is processed in the debugging workflow analysis module, if parameter search is needed, the debugging workflow parameter optimization module is entered, the module searches the parameters in the compiled workflow data, if any error occurs during execution, the error information is returned to the intelligent debugging process arrangement module for debugging until the correct search is performed, then the debugging workflow deployment module is entered for compilation and processing, the corresponding dynamic library and configuration file are generated, and the software is restarted online; if parameter search is not needed, the debugging workflow deployment module is directly entered, the corresponding dynamic library and parameter configuration file are generated after the module is compiled, any error occurring in the process is returned to the intelligent debugging process arrangement module for debugging until the compilation is passed to generate the dynamic library and the configuration file.

[0041] In some embodiments, the training process of the LLM model can use a customized GPT model, such as GPT2, GPT3, and can also use an open source commercially available large language model fine-tuned, such as ChatGPT, ChatGPT2, ChatGPT3, LLAMA2, LLAMA3, LLAMA3.1, MOSHI, etc. During the entire custom training process, the training data of the LLM model at least contains: the introduction of each module of the visual detection unit, including the corresponding API interface call, parameter setting, expected result, etc. information; software operation process, including imaging, defect card control algorithm, process, parameter setting information, etc. operation information, each process is described by corresponding API call instruction, etc.; general knowledge in the corresponding field, taking the photovoltaic solar cell field as an example, it can include solar cell production and process books, papers, etc. Document information sorting; and software debugging log recording document, which records the documents collected by the customer's demand during the actual visual detection unit detection, various processing and debugging process, which describes the response to each demand task, processing steps, parameter settings, final results and performance, etc. information. Through the training of the above documents, the intelligent debugging process arrangement module based on the LLM model can respond to various debugging demand tasks, and generate corresponding debugging work flow documents according to the debugging demand tasks. The debugging instruction part of the debugging work flow document clearly defines the input of the debugging process; the input and output of each calling interface; the execution order; and the output of the whole debugging process. It is a work flow instruction file that specifies input and output, and can generate a complete work flow during compilation. The intelligent debugging process arrangement module can also analyze and debug all subsequent module execution error information.

[0042] In some embodiments, Python can be used as the debugging instruction code. After the user proposes the task demand of "adding chamfer defect", the intelligent debugging process arrangement module generates a segment of operation process description document and a segment of python code after processing. The description document is displayed on the UI interface to prompt the customer to the required operation and steps, while the python code is used for the underlying process compilation and debugging. Each python function corresponds to a specific debugging operation, and specifies the input and output of the whole Python program; the configuration parameters show the step size and range of each parameter in the form of a dictionary. After the Python code is parsed by the subsequent work flow parsing module, the work flow of the parameters and interfaces is generated.

[0043] In some embodiments, the debugging workflow parsing module is used to process the debugging workflow information generated by the intelligent debugging flow programming module, wherein the workflow data can be compiled into executable interface code, and the debugging workflow parsing module performs error checking when an error is encountered during the execution of the parsing process, and returns the intelligent debugging flow programming module with error information and returns the correct debugging workflow information content until executable interface code is generated, which can be used for subsequent debugging workflow parameter optimization module and debugging workflow deployment module work, and the workflow specification document is attached to the corresponding generated interface code as historical information, which is input to the intelligent debugging flow programming module for debugging work after subsequent errors. In one embodiment, this step is a Python language parser that can parse the generated debugging workflow information and generate an interface sequence composed of visual inspection software interfaces, which is used for subsequent parameter search and compilation work.

[0044] In some embodiments, the debugging workflow parameter optimization module uses any one of gradient descent, grid search method, Bayesian optimization or particle optimization algorithm for parameter optimization. If the parsed debugging workflow information does not contain parameters, the program will not enter this module, and will directly enter the final debugging workflow deployment module for compilation after the debugging workflow information is parsed by the debugging workflow parsing module, to generate corresponding dynamic libraries and configuration files, and complete the debugging optimization work by loading the software; if the parsed debugging workflow information contains parameters, the module will use optimization algorithms to search for optimal parameters in the entire debugging workflow, and the optimization results will be passed into the final debugging workflow deployment module in the form of parameters and compiled to generate corresponding dynamic libraries and parameter files for software loading to complete the debugging optimization work. In one embodiment, the debugging workflow parameter optimization module uses the grid method to search for optimal parameters in each parameter range with a step size of 0.01 to achieve automatic setting of gain parameters, exposure parameters, threshold segmentation parameters, open operation parameters, closed operation, and or, intersection, etc. parameters, and to achieve detection of the corner defect on the input solar cell wafer.

[0045] In some embodiments, the debugging workflow deployment module is configured to process the parsed debugging workflow information, such as the search results of the attached parameters, and automatically merge the optimized parameters to generate a corresponding dynamic library and configuration parameter document; the dynamic library and configuration parameter document can be imported by the visual inspection unit to realize the online operation of the debugging workflow. In one embodiment, the step can be composed of a C++ compiler and a piece of Python code, the C++ compiler compiles the parsed debugging workflow information into a dynamic library and a configuration file. The Python code is used to implement the entire processing process and information transmission, including dynamic library and configuration file updating, software restart, etc. After the C++ compiler successfully compiles, the Python code first closes the software, uses the latest generated dynamic library and parameter document to overwrite the existing library and parameters of the software, then imports the new parameters and starts the software. If the step fails to execute, the original dynamic library and configuration file are retained, and error information is returned to the intelligent debugging process arrangement module for debugging.

[0046] Next, taking the task requirement of adding "chamfer corner defect" raised by the customer as an example, the detailed debugging process is introduced.

[0047] After the task requirement is input into the system, the intelligent debugging process arrangement module responds to the processing to generate the following workflow description document:

[0048] 1. Respond to and set the task number to 23416.

[0049] 2. The processing of the task includes five steps, the first step of which requires manual participation in processing, and the following is the content of each processing step:

[0050] Step 1: Defect image collection: This step requires manual assistance to collect images containing chamfer defects and normal chamfer images and place them in the "workdirs / 23416 / images / Chamfer Defect" and "workdirs / 23416 / images / OK" directories. During the collection process, the following checks should be carefully performed on the collected images: (1) Ensure that each chamfer defect image has a defect on the chamfer area, and each OK battery has no defects; (2) Try to make the chamfer defect images of the four chamfer areas as similar as possible, and use different batches of images for OK images; (3) If there are different chamfer sizes, collect at least 100 images for each different chamfer size, which can be placed in "workdirs / 23416 / images / Chamfer Defect / Size 1", "workdirs / 23416 / images / Chamfer Defect / Size 2", …, and several directories are established for each chamfer size, with at least 100 images of the same size; (4) If there are other differences, as shown in (3), directories are established accordingly and placed in the corresponding folders, such as: workdirs / 23416 / images / Chamfer Defect / Difference 1", "workdirs / 23416 / images / Chamfer Defect / Difference 2", "Difference 1" and "Difference 2" names can be set arbitrarily, and several difference type folders are established for several difference images, but ensure that the directory contains OK images, "workdirs / 23416 / images / Chamfer Defect / OK"; (5) If the number of images is less than 100, 50 images of each type can also be used, and the reduction of images may cause the effect to be worse; (6) The format of the collected images should be bmp or png, other formats such as jpeg, jpg can also be processed, but may cause the effect to be longer due to image compression; (where workdirs is a system pre-set working directory, which is automatically created after system startup).

[0051] Step 2: Defect image inspection: This step checks the collected images to ensure that the collected images meet the requirements, and if they do not meet the requirements, they may need to be re-collected, but during the re-collection process, matters needing attention will be given.

[0052] Step 3: Automatic processing: This step uses the collected images to automatically adjust the recipe and parameters, and automatically updates to the software, after the update is completed, a'software updated 23416 task' prompt will be given on the interface, after seeing this prompt, you can use the actual chamfer defect image to evaluate the effect of the debugging test.

[0053] Step 4: Effect evaluation: This step can be selected offline test or online test, default online test, do not do anything, directly run the film; offline test mode needs to be checked in the system settings menu offline mode, and manually restart to start, or can also be in the interface input'set offline test mode', the system will automatically switch the software to offline mode. Then the normal evaluation process can be used to evaluate the accuracy of the chamfer corner detection, if the effect is not good after evaluation test, one of the following two ways can be used to optimize: one is to continue the current task to optimize, more defects and OK can be collected to supplement the task image directory, and then optimize the algorithm (the increased defect / OK image needs to be put into the corresponding defect folder), on the other hand, the defect image can not be increased, so only the parameter re-optimization is done, after processing the above content, directly input 'debug 23416' to automatically execute.

[0054] Step 5: Please confirm whether the defect image collection of the first step has been completed, after confirmation, input 'continue 23416' to enter the next operation process.

[0055] The whole process is as follows: after the user inputs the task debugging requirement in the corresponding UI interaction interface, the intelligent debugging software automatically assigns the corresponding task serial number, and establishes the corresponding task serial number work folder in the work directory workdirs (the system pre-set work directory, automatically created after system startup), the corresponding task requirement information will flow into the intelligent debugging process programming module, and the task requirement analysis and process programming will be carried out. When the programming is completed, the debugging workflow information will be generated, which includes workflow specification document and workflow data. The workflow specification document is used for interface display, interaction and debugging, and the workflow data is used for background processing. When the user confirms the completion of the manual intervention part through UI interaction, the program will automatically enter the debugging workflow analysis module, which will execute the instructions defined in the workflow data one by one, and the execution error will return to the intelligent debugging process programming module for error checking. After the whole analysis work is completed, whether the parameter search is included will be processed, if the parameter search is needed, it will enter the debugging workflow parameter optimization module, which will search the parameters in the compiled debugging workflow information, and after the search is completed, it will enter the debugging workflow deployment module for compilation and system online operation.

[0056] In summary, the defect detection system based on the LLM model provided by the application takes the LLM pre-training large language model trained based on a large amount of industry knowledge and software debugging data as the core, and constructs an intelligent debugging system. The system can intelligently update the defects and conventional operations in the detection, simplifies the formula updating and debugging process of the visual detection system, uses the latest AI technology to realize the intelligent adjustment mode, and is expected to improve the iteration cycle and resource cost in the product updating and debugging process.

[0057] Obviously, those skilled in the art can make various modifications and variations to the application without departing from the spirit and scope of the application. Thus, if these modifications and variations of the application fall within the scope of the claims of the application and their equivalents, the application also intends to include these modifications and variations.

Claims

1. A defect detection system based on an LLM model, characterized in that, Includes a visual inspection unit and an intelligent debugging unit. The visual inspection unit has multiple API interfaces, which are connected to at least the camera, alarm module, database, communication module and defect management module, and support parameter configuration for each module. The intelligent debugging unit includes an interactive interface and a processor. The processor includes an intelligent debugging process orchestration module, a debugging workflow parsing module, a debugging workflow parameter optimization module, and a debugging workflow deployment module. The intelligent debugging process orchestration module is obtained by training an LLM model. The user inputs the required task through the interactive interface. The intelligent debugging process orchestration module parses the required task and generates corresponding debugging workflow information. The debugging workflow information includes a workflow description document and workflow data. The workflow description document is used for display and interaction on the interactive interface, and the workflow data is the debugging instructions for the entire workflow. The debugging workflow parsing module is used to parse the workflow data and generate an interface sequence consisting of the interfaces of the vision detection unit; The debugging workflow parameter optimization module is used to perform optimization search on each parameter in the interface sequence; The debugging workflow deployment module is used to generate a dynamic library and configuration parameter document based on the optimized parameters. The dynamic library and configuration parameter document are imported through the visual inspection unit to realize the online operation of the debugging workflow. Based on whether manual intervention is required, the debugging workflow information is divided into two types: one is a debugging process that requires manual intervention. The workflow documentation lists the content, operation steps, evaluation methods, and result saving formats that require manual intervention and displays them interactively on the interface. After manual intervention, the process enters the debugging workflow parsing module. Another type is a debugging process that does not require manual intervention, in which the program directly enters the debugging workflow parsing module after user confirmation.

2. The defect detection system based on the LLM model as described in claim 1, characterized in that, The camera's parameter configuration includes at least the camera serial number and identification number; pixel size related to imaging parameters; camera triggering methods, dark field, gamma, and frame rate related to triggering; trigger source, exposure time, and triggering model related to imaging data acquisition; gain related to white balance; and support for adding, deleting, modifying, and saving cameras.

3. The defect detection system based on the LLM model as described in claim 1, characterized in that, The parameter configuration of the defect management module includes: adding, modifying and deleting defect detection rules, and supporting the configuration of defect detection signals, priorities and control parameters.

4. The defect detection system based on the LLM model as described in claim 1, characterized in that, The training data of the LLM model includes at least: an introduction to each module of the visual detection unit; the software operation process; general knowledge in the corresponding field; and software debugging log records.

5. The defect detection system based on the LLM model as described in claim 1, characterized in that, The required tasks are input in the form of voice or text.

6. The defect detection system based on the LLM model as described in claim 1, characterized in that, For debugging processes that require manual intervention, the system also includes checking the results of manual intervention. If the check fails, a failure message is returned to the intelligent debugging process orchestration module, and corresponding prompts are generated and displayed on the interactive interface for manual adjustment until the requirements of the debugging workflow parsing module are met.

7. The defect detection system based on the LLM model as described in claim 6, characterized in that, Whether to enter the debugging workflow parameter optimization module depends on whether parameter search is required. If so, enter the debugging workflow parameter optimization module to optimize parameters. If not, proceed directly to the debugging workflow deployment module.

8. The defect detection system based on the LLM model as described in claim 7, characterized in that, The debugging workflow parameter optimization module uses any one of the following algorithms for parameter optimization: gradient descent, grid search, Bayesian optimization, or particle optimization.

9. The defect detection system based on the LLM model as described in claim 8, characterized in that, The debugging workflow parameter optimization module uses a grid method to search for the optimal parameters in each parameter range with a step size of 0.01 for the defects to be controlled.

Images