Production line visual defect detection optimization method, computing equipment and readable storage medium

Abstract

The invention relates to a production line visual defect detection optimization method, a computing device and a readable storage medium, and the method comprises the steps: carrying out the preliminary training of a large language model through a collected product database, forming a pre-trained large language model, carrying out the preliminary configuration of a visual detection system, and carrying out the defect detection of a product, when a defective product is detected subsequently, the image data of the defective product and the text guidance data can be bound to form secondary training data to carry out fine tuning on the pre-trained large language model, so that model optimization is realized, and the accuracy of product defect detection is improved. When a defect image of a to-be-detected product is input into the big language optimization model, a visual detection process and a parameter configuration guidance text corresponding to the defect image are output through the big language optimization model, the output form is presented in a dialogue mode based on the big language optimization model, and guided configuration of each process and parameter is completed. And the technical requirements of production line personnel are reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and specifically to a production line visual defect detection optimization method, a computing device and a readable storage medium. Background Art

[0002] In visual defect detection, since defect data in real-world samples is extremely rare, sometimes only one defect data appears in thousands or even tens of thousands of samples, resulting in a small amount of defect sample data. Therefore, previous models only need to be trained on normal samples to learn the data distribution of normal samples. During testing, parameters such as thresholds need to be manually specified to distinguish normal and abnormal instances of each item. However, for production line personnel who have no experience in visual inspection, it is difficult to master the parameter characteristics corresponding to various defects and make changes in real time.

[0003] Therefore, how to achieve intelligent training of defect detection models and intelligent guidance of production line personnel is the technical problem that this application aims to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a production line visual defect detection optimization method, computing equipment and readable storage medium, which can improve the detection accuracy of defective products, and at the same time provide configuration guidance for production line personnel and reduce job technical requirements.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: The present invention provides a production line visual defect detection optimization method, which includes the following working steps: Step S1: deploying an interactive platform based on a large language model on a control host of a visual inspection system, wherein the interactive platform has a human-computer interaction interface; Step S2: introducing a product database of a visual inspection system to pre-train a large language model in the interactive platform, so that the interactive platform has the ability to process defect detection of corresponding products; Step S3: configuring the visual inspection process and inspection parameters of the visual inspection system, and forming corresponding text guidance data; Step S4: inspect the product through the configured visual inspection system and bind the defective product image data with the text guidance data to form secondary training data; Step S5: performing secondary fine-tuning on the pre-trained large language model at the interactive platform through the secondary training data to obtain a large language optimization model, and the large language optimization model is used as the final processing tool of the interactive platform.

[0006] The interactive platform integrates advanced natural language processing technology to form secondary training data for intelligent optimization of visual inspection processes and parameter configurations, which can improve the accuracy and efficiency of defect detection by the visual inspection system.

[0007] According to one embodiment of the present invention, in step S1, the interactive platform is locally deployed based on llama3.

[0008] According to one embodiment of the present invention, in step S1, the interactive interface is based on a WEB UI or an independently designed UI, and the visual inspection result is presented through the interactive interface, or the visual inspection process and inspection parameters configured for the visual inspection are presented.

[0009] According to one embodiment of the present invention, in step S5, a fine-tuning script or a llama-factory tool is used to perform model fine-tuning training to obtain the large language optimization model.

[0010] According to one embodiment of the present invention, when the large language model pre-trained at the interactive platform is fine-tuned for a second time using the secondary training data, it is organized according to the dialogue format, and then the model is fine-tuned using a fine-tuning script or llama-factory tool using the organized secondary training data.

[0011] According to one embodiment of the present invention, step S6 is included: taking the defect image of the product to be inspected as input to the large language optimization model, and outputting the visual inspection process and parameter configuration guidance text corresponding to the defect image through the large language optimization model.

[0012] According to one embodiment of the present invention, the visual inspection process and parameter configuration guidance text output corresponding to the defect image is used to guide production line personnel to configure the direction of visual defect detection, including the configuration of morphological, filtering, and transform analysis processes and their parameters.

[0013] According to one embodiment of the present invention, it also includes a manual inspection step of the visual inspection process and parameter configuration guidance text corresponding to the output of the defect image, manually correcting the invalid model description in the visual inspection process and parameter configuration guidance text, collecting the corresponding image data and the corrected guidance text, and repeating steps S4 and S5.

[0014] In particular, the present invention also provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0015] The present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method described above is implemented. The computer-readable storage medium is preferably a non-volatile readable storage medium.

[0016] Compared with the prior art, the advantages and beneficial effects of the patent application of the present invention are: The production line visual defect detection optimization method of the present application first uses the collected product database, which may include normal product data and abnormal product data with defects, to preliminarily train the large language model to form a pre-trained large language model. After the visual inspection system is preliminarily configured, it can be used for product defect detection. Subsequently, when a defective product is detected, the defective product image data and text guidance data can be bound to form secondary training data to fine-tune the pre-trained large language model, thereby achieving model optimization and improving the accuracy of product defect detection.

[0017] Furthermore, when the defect image of the product to be inspected is used as input to the big language optimization model, the big language optimization model outputs the visual inspection process and parameter configuration guidance text corresponding to the defect image. The output form is presented in a dialogue based on the big language optimization model to complete the guided configuration of each process and parameter, thereby reducing the operational technical requirements of the production line personnel, and can effectively maintain the normal operation of the production line while reducing the maintenance cost of the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings: Figure 1 is a flow chart of a production line visual defect detection optimization method according to an embodiment of the present invention; Figure 2 is a flow chart of a production line visual defect detection optimization method according to another embodiment of the present invention; Figure 3 is a structural diagram of a computing device according to another embodiment of the present application; Figure 4 It is a structural diagram of a computer-readable storage medium according to another embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] This embodiment describes a production line visual defect detection optimization method, such as Figure 1 As shown, it includes the following working steps: Step S1: deploying an interactive platform based on a large language model on a control host of a visual inspection system, wherein the interactive platform has a human-computer interaction interface; Step S2: introducing a product database of a visual inspection system to pre-train a large language model in the interactive platform, so that the interactive platform has the ability to process defect detection of corresponding products; Step S3: configuring the visual inspection process and inspection parameters of the visual inspection system, and forming corresponding text guidance data; Step S4: inspect the product through the configured visual inspection system and bind the defective product image data with the text guidance data to form secondary training data; Step S5: performing secondary fine-tuning on the pre-trained large language model at the interactive platform through the secondary training data to obtain a large language optimization model, and the large language optimization model serves as the final processing tool of the interactive platform.

[0022] The interactive platform integrates advanced natural language processing technology to form secondary training data for intelligent optimization of visual inspection processes and parameter configurations, which can improve the accuracy and efficiency of defect detection by the visual inspection system.

[0023] In one implementation mode, in step S1, the interactive platform is locally deployed based on llama3, the interactive interface is based on a WEB UI or an independently designed UI, and the visual inspection results are presented through the interactive interface for user viewing, or the visual inspection process and inspection parameters configured for the visual inspection are presented.

[0024] In one embodiment, in step S5, a fine-tuning script or a llama-factory tool is used to perform model fine-tuning training to obtain the large language optimization model. In addition, when the large language model pre-trained at the interactive platform is fine-tuned for the second time using the secondary training data, it is organized according to the dialogue format, and then the fine-tuning script or the llama-factory tool is used to perform model fine-tuning training using the organized secondary training data.

[0025] 1. Use the fine-tuning script to perform model fine-tuning training Taking fine-tuning LLaMA 3 as an example, the process of fine-tuning the model using the fine-tuning script is as follows: Choose a pre-trained model: LLaMA 3 is a large language model, and the pre-trained model weights and configurations can be obtained through Hugging FaceTransformers or other related platforms.

[0026] Prepare a dataset: The dataset is associated with the problem or task to be solved. For this embodiment, it contains product image data with defects and is bound with text guidance data to form secondary training data so that the model can learn task-specific patterns and features through fine-tuning.

[0027] Install necessary libraries and tools: Make sure that you have appropriate deep learning libraries installed in your environment, such as TensorFlow or PyTorch, and a compatible Transformers library.

[0028] Load pre-trained model and dataset: Use the corresponding library functions to load the pre-trained LLaMA 3 model and its configuration, as well as the previously prepared dataset. The dataset should be compatible with the model's input format and include training data and possible validation data.

[0029] Define training parameters and optimizer: Set training parameters such as device (such as CPU or GPU), training rounds (epochs), batch size, and learning rate, and define the optimizer and learning rate scheduler.

[0030] Train the model: Train the model using the prepared dataset and defined training parameters. This typically involves defining a training loop where you perform a forward pass, calculate the loss, perform a backpropagation, and update the model weights.

[0031] Evaluation and tuning: After training is complete, evaluate the model’s performance on the validation set and perform model tuning and adjustments as needed.

[0032] Applying the model: using the fine-tuned model for prediction or application, and saving the model for subsequent use, the obtained model is the large language optimization model.

[0033] 2. Use LLaMA-Factory tool to fine-tune the model LLaMA-Factory is an open source model fine-tuning framework that is dedicated to simplifying the customization process of large language models. The process of model fine-tuning training using LLaMA-Factory is roughly as follows: Environmental preparation: Make sure that Python 3.9 or higher and necessary deep learning frameworks such as PyTorch are installed in your development environment.

[0034] Create an instance with enough video memory to meet the computational needs of model fine-tuning.

[0035] Get LLaMA-Factory: Open the terminal or command line interface and execute the corresponding command to clone the LLaMA-Factory code repository to your local computer.

[0036] Enter the cloned repository directory and execute the command to install dependencies.

[0037] Prepare the dataset: LLaMA-Factory supports multiple dataset formats. If you need to use a custom dataset, you need to process the data in the format supported by LLaMA-Factory and place it in the specified directory. For this embodiment, it is to bind the defective product image data with the text guidance data to form secondary training data so that the model can learn task-specific patterns and features through fine-tuning.

[0038] At the same time, the corresponding configuration files need to be updated to ensure that the dataset is correctly identified and loaded.

[0039] Select Model: In LLaMA-Factory, you can choose a variety of pre-trained models as the base model, such as LLaMA, Mistral, Falcon, etc.

[0040] Configure training parameters: In the WebUI interface of LLaMA-Factory, you can configure various training parameters, such as learning rate, batch size, training rounds, etc. You can also choose fine-tuning methods, such as full parameter fine-tuning, LoRA fine-tuning, etc.

[0041] Start training: After configuring the training parameters, click the Start Training button (you can also set it to automatically enter training after input), and LLaMA-Factory will start training the model.

[0042] During the training process, you can view indicators such as training progress and loss function value in the WebUI interface.

[0043] Evaluation and Tuning: After training is complete, the performance of the model can be evaluated on the validation set and the model can be fine-tuned and adjusted as needed.

[0044] You can view the evaluation results in the WebUI interface and adjust the training parameters or fine-tune the method based on the results.

[0045] Export and apply the model: After model fine-tuning is complete, the trained model can be exported into a deployable format.

[0046] The derived model is used for prediction or application, and further optimization and adjustment can be performed as needed. The obtained model is the large language optimization model. In summary, the process of fine-tuning model training using the fine-tuning script or LLaMA-Factory tool involves multiple steps, including environment preparation, obtaining tools, preparing datasets, selecting models, configuring training parameters, starting training, evaluating and tuning, and exporting and applying models. The specific operations and details of these steps may vary depending on the tools and tasks.

[0047] Therefore, the production line visual defect detection optimization method of the present application first collects a product database, which may include normal product data and abnormal product data with defects, and preliminarily trains the large language model to form a pre-trained large language model. After the visual inspection system is preliminarily configured, it can be used for product defect detection. Subsequently, when a defective product is detected, the defective product image data and text guidance data can be bound to form secondary training data to fine-tune the pre-trained large language model, thereby achieving model optimization and improving the accuracy of product defect detection.

[0048] In one embodiment, if Figure 2 As shown, the method further includes step S6: using the defect image of the product to be inspected as input to the large language optimization model, and outputting the visual inspection process and parameter configuration guidance text corresponding to the defect image through the large language optimization model. The output visual inspection process and parameter configuration guidance text corresponding to the defect image are used to guide production line personnel to configure the visual defect detection direction, including the configuration of each process and its parameters of morphology, filtering, and transformation analysis.

[0049] It can be seen that when the defect image of the product to be inspected is taken as input to the big language optimization model, the big language optimization model outputs the visual inspection process and parameter configuration guidance text corresponding to the defect image. The output form is presented in a dialogue based on the big language optimization model to complete the guided configuration of each process and parameter, reduce the technical requirements of the production line personnel, and effectively maintain the normal operation of the production line while reducing the maintenance cost of the production line.

[0050] In addition, the data can be manually checked and corrected periodically. For example, the method also includes a step of manually checking the visual inspection process and parameter configuration guidance text corresponding to the output of the defect image, manually correcting the invalid model description in the visual inspection process and parameter configuration guidance text, collecting the corresponding image data and the corrected guidance text, and repeating steps S4 and S5 to achieve iterative training of the large language optimization model.

[0051] Based on the detailed description of the specific embodiments of the present application in combination with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application.

[0052] The present application also provides a computing device, referring to Figure 3 The computing device includes a memory 1120, a processor 1110, and a computer program stored in the memory 1120 and executable by the processor 1110. The computer program is stored in a space 1130 for program code in the memory 1120. When the computer program is executed by the processor 1110, it is implemented to execute any one of the method steps 1131 according to the present invention.

[0053] The present application also provides a computer-readable storage medium. Figure 4 The computer-readable storage medium comprises a storage unit for program code, the storage unit being provided with a program 1131 ′ for executing the method steps according to the present invention, the program being executed by a processor.

[0054] The embodiment of the present application further provides a computer program product including instructions. When the computer program product is run on a computer, the computer is caused to execute the method steps according to the present invention.

[0055] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer loads and executes the computer program instructions, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0056] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0057] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state drive, a magnetic tape, a floppy disk, an optical disc, and any combination thereof.

[0058] In particular, the present invention also provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0059] The present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method described above is implemented. The computer-readable storage medium is preferably a non-volatile readable storage medium.

[0060] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with this technology to understand the contents of the present invention and implement them accordingly. They cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A production line visual defect detection optimization method, characterized in that: The following steps are included: Step S1: deploying an interactive platform based on a large language model on a control host of a visual inspection system, wherein the interactive platform has a human-computer interaction interface; Step S2: introducing a product database of a visual inspection system to pre-train a large language model in the interactive platform, so that the interactive platform has the ability to process defect detection of corresponding products; Step S3: configuring the visual inspection process and inspection parameters of the visual inspection system, and forming corresponding text guidance data; Step S4: inspect the product through the configured visual inspection system and bind the defective product image data with the text guidance data to form secondary training data; Step S5: performing secondary fine-tuning on the pre-trained large language model at the interactive platform through the secondary training data to obtain a large language optimization model, and the large language optimization model is used as the final processing tool of the interactive platform.

2. The production line visual defect detection optimization method according to claim 1, characterized in that: In step S1, the interactive platform is locally deployed based on llama3.

3. The production line visual defect detection optimization method according to claim 2, characterized in that: In step S1, the interactive interface is based on a WEB UI or an independently designed UI, and the visual inspection result is presented through the interactive interface, or the visual inspection process and inspection parameters configured for the visual inspection are presented.

4. The production line visual defect detection optimization method according to claim 1, characterized in that: In step S5, a fine-tuning script or a llama-factory tool is used to perform model fine-tuning training to obtain the large language optimization model.

5. The production line visual defect detection optimization method according to claim 4, characterized in that: When the large language model pre-trained at the interactive platform is fine-tuned for the second time by using the secondary training data, it is sorted according to the dialogue format, and then the model is fine-tuned by using the fine-tuning script or llama-factory tool through the sorted secondary training data.

6. The production line visual defect detection optimization method according to claim 1, characterized in that: The method comprises step S6: taking the defect image of the product to be inspected as input to the large language optimization model, and outputting the visual inspection process and parameter configuration guidance text corresponding to the defect image through the large language optimization model.

7. The production line visual defect detection optimization method according to claim 6, characterized in that: The output visual inspection process and parameter configuration guidance text corresponding to the defect image is used to guide production line personnel to configure the visual defect detection direction, including the configuration of morphology, filtering, and transformation analysis processes and their parameters.

8. The production line visual defect detection optimization method according to claim 6 or 7, characterized in that: It also includes a manual inspection step of the visual inspection process and parameter configuration guidance text corresponding to the output of the defect image, manually correcting the invalid model description in the visual inspection process and parameter configuration guidance text, collecting the corresponding image data and the corrected guidance text, and repeating steps S4 and S5.

9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored therein, wherein the computer program implements the method according to any one of claims 1 to 8 when executed by a processor.

Images