Defect positioning method and system based on language model

Abstract

The invention relates to the technical field of workpiece detection, in particular to a defect positioning method and system based on a language model, and the method comprises the steps: S100, obtaining an image of the surface of a workpiece, and carrying out the preprocessing; s200, inputting the preprocessed image data into a preliminary positioning model to obtain a preliminary positioning result; s300, generating semantic description according to the image features extracted by the CNN; querying a historical image according to the preliminary positioning result, and extracting historical image features according to the CNN to generate a target semantic description; s400, converting the semantic description into a semantic vector, and converting the target semantic description into a target semantic vector; s500, the semantic vector is optimized; and S600, determining the position of the workpiece defect through the optimized semantic vector to obtain a current positioning result, and comparing the current positioning result with the preliminary positioning result to obtain a final position. According to the method, the workpiece defect detection efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of workpiece detection, and in particular to a defect location method and system based on a language model. Background Art

[0002] With the development of industrial automation and intelligent manufacturing, workpiece defect detection has become increasingly important in ensuring product quality and improving production efficiency. Traditional defect detection methods mainly rely on manual inspection and rule-based image processing technology. These methods usually require operators to have high professional skills and rich experience, but even so, it is difficult to avoid missed detection and false detection problems caused by human eye fatigue or subjective factors. In addition, rule-based image processing technology often shows problems of insufficient robustness and poor adaptability when processing complex workpiece surfaces, resulting in low detection efficiency and poor accuracy.

[0003] With the development of computer vision and machine learning technology, defect detection methods based on deep learning have gradually attracted attention. Deep learning models, especially convolutional neural networks (CNNs), have shown excellent performance in image feature extraction and classification tasks. In the prior art, during defect detection, image data is used as input, and the defect location results are used as output, and a preliminary location model is obtained directly through CNN training; however, most existing deep learning methods require a large amount of labeled data for training, which is often difficult to obtain in actual industrial applications. In addition, these models have high requirements on hardware resources, and the training and reasoning processes are time-consuming, which makes it difficult to meet the requirements of real-time and high efficiency in industrial production processes. Therefore, there is an urgent need for a new method for workpiece defect location to overcome the shortcomings of the prior art and improve the efficiency and accuracy of defect detection. Summary of the invention

[0004] The purpose of the present invention is to provide a defect localization method and system based on a language model, which can improve the efficiency and accuracy of workpiece defect detection.

[0005] The basic solution provided by the present invention is a defect location method and system based on a language model, comprising the following steps:

[0006] S100, data preprocessing step: acquiring an image of the workpiece surface and performing preprocessing;

[0007] S200, preliminary positioning step: inputting the preprocessed image data into the preliminary positioning model to obtain a preliminary positioning result;

[0008] S300, language model training step: generating a semantic description based on the image features extracted by CNN; querying historical images based on the preliminary positioning results, and generating a target semantic description based on the historical image features extracted by CNN;

[0009] S400, semantic vector generation step: converting the semantic description into a semantic vector, and converting the target semantic description into a target semantic vector;

[0010] S500, genetic algorithm optimization step: optimizing the semantic vector;

[0011] S600, defect location step: determine the location of the workpiece defect through the optimized semantic vector, obtain the current location result, compare the current location result with the preliminary location result, and when the overlap meets the threshold, use the current location result as the final location; when the overlap does not meet the threshold, add the current location result to the training set of the preliminary location model, re-train the model, and use the preliminary location result as the final location.

[0012] Furthermore, the step S100 includes: acquiring a high-resolution image of the workpiece surface; graying the image; removing noise using Gaussian filtering; and performing edge detection using Canny edge detection.

[0013] Furthermore, the step S300 includes: using a CNN convolutional neural network to extract image features from the preprocessed image; and using a Transformer model to convert the extracted image features into semantic descriptions.

[0014] Furthermore, the step S400 includes: using the BERT model to convert the generated semantic description into a semantic vector representation, and converting the target semantic description into a target semantic vector representation.

[0015] Furthermore, the step S500 includes: generating an initial population of semantic vectors; selecting semantic vectors with high fitness for reproduction; generating a new generation of semantic vectors through crossover and mutation operations; defining a fitness function to evaluate the accuracy of each semantic vector; and iterating selection, crossover and mutation operations for evolution.

[0016] Further, the fitness function is defined as:

[0017]

[0018] Among them, v i Semantic vector, is the target semantic vector.

[0019] The principles and beneficial effects of the present invention are:

[0020] 1. High-precision defect detection: Through the combination of convolutional neural network (CNN) and Transformer model, image features can be efficiently extracted and accurate semantic descriptions can be generated, thereby significantly improving the accuracy of defect detection.

[0021] 2. High degree of automation: Genetic algorithms reduce human intervention through automatic selection, crossover and mutation operations in semantic vector optimization, realize efficient automatic optimization process, and greatly improve the automation level of the detection system.

[0022] Furthermore, it also includes using the particle swarm optimization algorithm to further optimize the semantic vector: initializing the particle swarm: taking the optimal individual in the genetic algorithm stage as the initial particle swarm of PSO; updating the particle speed and position according to the particle's current position and speed update formula; calculating the fitness value of each particle, updating the individual optimal value and the global optimal value; iterating until the fitness function converges to the optimal solution; alternating iterations between the genetic algorithm and the particle swarm optimization algorithm to optimize the semantic vector.

[0023] The speed and position update formula in the particle swarm optimization algorithm is:

[0024] v i (t+1)=ωv i (t)+c1r1(p i -x i (t))+c2r2(gx j (t))

[0025] x i (t+1)=t i (x)+v i (t+1)

[0026] Among them, ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, and p i is the individual optimal position of the particle, and g is the global optimal position.

[0027] The beneficial effects of this solution are as follows: by introducing the particle swarm optimization algorithm (PSO) on the basis of the genetic algorithm, the semantic vector is further optimized. The genetic algorithm provides global search capabilities, while the PSO further improves the optimization accuracy through fast local search. The combination of the two makes the optimization process more precise and improves the accuracy of defect detection. In addition, the alternating iteration of the two algorithms not only avoids falling into the local optimum, but also effectively shortens the optimization time and improves the overall efficiency.

[0028] Furthermore, it also includes manual verification of the generated semantic description to improve the accuracy of the semantic vector.

[0029] The beneficial effects of this solution are: manual verification can identify and correct erroneous or inaccurate semantic descriptions generated by the language model, thereby reducing false positives and missed positives. By reviewing and correcting the semantic descriptions generated by the language model, erroneous descriptions caused by model errors or data bias can be effectively avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The flowchart of a defect localization method based on language model. DETAILED DESCRIPTION

[0031] The following is further described in detail through specific implementation methods:

[0032] Embodiment 1

[0033] In this embodiment, by providing a defect location method and system based on a language model, the efficiency and accuracy of workpiece defect detection can be improved.

[0034] like Figure 1 A defect location method based on a language model is shown, comprising the following steps: S100, data preprocessing step: acquiring an image of the workpiece surface and performing preprocessing; S200, preliminary location step: inputting the preprocessed image data into the preliminary location model to obtain a preliminary location result; S300, language model training step: generating a semantic description based on the image features extracted by CNN; querying historical images based on the preliminary location result, and generating a target semantic description based on the historical image features extracted by CNN; S400, semantic vector generation step: converting the semantic description into a semantic vector, and converting the target semantic description into a target semantic vector; S500, genetic algorithm optimization step: optimizing the semantic vector; S600, defect location step: determining the position of the workpiece defect through the optimized semantic vector, obtaining the current location result, comparing the current location result with the preliminary location result, and when the overlap meets the threshold, taking the current location result as the final location; when the overlap does not meet the threshold, adding the current location result to the training set of the preliminary location model, retraining the model, and taking the preliminary location result as the final location.

[0035] Step S100: a data preprocessing step, using a high-resolution industrial camera to capture an image of the workpiece surface.

[0036] In this embodiment, a camera with 5 million pixels or higher is selected to ensure good optical performance and stable exposure control. The shooting environment should be evenly illuminated to avoid shadows and light spots. A ring light source or a line light source can be used to provide uniform lighting to reduce surface reflection interference. In this environment, each workpiece surface is photographed from multiple angles and positions to ensure that the acquired image coverage is comprehensive and the resolution meets the requirements of subsequent processing, usually above 3000x2000 pixels.

[0037] Convert the acquired color image to a grayscale image to simplify the image data and reduce the computational complexity. Grayscale processing can be achieved through weighted averaging. In image processing software or programming environment, use built-in functions or custom functions to perform grayscale processing. Use Gaussian filtering to remove noise from grayscale images. Gaussian filtering uses weighted averaging of Gaussian functions to smooth the image and remove random noise in the image. Use the Canny edge detection algorithm to extract edge information from the image.

[0038] Step S300: language model training step.

[0039] Convolutional neural network (CNN) is used to extract image features from preprocessed images. Convolutional neural network extracts important features from images layer by layer through multi-layer convolution and pooling operations. Specifically, CNN consists of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer slides on the image through the convolution kernel to extract local features, and the pooling layer reduces the size of the feature map through downsampling operations, thereby retaining important information while reducing the amount of calculation. The superposition of multiple convolutional layers and pooling layers enables CNN to extract image features at different levels, including edge, texture, shape and other information. These features will serve as input for the subsequent semantic generation step.

[0040] After feature extraction is completed, the extracted image features are converted into semantic descriptions using the Transformer model. The Transformer model is a deep learning model based on the self-attention mechanism, which is good at processing sequence data and has strong semantic understanding capabilities.

[0041] The Transformer model consists of an encoder and a decoder. The encoder encodes the input image features into implicit representations, and the decoder decodes the implicit representations into natural language descriptions. In the encoder part, the self-attention mechanism can capture the global dependencies between image features and enhance the model's expressiveness through the multi-head attention mechanism. In the decoder part, the natural language description is generated by combining the implicit representation output by the encoder and the contextual information of the current decoding step. In order to train the Transformer model, a large-scale annotated dataset is required, which contains images and their corresponding semantic descriptions. By optimizing the model parameters, the generated semantic description can accurately reflect the information in the image. Through the above steps, the training of the language model is completed, and the generated semantic description will serve as the basis for the subsequent semantic vector generation step.

[0042] Step S400: semantic vector generation step: Use the BERT model (Bidirectional Encoder Representations from Transformers) to transform these semantic descriptions into semantic vectors, and transform the target semantic description into the target semantic vector.

[0043] The BERT model is a pre-trained language model based on Transformer. Through the structure of a bidirectional encoder, it can capture the complex relationship between words in the context, thereby generating a high-dimensional vector representation of semantics.

[0044] The generated semantic description is used as the input of the BERT model. The input of the BERT model needs to undergo certain preprocessing, including word segmentation, adding special tags (such as [CLS] and [SEP] tags), and constructing input tensors. The preprocessed semantic description is input into the encoder of the BERT model. The encoder of the BERT model consists of multiple layers of Transformers, each of which contains a self-attention mechanism and a feedforward neural network. The self-attention mechanism can capture the interdependencies between words in the semantic description, and the feedforward neural network further processes these dependencies to generate high-quality feature representations.

[0045] During the encoding process, the BERT model performs context modeling on each word, that is, it considers the words on the left and right sides of the word at the same time, thereby generating a bidirectional feature representation containing context information. After multiple layers of encoding, the representation vector of each word output by the BERT model contains rich semantic information. In particular, the vector representation of the [CLS] tag at the first position in the output of the BERT model can be used as a global feature vector for the entire semantic description. These vector representations will be used as semantic vectors for subsequent optimization and defect location steps.

[0046] Step S500: Genetic algorithm optimization step: After the semantic vectors are generated, the genetic algorithm is used to optimize the semantic vectors.

[0047] An initial population of semantic vectors is randomly generated, each vector representing a possible defect feature. These initial populations of semantic vectors will serve as the starting point for the optimization process.

[0048] The semantic vector with high fitness is selected for reproduction according to the fitness function. The fitness function is used to evaluate the accuracy of each semantic vector, which is defined as:

[0049]

[0050] Among them, v i Semantic vector, is the target semantic vector.

[0051] The fitness function calculates the Euclidean distance between each vector and the target vector, and converts the distance into a fitness value by taking the inverse. The higher the fitness value, the closer the current semantic vector is to the target semantic vector. In this way, the quality of each semantic vector can be quantified, and those with high fitness can be selected as parents for reproduction.

[0052] After selecting the parents with high fitness, a new generation of semantic vectors is generated through crossover and mutation operations. The crossover operation refers to exchanging some values ​​of two parent vectors to generate new offspring vectors. This can be achieved by randomly selecting crossover points, such as swapping parts of two vectors to produce diverse offspring. The mutation operation refers to randomly changing certain vector values ​​to introduce new features and diversity. The mutation operation can generate new mutant individuals by randomly selecting one or more vector elements and randomly adjusting them.

[0053] Subsequently, a fitness function is defined to evaluate each newly generated semantic vector, and the best individual is selected for reproduction according to the fitness value. The semantic vector is gradually optimized through multiple generations of selection, crossover and mutation operations. In each generation, individuals with high fitness are selected for crossover and mutation, and the iteration is continuous, so that the semantic vector gradually approaches the target semantic vector.

[0054] Step S600: Defect location step. The location of the workpiece defect is determined by the optimized semantic vector to obtain the current location result, and the current location result is compared with the preliminary location result. When the overlap meets the threshold, the current location result is used as the final location; when the overlap does not meet the threshold, the current location result is added to the training set of the preliminary location model to retrain the model, and the preliminary location result is used as the final location.

[0055] Specifically, the optimized semantic vector contains rich semantic information and can effectively represent various defect features. These semantic vectors are compared pixel by pixel with the features in the workpiece image, and the locations where defects may exist on the workpiece surface are found as the current positioning results through the similarity matching method.

[0056] In this embodiment, cosine similarity is used to calculate the similarity between the semantic vector and the image feature.

[0057] A sliding window operation is performed on the workpiece surface image to extract small areas from the image, each of which is used as an image feature. Each image feature is also converted into a vector representation.

[0058] For each image feature vector x, calculate its cosine similarity with the optimized semantic vector v:

[0059]

[0060] Where x·v represents the dot product of vectors x and v, and ||x|| and ||v|| represent the norms of vectors x and v, respectively. The value of cosine similarity is between [-1,1]. The closer the value is to 1, the more similar the two vectors are. Traverse all feature vectors of the workpiece image and calculate the cosine similarity between each feature vector and the optimized semantic vector. Select the area with the highest similarity as the defect location.

[0061] After the defect is located, a test report is generated. The test report records the specific location and type of the defect in detail and provides it to the user for further analysis and processing. The test report usually includes the following: identification information of the workpiece, location coordinates of the defect, type of defect (such as cracks, pits, scratches, etc.), severity of the defect, and other relevant information. This information is presented in the form of charts, text descriptions, and schematic diagrams, so that users can intuitively understand the defect situation. The test report can also include the overall quality assessment results of the workpiece to provide guidance for subsequent workpiece repair and quality control.

[0062] By generating detailed inspection reports, users can quickly locate and repair defects on workpieces, thereby improving production efficiency and product quality. The data in the report can also be used for process improvement and quality management, helping companies to continuously optimize the production process.

[0063] Embodiment 2

[0064] In this embodiment, the semantic vector is further optimized using a particle swarm optimization algorithm: initializing the particle swarm: taking the optimal individual in the genetic algorithm stage as the initial particle swarm of PSO; updating the particle speed and position according to the particle's current position and speed update formula; calculating the fitness value of each particle, updating the individual optimal value and the global optimal value; iterating until the fitness function converges to the optimal solution; alternating iterations between the genetic algorithm and the particle swarm optimization algorithm to optimize the semantic vector.

[0065] The speed and position update formula in the particle swarm optimization algorithm is:

[0066] v i (t+1)=ωv i (t)+c1r1(p i -x i (t))+c2r2(gx i (t))

[0067] x i (t+1)=x i (t)+v i (t+1)

[0068] Among them, ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, and pi is the individual optimal position of the particle, and g is the global optimal position.

[0069] Through the speed update formula, the position and speed of the particle will be adjusted according to its own experience and the group experience, so as to search efficiently in the solution space. The new position of the particle is determined by the current position and the updated speed, so that each particle will move in the solution space according to the change of speed, gradually approaching the optimal solution. In this way, the particle swarm can balance between the global optimum and the local optimum, which not only maintains the diversity of the search, but also speeds up the convergence speed.

[0070] After each update, the fitness value of each particle is calculated, and the individual optimal value and the global optimal value are updated. The fitness function is also used to evaluate the accuracy of each semantic vector. The individual optimal position is updated by comparing the fitness value of the current particle with its historical optimal fitness value; the global optimal position is updated by comparing the fitness values ​​of all particles. This process is iterated until the fitness function converges to the optimal solution, that is, the fitness value no longer increases significantly after multiple iterations.

[0071] In the whole process, genetic algorithm and particle swarm optimization algorithm can be carried out alternately. Genetic algorithm provides global search capability, and particle swarm optimization algorithm provides fast local optimization capability. The combination of the two makes semantic vector optimization more efficient and accurate.

[0072] Through further optimization by the particle swarm optimization algorithm, the accuracy and reliability of the semantic vector are improved, providing more accurate input for the subsequent defect location steps.

[0073] Embodiment 3

[0074] In this embodiment, the geometric and process information of the workpiece is combined to further achieve accurate positioning of defects.

[0075] The geometric information of a workpiece refers to the shape, size, surface features and other geometric properties of the workpiece. The geometric information can be obtained through CAD models, design drawings or measured data. The geometric information includes: the length, width, height, curved surface and edge of the workpiece; the curvature, concave and convex points, holes and other geometric details of the workpiece surface.

[0076] The process information refers to the process parameters and process routes used in the manufacturing process of the workpiece. The process information includes: the processing method of the workpiece, such as milling, drilling, stamping, welding; specific processing parameters, such as feed speed, tool type, cutting depth; process route: the processing sequence of the workpiece and the specific content of each process.

[0077] By combining the recognition results, geometric information and process information, defects on the workpiece surface can be located more accurately. The specific implementation steps are as follows:

[0078] Matching recognition results with geometric information: The recognition results provide the approximate location and characteristics of the defects (such as cracks, pits, etc.);

[0079] Combined with the geometric information of the workpiece, the specific location of the defect on the workpiece surface can be determined. For example, a crack may be located on a specific surface or edge.

[0080] Analyze the cause of defects by combining process information: Understanding process information can help analyze the cause of defects. For example, a crack may be caused by thermal stress generated during the welding process.

[0081] By combining process parameters and process routes, the accuracy of defect locations can be further verified and other potential defect areas can be analyzed.

[0082] Combine geometric and process information to mark the specific location and characteristics of defects on the CAD model or design drawing of the workpiece. Generate a detailed defect location report containing the precise coordinates, type, size and cause analysis of the defect.

[0083] Through the above process, combined with the recognition results, geometric information and process information, the defects can be accurately located, and the accuracy and efficiency of defect detection and quality control can be improved.

[0084] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A defect location method based on a language model, characterized in that: The following steps are involved: S100, data preprocessing step: acquiring an image of the workpiece surface and performing preprocessing; S200, preliminary positioning step: inputting the preprocessed image data into the preliminary positioning model to obtain a preliminary positioning result; S300, language model training step: generating a semantic description based on the image features extracted by CNN; querying historical images based on the preliminary positioning results, and generating a target semantic description based on the historical image features extracted by CNN; S400, semantic vector generation step: converting the semantic description into a semantic vector, and converting the target semantic description into a target semantic vector; S500, genetic algorithm optimization step: optimizing the semantic vector; S600, defect location step: determine the location of the workpiece defect by using the optimized semantic vector, obtain the current location result, compare the current location result with the preliminary location result, and when the overlap meets the threshold, use the current location result as the final location; When the overlap does not meet the threshold, the current positioning result is added to the training set of the preliminary positioning model to retrain the model, and the preliminary positioning result is used as the final position.

2. The defect localization method based on language model according to claim 1, characterized in that: The step S100 includes: acquiring a high-resolution image of the workpiece surface; graying the image; removing noise using Gaussian filtering; and performing edge detection using Canny edge detection.

3. The defect localization method based on language model according to claim 2, characterized in that: The step S300 includes: extracting image features from the preprocessed image using a CNN convolutional neural network; and converting the extracted image features into semantic descriptions using a Transformer model.

4. The defect localization method based on language model according to claim 3 is characterized in that: The step S400 includes: using the BERT model to convert the generated semantic description into a semantic vector representation, and converting the target semantic description into a target semantic vector representation.

5. The defect localization method based on language model according to claim 4, characterized in that: The step S500 includes: generating an initial semantic vector population; selecting semantic vectors with high fitness for reproduction; generating a new generation of semantic vectors through crossover and mutation operations; defining a fitness function to evaluate the accuracy of each semantic vector; and iterating selection, crossover and mutation operations for evolution.

6. The defect localization method based on language model according to claim 5, characterized in that: The fitness function is defined as: Among them, v i Semantic vector, is the target semantic vector.

7. The defect localization method based on language model according to claim 1, characterized in that: It also includes further optimization of semantic vectors using particle swarm optimization algorithm: The optimal individual in the genetic algorithm stage is used as the initial particle swarm of the particle swarm optimization algorithm; the particle speed and position are updated according to the current position and speed update formula of the particle; the fitness value of each particle is calculated, and the individual optimal value and the global optimal value are updated; it is iterated until the fitness function converges to the optimal solution; it is iterated alternately between the genetic algorithm and the particle swarm optimization algorithm to optimize the semantic vector.

8. The defect localization method based on language model according to claim 7, characterized in that: The speed and position update formula in the particle swarm optimization algorithm is: v i (t+1)=ωv i (t)+c1r1(p i -x i (t))+c2r2(g-x i (t)) x i (t+1)=x i (t)+v i (t+1) Among them, ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, and p i is the individual optimal position of the particle, and g is the global optimal position.

9. The defect localization method based on language model according to claim 1, characterized in that: It also includes manual verification of the generated semantic descriptions to improve the accuracy of the semantic vectors.

10. A defect localization system based on a language model, characterized in that: The system uses the language model-based defect localization method as described in any one of claims 1-9.

Images