Multi-light-source part surface defect detection method, device and equipment and storage medium

Abstract

The invention relates to an artificial intelligence technology, and discloses a multi-light-source part surface defect detection method and device, equipment and a storage medium. The method comprises the steps that a to-be-detected object is subjected to multi-angle irradiation, camera shooting operation is conducted on the to-be-detected object in the irradiation state, an object image set is obtained, and each object image in the object image set is marked with a defect text expression and a light source array corresponding to a multi-angle dome light source; and performing feature extraction operation on the object image set by using a multi-light-source surface defect recognition model to obtain component texture features, text features and light source array features, and performing surface defect recognition operation on the component texture features, the text features and the light source array features to obtain object defects of the to-be-detected object. The problem that a traditional defect detection method is difficult to utilize light source parameter information in a complex illumination scene can be solved, and the part surface defect recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and computer-readable storage medium for detecting surface defects of multi-light source components. Background Art

[0002] Due to the uncontrollability of industrial equipment and production environment, defects will inevitably appear on the surface of products, affecting manufacturing efficiency and economic benefits. Surface defect detection (SDD) identifies defective areas in industrial product images through pixel-level prediction methods, and has become an indispensable part of modern production lines. In most current industrial defect detection scenarios, in order to achieve automation and high-precision and efficiency requirements, most design schemes still use conventional image segmentation and target detection technology. First, after investigating the actual production site equipment and production line conditions, as well as the structure and surface texture of the object to be tested, a corresponding lighting scheme is formulated, that is, the corresponding image acquisition equipment is selected. Then the selected optical equipment and image acquisition equipment are deployed to the production line, and lighting and image acquisition are performed according to the communication protocol between the equipment. According to the established defect definition and operation standards, the defect annotation of the image is performed to obtain a defect data set. Finally, after model training and optimization, the obtained target model will be deployed on the production line for real-time detection and output of the detection results.

[0003] However, most traditional visual neural networks are currently designed for a single image detection task, and their performance in complex light source defect detection scenarios is still limited. Therefore, it is particularly necessary to introduce the light source information in complex light source defect detection scenarios into the design of the visual neural network architecture. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for detecting surface defects of multi-light source components, the main purpose of which is to solve the problem that traditional defect detection methods are difficult to use light source parameter information in complex lighting scenes.

[0005] To achieve the above object, the present invention provides a method for detecting surface defects of multi-light source components, comprising:

[0006] The object to be inspected is illuminated by a pre-built multi-angle dome light source, and the object to be inspected is photographed by a pre-built camera device under the illumination state to obtain a set of object images, wherein each object image in the set of object images is marked with a defect text description and a light source array corresponding to the multi-angle dome light source;

[0007] Using a pre-trained multi-light source surface defect recognition model, a feature extraction operation is performed on the object image set to obtain component texture features, text features and light source array features, and a surface defect recognition operation is performed on the component texture features, text features and light source array features to obtain object defects of the object to be detected.

[0008] Optionally, before using the pre-trained multi-light source surface defect recognition model, the method further includes:

[0009] Acquire a multi-light source image sample set, wherein the multi-light source image sample set includes a multi-light source sample image, a defect mask, a defect description label, and a light source array label;

[0010] Obtain a multi-light source surface defect recognition model including an image encoder, a text prompt encoder, and a mask decoder;

[0011] A pre-trained target segmentation and recognition model is obtained, and network model parameters of the target segmentation and recognition model are obtained; based on the network model parameters, transfer learning is performed on the multi-light source surface defect recognition model using the multi-light source image sample set to obtain a fine-tuned multi-light source surface defect recognition model.

[0012] Optionally, acquiring a multi-light source image sample set includes:

[0013] Acquire a pre-built sample object set, wherein each sample object in the sample object set has manually marked real defect description text;

[0014] Obtaining a sample object from the sample object set in turn, and identifying the size of the sample object;

[0015] Using a pre-constructed light intensity distribution table, querying the number of light paths corresponding to the size, and configuring the multi-angle dome-type light source using the number of light paths to obtain a configured multi-angle dome-type light source;

[0016] The sample object is illuminated by using the configured multi-angle dome-shaped light source, and the illumination process is adjusted according to a preset optimal light source monitoring strategy to obtain an optimal illumination angle set;

[0017] Acquire the light source array information corresponding to the optimal illumination angle set to obtain the optimal light source array set;

[0018] According to the optimal light source array set, the sample object is illuminated and photographed to obtain a multi-light source image sample subset, wherein each multi-light source image sample in the multi-light source image sample subset is marked with the real defect description text and the optimal light source array corresponding to the multi-light source image sample;

[0019] A subset of multi-light source image samples corresponding to all sample objects in the sample object set is output to obtain a multi-light source image sample set, and defect masks are configured for the multi-light source images in the multi-light source image sample set according to the real defect description text.

[0020] Optionally, after obtaining the multi-light source image sample set, the method further includes:

[0021] According to a preset data enhancement strategy, a data set preprocessing operation is performed on the multi-light source image sample set to obtain a preprocessed data set;

[0022] According to a preset optimization strategy, the preprocessed data set is configured using a pre-built optimizer to obtain a multi-light source image sample set to be trained.

[0023] Optionally, the step of acquiring a multi-light source surface defect recognition model including an image encoder, a text prompt encoder, and a mask decoder includes:

[0024] Build an image encoder using the pre-built patch embedding layer, position embedding layer, transformer encoder layer, and neck layer.

[0025] Build a text prompt encoder using the pre-built pre-processing layers, input embedding layers, transformer encoder layers, and output embedding layers.

[0026] Constructing a converter block using pre-built bidirectional attention blocks and multi-head attention, constructing an upsampling block using pre-built deconvolution layers, GELU activation and normalization layers, constructing an output super network block using a preset number of standard multi-layer perceptron layers, and constructing a prediction block using one of the standard multi-layer perceptron layers;

[0027] Constructing a mask decoder using the transformer block, the upsampling block, the output supernetwork block, and the prediction block;

[0028] A multi-light source surface defect recognition model is constructed based on the image encoder, text prompt encoder and mask decoder.

[0029] Optionally, performing transfer learning on the multi-light source surface defect recognition model using the multi-light source image sample set according to the network model parameters to obtain a fine-tuned multi-light source surface defect recognition model includes:

[0030] According to a preset structural training strategy, a training target of the multi-light source surface defect recognition model is identified, and an adapter is configured for the training target to obtain an adaptation model;

[0031] Using the multi-light source image sample set, and according to the network model parameters, performing a training operation based on a training target on the adaptation model;

[0032] When the image encoder, the text prompt encoder and the mask decoder are all trained, a trained multi-light source surface defect recognition model is obtained.

[0033] Optionally, configuring an adapter for the training target includes:

[0034] When the training target is the image encoder, deploying two adapters on each pre-built transformer encoder in the image encoder;

[0035] When the training target is the text prompt encoder, deploying two adapters on each pre-built transformer encoder in the text prompt encoder;

[0036] When the training target is the mask decoder, three adapters are deployed on the first dual-head attention block on each pre-built transformer encoder in the mask decoder.

[0037] In order to solve the above problems, the present invention also provides a multi-light source component surface defect detection device, the device comprising:

[0038] An image acquisition module, used to illuminate the object to be detected by using a pre-built multi-angle dome light source, and to photograph the object to be detected by using a pre-built camera device under the illumination state, to obtain a set of object images, wherein each object image in the set of object images is marked with a light source array corresponding to the multi-angle dome light source;

[0039] The defect recognition module is used to use a pre-trained multi-light source surface defect recognition model to perform a feature extraction operation on the object image set to obtain component texture features and light source array features, and to perform a surface defect recognition operation on the component texture features and light source array features to obtain the object defects of the object to be detected.

[0040] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0041] at least one processor; and,

[0042] a memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned multi-light source component surface defect detection method.

[0044] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned multi-light source component surface defect detection method.

[0045] The encoder-decoder structure designed according to the multimodal task in the embodiment of the present invention introduces different modal information such as defect text description and light source array parameters into the defect detection task of the traditional single visual modality, thereby realizing comprehensive acquisition and in-depth mining of surface defect features under different lighting conditions in the detection scene. Multimodal feature fusion not only enhances the detection sensitivity of the model to defects with different characteristics, but also ensures the stable performance of the model output under complex lighting conditions; in addition, the system designs an integrated surface defect detection system based on a multi-angle dome light source. Compared with the traditional serial detection equipment with multiple machines and multiple light sources on the production line, the present invention adopts a single multi-angle dome light source to design a lighting system with multi-angle and real-time light source array parameter changes, so that the surface defect detection task of objects with different structural features can be completed on a single machine; in addition, the multi-light source surface defect recognition model in the present invention is fine-tuned based on the pre-trained model and the inserted adapter module in the large model framework target segmentation recognition model, and at the same time, according to the predefined defect text description and light source array parameters, more accurate defect detection and classification can be achieved, and multi-modal customized defect online detection can be achieved. There is no need to retrain the model. By modifying the predefined defect text description and light source array parameters, new defects can be detected, which greatly reduces the labor cost of sample calibration and the computing power cost of model training. Therefore, a multi-light source component surface defect detection method, device, equipment and storage medium provided by the embodiment of the present invention can solve the problem that traditional defect detection methods are difficult to use light source parameter information in complex lighting scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a process flow of a multi-light source component surface defect detection method provided by an embodiment of the present invention;

[0047] Figure 2 This is a distribution diagram of the LED array of the 48-channel multi-angle dome light source provided by the present invention.

[0048] Figure 3 The specific structure and insertion position diagram of the Adapter module provided by the present invention.

[0049] Figure 4 A structural diagram of a multi-modal industrial parts surface defect image detection system based on a multi-angle dome light source provided by the present invention;

[0050] Figure 5 A functional module diagram of a multi-light source component surface defect detection device provided by an embodiment of the present invention;

[0051] Figure 6 A schematic diagram of the structure of an electronic device for implementing the multi-light source component surface defect detection method provided by an embodiment of the present invention;

[0052] Figure 7 This is an overall framework diagram of the multimodal industrial parts surface defect image segmentation model provided by the present invention.

[0053] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0055] The embodiment of the present application provides a method for detecting surface defects of multi-light source components. In the embodiment of the present application, the execution subject of the method for detecting surface defects of multi-light source components includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for detecting surface defects of multi-light source components can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0056] Embodiment 1:

[0057] Reference Figure 1 FIG. 1 is a flow chart of a method for detecting surface defects of a multi-light source component provided by an embodiment of the present invention. In this embodiment, the method for detecting surface defects of a multi-light source component includes:

[0058] S1. Using a pre-built multi-angle dome light source to illuminate an object to be inspected, and using a pre-built camera device to photograph the object to be inspected under the illumination state to obtain an object image set, wherein each object image in the object image set is marked with a defect text description and a light source array corresponding to the multi-angle dome light source.

[0059] The multi-angle dome light source is a light source composed of 36 or 48 LED arrays, and the number of LED arrays is selected according to the size of the object to be measured. Figure 2 The array distribution diagram of the 48-way multi-angle dome light source is shown. Each LED array has 4000 adjustable brightness levels, represented by hexadecimal characters. Here, taking the 48-way LED array as an example, the state of each array can be represented by 3 characters, and the entire multi-angle dome light source consists of 144 characters. For example, when a certain array is turned off, the state is '000', and when the brightness is 4000 levels, it is 'FA0'.

[0060] Specifically, in the embodiment of the present invention, the stereoscopic light source of this embodiment can adopt an editable multi-angle dome light source for defect detection provided by Guangzhou Xuan*Jia Electronic Technology Co., Ltd., wherein the whole set of light sources includes a light source controller and a set of multi-angle dome light sources. The light source controller can pre-store the light source array parameters, which is convenient for realizing real-time control of the multi-angle dome light source on the production line. The light source controller realizes communication with external devices through the RS232 protocol, and the 36 or 48-channel LED array in the multi-angle dome light source is attached to the inner wall of the multi-angle dome structure. Each array has 4000 brightness levels, and multi-angle and multi-brightness dimming can be realized by adjusting the brightness of each array, so that defects with different characteristics on the surface of the object are presented.

[0061] The industrial camera of this embodiment can adopt the *Convision (MV-CS060-10GC-PRO) industrial camera, which has a GigE interface and meets the system's Gigabit Ethernet communication protocol. The lens adopts MVL-HF5028M-6MPE. Both have 6 million pixels, a resolution of 3072×2048, and a frame rate of 30.7fps, which can meet the needs of most defect detection scenarios of the present invention.

[0062] In the embodiment of the present invention, the object to be detected is illuminated by the above-mentioned device, and the object to be detected is photographed by a pre-built camera device under the illumination state to obtain a set of object images.

[0063] Wherein, each object image in the object image set is marked with a shooting angle and a light source array corresponding to the multi-angle dome light source.

[0064] S2. Using a pre-trained multi-light source surface defect recognition model, perform a feature extraction operation on the object image set to obtain component texture features, text features and light source array features, and perform a surface defect recognition operation on the component texture features, text features and light source array features to obtain object defects of the object to be detected.

[0065] The full name of the multi-light source surface defect recognition model is Segment Anything Model in Surface Defect (SDSAM), which consists of an image encoder, a text prompt encoder, and a lightweight mask decoder. It also includes multi-modal input methods such as image data, text, and light source array parameters. The specific network framework is as follows: Figure 7 The multi-light source surface defect recognition model is used to obtain the surface image of the object to be detected according to the preset light source array parameters, the surface image is input into the image encoder to obtain image embedding, the defect description text and the light source array parameters are input into the text prompt encoder to obtain text embedding, and finally the image embedding and the text embedding are input into the lightweight mask decoder to obtain the defect segmentation result.

[0066] In detail, in the embodiment of the present invention, before using the pre-trained multi-light source surface defect recognition model, the method further includes:

[0067] Acquire a multi-light source image sample set, wherein the multi-light source image sample set includes a multi-light source sample image, a defect mask, a defect description label, and a light source array label;

[0068] Obtain a multi-light source surface defect recognition model including an image encoder, a text prompt encoder, and a mask decoder;

[0069] A pre-trained target segmentation and recognition model is obtained, and network model parameters of the target segmentation and recognition model are obtained; based on the network model parameters, transfer learning is performed on the multi-light source surface defect recognition model using the multi-light source image sample set to obtain a fine-tuned multi-light source surface defect recognition model.

[0070] In an embodiment of the present invention, a multi-light source image sample set is first obtained, then a multi-light source surface defect recognition model is obtained, and then a target segmentation recognition model is used. After the multi-light source surface defect recognition model is preliminarily trained with the target segmentation recognition model, the multi-light source image sample set is used for fine-tuning training to obtain a trained multi-light source surface defect recognition model.

[0071] Specifically, in the embodiment of the present invention, since the training samples are composed of multi-light source sample images, defect masks, defect description labels, and light source array labels. In the process of acquiring defect images, the multi-angle dome light source will automatically switch the previously set N light source array parameters, and the multi-angle dome light source controller will hard-trigger the industrial camera through the IO port to realize taking pictures. After each switching of the light source array, a signal is sent to the camera to obtain a picture, and a group of N defect images are taken at the target angle of the same object to be tested at one time. After completing a group of light source array parameter photography, the angle to be tested of the object to be tested is adjusted, and the acquisition operation is repeated until all images are acquired. When annotating the defect image, the pixel-level mask mark is annotated by the semi-automatic annotation function of the target segmentation and recognition model (SAM), and the mask mark obtained by annotation is saved as a grayscale image in PNG format; the text mark is used to describe the surface defects of various objects to be detected, which can be described from the aspects of the morphology, color, position, etc. of the surface defects. Taking the detection of surface defects of linear guide rails as an example, the text mark of the defect description can be established in the following way. For example, the type of defects such as bumps can be described as: diamond-shaped bumps, long strips of bumps, center area bumps, black and silver deep bumps, etc.

[0072] In detail, in the embodiment of the present invention, the step of obtaining a multi-light source image sample set includes:

[0073] Acquire a pre-built sample object set, wherein each sample object in the sample object set has manually marked real defect description text;

[0074] Obtaining a sample object from the sample object set in turn, and identifying the size of the sample object;

[0075] Using a pre-constructed light intensity distribution table, querying the number of light paths corresponding to the size, and configuring the multi-angle dome-type light source using the number of light paths to obtain a configured multi-angle dome-type light source;

[0076] The sample object is illuminated by using the configured multi-angle dome-shaped light source, and the illumination process is adjusted according to a preset optimal light source monitoring strategy to obtain an optimal illumination angle set;

[0077] Acquire the light source array information corresponding to the optimal illumination angle set to obtain the optimal light source array set;

[0078] According to the optimal light source array set, the sample object is illuminated and photographed to obtain a multi-light source image sample subset, wherein each multi-light source image sample in the multi-light source image sample subset is marked with the real defect description text and the optimal light source array corresponding to the multi-light source image sample;

[0079] A subset of multi-light source image samples corresponding to all sample objects in the sample object set is output to obtain a multi-light source image sample set, and defect masks are configured for the multi-light source images in the multi-light source image sample set according to the real defect description text.

[0080] The light intensity distribution table and the optimal light source monitoring strategy are empirical rules for technicians to perform object inspection.

[0081] Specifically, in an embodiment of the present invention, when setting the light source array parameters, first, according to the light intensity distribution table, the object to be tested is fixed within the lighting range of the multi-angle dome light source (36 channels or 48 channels), and the light distance between the multi-angle dome light source and the object is appropriately adjusted and recorded according to the actual lighting effect and the optimal light source monitoring strategy; then, the exposure parameters of the camera are adjusted and fixed according to the appearance effect of the object defects; finally, according to the characteristics of the object defects, several light source array parameters for showing the defects are set, and these array parameters are saved to obtain the best light source array set. For example, the defect appearance effect is better when a low-angle light source array is set for scratches on the linear guide and a high-angle light source array is set for bumps.

[0082] In addition, when setting the array parameters, the placement angle of the object needs to be appropriately adjusted to ensure that the defect is visible in these light source array parameters at any angle when it appears on the surface of the object.

[0083] Furthermore, in an embodiment of the present invention, after obtaining the multi-light source image sample set, the method further includes:

[0084] According to a preset data enhancement strategy, a data set preprocessing operation is performed on the multi-light source image sample set to obtain a preprocessed data set;

[0085] According to a preset optimization strategy, the preprocessed data set is configured using a pre-built optimizer to obtain a multi-light source image sample set to be trained.

[0086] Specifically, in an embodiment of the present invention, the data set includes a defect image, a defect mask, a text label, and light source array parameters;

[0087] The data enhancement strategy includes but is not limited to formatting, cropping, random flipping, random resizing, normalization, etc.

[0088] In addition, you can also set the optimization strategy: use Adamw as the optimizer; use MSELoss as the loss function, use the cosine annealing strategy as the learning rate strategy, and optimize the configuration to improve the subsequent training efficiency.

[0089] In the embodiment of the present invention, a multi-light source image sample set that can be used for training is collected according to the above steps. Next, a multi-light source surface defect recognition model is constructed.

[0090] In detail, in an embodiment of the present invention, the step of obtaining a multi-light source surface defect recognition model including an image encoder, a text prompt encoder, and a mask decoder includes:

[0091] Build an image encoder using the pre-built patch embedding layer, position embedding layer, transformer encoder layer, and neck layer.

[0092] Build a text prompt encoder using the pre-built pre-processing layers, input embedding layers, transformer encoder layers, and output embedding layers.

[0093] Constructing a converter block using pre-built bidirectional attention blocks and multi-head attention, constructing an upsampling block using pre-built deconvolution layers, GELU activation and normalization layers, constructing an output super network block using a preset number of standard multi-layer perceptron layers, and constructing a prediction block using one of the standard multi-layer perceptron layers;

[0094] Constructing a mask decoder using the transformer block, the upsampling block, the output supernetwork block, and the prediction block;

[0095] A multi-light source surface defect recognition model is constructed based on the image encoder, text prompt encoder and mask decoder.

[0096] Specifically, in an embodiment of the present invention, the image encoder is a standard VIT (Vision Transformer) network, and VIT is composed of a Patch Embedding layer (i.e., the patch embedding layer), a Positional Embedding layer (i.e., the position embedding layer), a Transformer Encoder layer (i.e., the transformer encoder layer) and a Neck layer (i.e., the neck layer). The Patch Embedding layer divides the input image two-dimensional matrix vector into several sequence blocks, and then flattens it into a one-dimensional vector after dimension mapping; the Positional Embedding layer is responsible for attaching a position embedding representing its relative position in the original image matrix to each flattened one-dimensional feature vector; the feature vector preprocessed by the first two layers is fed as input to a series of Transformer Encoder layers for encoding. Each Transformer Encoder layer contains a multi-head attention mechanism (Multi-Head Attention) and a feed-forward neural network (Feed-Forward Network). At the same time, there is a layer normalization (Layer Normalization) between each Transformer Encoder layer to maintain the consistency of feature distribution; the main function of the Neck layer is to reduce the dimension or adjust the features output from a series of Transformer Encoder layers in order to better meet the needs of image segmentation tasks in lightweight mask decoders; it is responsible for further processing and integration of the features extracted by the Transformer Encoder layer, and consists of two groups of convolutional layers and regularization layers to enhance the expressiveness of features.

[0097] Specifically, in an embodiment of the present invention, the text encoder is a text encoder in a standard CLIP (Contrastive Language-Image Pre-Training) model, which consists of a pre-processing layer, an input embedding layer, a TransformerEncoder layer, and an output embedding layer.

[0098] The pre-processing layer consists of two parts: Tokenization and AST (Adding Special Tokens); Tokenization is mainly responsible for segmenting the input natural language text into a series of tokens (markers, here refers to noun phrases). The n-gram algorithm is used in the example to extract noun phrases. This method is based on a statistical language model and can identify common words or phrases in the text. After word segmentation, the model will map each word to a unique ID based on a preset vocabulary; AST will insert special markers at the beginning and end of the text, such as [CLS] is used to indicate the beginning of a sentence, and [SEP] is used to indicate the end of a sentence. Through these markers, the model can better understand the boundaries of the text.

[0099] The input embedding layer consists of two parts: TokenEmbedding and PositionEmbedding. TokenEmbedding maps each token to the corresponding word embedding vector. This is achieved by querying the embedding matrix. Each token is projected into a high-dimensional space by the embedding matrix. The size of the embedding matrix is ( size vocab ,dim embedding ) , where size vocab is the size of the vocabulary, dim embedding is the dimension of the embedding vector. PositionEmbedding adds position encoding to each token so that the model can identify the position of each token in the sequence.

[0100] The TransformerEncoder layer consists of a multi-head attention mechanism and a feed-forward neural network. The multi-head attention mechanism consists of three parts: Linear Transformations, Scaled Dot-ProductAttention, and Concatenation and LinearTransformation; while the feed-forward neural network usually consists of two linear transformation layers and a nonlinear activation function (such as ReLU).

[0101] The output embedding layer usually takes the embedding vector of the [CLS] tag as the representation of the entire text and performs L2 regularization. The specific formula is as follows. The embedding vector is considered to contain the semantic information of the entire sentence.

[0102]

[0103] The output embedding layer usually takes the embedding vector of the [CLS] tag as the representation of the entire text and performs L2 regularization. The specific formula is as follows. The embedding vector is considered to contain the semantic information of the entire sentence.

[0104] The input of the text encoder includes text tags and light source array parameters. Considering the structural particularity of the light source array parameters and the inconsistent correlation ratio of each light source angle in each defect, this example designs a light source array adaptive extraction module (LAE) to calculate the weight parameter ω of each light source array. The specific calculation is as follows:

[0105] ω=σ(linear 2 (ReLU(linear 1 (GAP(F t )))))

[0106] GAP is the global average pooling, which is responsible for transforming the text feature F t From the vocabulary size vocab Converted to the light source array parameter length 3N, σ is the Sigmoid activation function, which can limit the weight parameter between 0 and 1. ω is multiplied by the light source array parameter and the text feature F t Together as the input of the text encoder, the output text of the text encoder is finally obtained as text embedding embedding .

[0107] Specifically, in the embodiment of the present invention, the lightweight mask decoder is mainly composed of a Transformer module, an output_upscaling module, an output_hypernetworks_mlps module and an iou_prediction_head module.

[0108] The Transformer module consists of multiple repeatedly stacked TwoWayAttentionBlocks and 1 Multi-HeadAttention, which embeds the image output by the image encoder into image embedding and the text embedding text output by the text encoder embedding Perform feature fusion and obtain the rough mask src and the processed text embedding hs. The formula is as follows:

[0109] src,hs=Transformer(image embedding ,text embedding )

[0110] The output_upscaling module consists of two deconvolutions, two GELU activations, and a LayerNorm, which implements a four-fold upsampling operation on the coarse mask src in this module to get the image embedding upscaled embedding , the formula is as follows:

[0111] upscaled embedding =GELU(ConvT(GELU(LN(ConvT(src)))))

[0112] The output_hypernetworks_mlps module consists of 4 standard MLPs, which transforms hs into a hyper-embedding by calculating weighted weights in , the specific formula is as follows:

[0113] hyper in =MLP (4) (hs)

[0114] The iou_prediction_head module consists of a standard MLP to predict the iou score. The specific formula is as follows:

[0115] io pred =MLP(hs)

[0116] Finally, hyper embed in and the image is embedded upscaled embedding Perform matrix multiplication to obtain the predicted mask result mask:

[0117] io pred =hyper in Upscaled embedding

[0118] In the embodiment of the present invention, a multi-light source surface defect recognition model is constructed according to the above structure, and then training is performed.

[0119] In detail, in an embodiment of the present invention, the multi-light source surface defect recognition model is transferred and learned using the multi-light source image sample set according to the network model parameters to obtain a fine-tuned multi-light source surface defect recognition model, including:

[0120] According to a preset structural training strategy, a training target of the multi-light source surface defect recognition model is identified, and an adapter is configured for the training target to obtain an adaptation model;

[0121] Using the multi-light source image sample set, and according to the network model parameters, performing a training operation based on a training target on the adaptation model;

[0122] When the image encoder, the text prompt encoder and the mask decoder are all trained, a trained multi-light source surface defect recognition model is obtained.

[0123] In the embodiment of the present invention, the step of configuring the adapter for the training target includes:

[0124] When the training target is the image encoder, deploying two adapters on each pre-built transformer encoder in the image encoder;

[0125] When the training target is the text prompt encoder, deploying two adapters on each pre-built transformer encoder in the text prompt encoder;

[0126] When the training target is the mask decoder, three adapters are deployed on the first dual-head attention block on each pre-built transformer encoder in the mask decoder.

[0127] Specifically, in this embodiment, the method is to fine-tune the SAM architecture for the task of industrial parts surface defect image segmentation to improve the quality of industrial products.

[0128] Instead of completely changing all parameters when training the image encoder and text encoder of SDSAM, we freeze the pre-trained SAM parameters and insert the adapter module Adapter at a specific location in the architecture. Figure 3 As shown:

[0129] In the image encoder and text encoder, two Adapters are deployed in each standard TransformerEncoder in Encoder_Block. For a standard Transformer Encoder, the first Adapter is placed after the multi-head attention and before the residual connection; the second Adapter is in the residual path of the MLP layer after the multi-head attention, and is connected to a scaling block to scale the image embedding.

[0130] At the same time, the pre-trained SAM parameters are frozen when training the mask decoder, and three Adapters are deployed in the first TwoWayAttentionBlock in the Transformer module. The first Adapter is deployed after the first multi-head cross attention in the TwoWayAttentionBlock, and the light source array parameter part of the text embedding is added. Here, a down-projection is added to compress the text embedding and added to the embedded Adapter before the ReLU layer. Compared with the previous modifications, this helps the Adapter adjust parameters according to the light source array information, and is more flexible and general for different modalities and downstream mask prediction tasks. The second Adapter is still in the residual path of the MLP layer after the multi-head attention, and a scaling block is connected to scale the image embedding. The third Adapter is deployed after the residual connection to prompt the cross attention. After the Adapter calculation, the residual connection and layer normalization connection are performed, and the final result is output.

[0131] The Adapter is composed of three parts: down-projection, ReLU activation and up-projection. The specific structure and insertion position are as follows: Figure 4 As shown in the figure, the down-projection uses a simple MLP layer to compress the given embedding to a smaller dimension; the up-projection uses another MLP layer to expand the compressed embedding back to its original dimension; and the activation function uses the relu activation function.

[0132] According to the above operations, the model is trained, and then the model quality is evaluated and the subsequent optimization strategy is formulated through evaluation indicators, including but not limited to IoU (intersection over union ratio), TPR (detection rate), FPR (false positive rate), FNR (missed positive rate), etc. Thus, a trained multi-light source surface defect recognition model is obtained.

[0133] Furthermore, the multi-light source surface defect recognition model completed through training in the embodiment of the present invention performs forward calculation on the object image set network to obtain the object defects of the object to be detected.

[0134] Specifically, in the embodiment of the present invention, Figure 4 (Left) shows that it also includes computing equipment, which can be composed of a host equipped with Intel Core(TM)I9-13900xCPU@5.8GHz (running memory 32GB) and NVIDIA GeForce RTX-4090 graphics processing unit (GPU) (memory size is 24GB). The 2TB hard disk is used to store computer programs, defect images, multimodal model weights and production line data, and the CPU and GPU processors are used to implement the multimodal defect detection method when executing computer programs.

[0135] like Figure 4 As shown in (middle), the multimodal defect segmentation model detection module is mainly composed of the multimodal industrial parts surface defect image segmentation model SDSAM. The weight file obtained after training and the neural network architecture itself are used as the model body; the defect image, the preset defect text description and the light source array parameters are used as the input of the model, and the pixel-level defect mask is used as the output of the model.

[0136] like Figure 4 (right) As shown, the post-processing module of the test results mainly includes a series of post-processing logics and external communication protocols. The model will obtain a pixel-level defect mask based on each set of light source array parameters, and each test angle of the object to be tested will obtain a mask through at least one set of light source array parameters, which means that the multiple mask areas obtained will form overlapping areas. This is achieved by eliminating pixel-level uncertainty through image comparison and consistency annotation in post-processing and through OR operations, thereby reducing misjudgments and missed judgments. The external communication protocol helps the system to realize information interaction with external devices. For example, the system serves as the information source for subsequent operations such as sorting. The system includes communication protocols such as Modbus and TCP.

[0137] Embodiment 2:

[0138] The present invention takes the surface defect detection of a linear guide rail as an example, and is specifically divided into the following steps:

[0139] Step 1: Sample preparation and light source adjustment. Move the linear guide to be tested to the camera and multi-angle dome light source through the conveyor belt, adjust the object distance and lens focus knob to ensure that the details of the linear guide surface are clearly visible. The visual light source software adjusts the lighting angle by adjusting the brightness and darkness of each LED array to clearly present surface defects, and configures N optimal light source arrays to the light source controller for subsequent collection and detection.

[0140] Step 2: Collect training samples of linear guide rail defects. Set the multi-angle dome light source to the automatic light cutting mode, so that it switches one by one in the preset light source array. Control the real-time transmission of the conveyor belt, capture the surface image of each angle to be measured on the linear guide rail surface under each light source array, and save it to the computing device terminal.

[0141] Step 3: Dataset construction. Use the pre-trained SAM to annotate each linear guide surface image with pixel-level defect masks, such as scratches, dents, rust spots, and other defects. Then, use the defect mask map, defect text description, and corresponding light source array parameters of each defect image as a set of training samples to create a dataset.

[0142] Step 4: Model training. Based on the pre-trained SAM model weights and the SDSAM network architecture, input training samples in rounds for model training, and then optimize the training according to the evaluation indicators to finally obtain the SDSAM model.

[0143] Step 5: Real-time detection. After the trained SDSAM model is converted into an onnx format model and deployed on the production line, real-time detection begins: the conveyor belt delivers the linear guide to be detected to the detection area, and sends signals to the light source controller and the industrial camera through the IO port to implement the preset light source array light cutting and photography. The acquired pictures are forwarded to the SDSAM model in real time to realize real-time detection, and the detection results and sorting signals are respectively transmitted to the front-end page display and the conveyor belt for sorting operations.

[0144] Step 6: Batch inspection and report generation. Repeat step 5 until the inspection task of the production line is completed, and the system automatically generates a production line report for the user to view.

[0145] Compared with traditional industrial quality inspection technology, the present invention can effectively reduce the consumption of manpower and material resources, and through multimodal fusion detection technology, it utilizes complementary information under different light source arrays to greatly improve the adaptability, generalization and robustness of detection, showing great practical application potential.

[0146] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0147] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0148] The encoder-decoder structure designed according to the multimodal task in the embodiment of the present invention introduces different modal information such as defect text description and light source array parameters into the defect detection task of the traditional single visual modality, thereby realizing comprehensive acquisition and in-depth mining of surface defect features under different lighting conditions in the detection scene. Multimodal feature fusion not only enhances the detection sensitivity of the model to defects with different characteristics, but also ensures the stable performance of the model output under complex lighting conditions; in addition, the system designs an integrated surface defect detection system based on a multi-angle dome light source. Compared with the traditional serial inspection equipment with multiple machines and multiple light sources on the production line, the present invention uses a single multi-angle dome light source to design a lighting system with multi-angle and real-time light source array parameter changes, so that the surface defect detection task of objects with different structural features can be completed on a single machine; in addition, the multi-light source surface defect recognition model in the present invention is fine-tuned based on the pre-trained model and the inserted adapter module in the large model framework target segmentation recognition model, and at the same time, according to the predefined defect text description and light source array parameters, more accurate defect detection and classification can be achieved, and multi-modal customized defect online detection can be achieved. There is no need to retrain the model. By modifying the predefined defect text description and light source array parameters, new defects can be detected, which greatly reduces the labor cost of sample calibration and the computing power cost of model training. Therefore, a multi-light source component surface defect detection method provided by an embodiment of the present invention can solve the problem that traditional defect detection methods are difficult to use light source parameter information in complex lighting scenes.

[0149] like Figure 5 , is a functional module diagram of a multi-light source component surface defect detection device provided by an embodiment of the present invention.

[0150] The multi-light source component surface defect detection device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the multi-light source component surface defect detection device 100 may include an image acquisition module 101 and a defect recognition module 102. The module of the present invention may also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, and is stored in the memory of the electronic device.

[0151] In this embodiment, the functions of each module / unit are as follows:

[0152] The image acquisition module 101 is used to illuminate the object to be inspected using a pre-built multi-angle dome light source, and to photograph the object to be inspected using a pre-built camera device under the illumination state to obtain an object image set, wherein each object image in the object image set is marked with a defect text description and a light source array corresponding to the multi-angle dome light source;

[0153] The defect recognition module 102 is used to use a pre-trained multi-light source surface defect recognition model to perform a feature extraction operation on the object image set to obtain component texture features, text features and light source array features, and to perform a surface defect recognition operation on the component texture features, text features and light source array features to obtain the object defects of the object to be detected.

[0154] In detail, each module described in the multi-light source component surface defect detection device 100 described in the embodiment of the present application is used in the same manner as described above. Figures 1 to 4 , Figure 7 The multi-light source component surface defect detection method described in the invention has the same technical means and can produce the same technical effects, so it will not be repeated here.

[0155] like Figure 6 , is a schematic structural diagram of an electronic device 1 for implementing a multi-light source component surface defect detection method provided by an embodiment of the present invention.

[0156] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a multi-light source component surface defect detection program.

[0157] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (ControlUnit) of the electronic device 1, and uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes the programs or modules stored in the memory 11 (for example, executing the multi-light source component surface defect detection program, etc.), and calls the data stored in the memory 11 to execute various functions of the electronic device and process data.

[0158] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the multi-light source component surface defect detection program, but also can be used to temporarily store data that has been output or is to be output.

[0159] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0160] The communication interface 13 is used for communication between the above-mentioned electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0161] Figure 6 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 6The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0162] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0163] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0164] The multi-light source component surface defect detection program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0165] The object to be inspected is illuminated by a pre-built multi-angle dome light source, and the object to be inspected is photographed by a pre-built camera device under the illumination state to obtain a set of object images, wherein each object image in the set of object images is marked with a defect text description and a light source array corresponding to the multi-angle dome light source;

[0166] Using a pre-trained multi-light source surface defect recognition model, a feature extraction operation is performed on the object image set to obtain component texture features, text features and light source array features, and a surface defect recognition operation is performed on the component texture features, text features and light source array features to obtain object defects of the object to be detected.

[0167] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0168] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0169] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0170] The object to be inspected is illuminated by a pre-built multi-angle dome light source, and the object to be inspected is photographed by a pre-built camera device under the illumination state to obtain a set of object images, wherein each object image in the set of object images is marked with a defect text description and a light source array corresponding to the multi-angle dome light source;

[0171] Using a pre-trained multi-light source surface defect recognition model, a feature extraction operation is performed on the object image set to obtain component texture features, text features and light source array features, and a surface defect recognition operation is performed on the component texture features, text features and light source array features to obtain object defects of the object to be detected.

[0172] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0173] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0175] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0176] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0177] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0178] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0179] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-light source component surface defect detection method, characterized in that: The method comprises: The object to be inspected is illuminated by a pre-built multi-angle dome light source, and the object to be inspected is photographed by a pre-built camera device under the illumination state to obtain a set of object images, wherein each object image in the set of object images is marked with a defect text description and a light source array corresponding to the multi-angle dome light source; Using a pre-trained multi-light source surface defect recognition model, a feature extraction operation is performed on the object image set to obtain component texture features, text features and light source array features, and a surface defect recognition operation is performed on the component texture features, text features and light source array features to obtain object defects of the object to be detected.

2. The multi-light source component surface defect detection method according to claim 1, characterized in that: Before using the pre-trained multi-light source surface defect recognition model, the method further includes: Acquire a multi-light source image sample set, wherein the multi-light source image sample set includes a multi-light source sample image, a defect mask, a defect description label, and a light source array label; Obtain a multi-light source surface defect recognition model including an image encoder, a text prompt encoder, and a mask decoder; A pre-trained target segmentation and recognition model is obtained, and network model parameters of the target segmentation and recognition model are obtained; based on the network model parameters, transfer learning is performed on the multi-light source surface defect recognition model using the multi-light source image sample set to obtain a fine-tuned multi-light source surface defect recognition model.

3. The multi-light source component surface defect detection method according to claim 2, characterized in that: The step of acquiring a multi-light source image sample set comprises: Acquire a pre-built sample object set, wherein each sample object in the sample object set has manually marked real defect description text; Obtaining a sample object from the sample object set in turn, and identifying the size of the sample object; Using a pre-constructed light intensity distribution table, querying the number of light paths corresponding to the size, and configuring the multi-angle dome-type light source using the number of light paths to obtain a configured multi-angle dome-type light source; The sample object is illuminated by using the configured multi-angle dome-shaped light source, and the illumination process is adjusted according to a preset optimal light source monitoring strategy to obtain an optimal illumination angle set; Acquire the light source array information corresponding to the optimal illumination angle set to obtain the optimal light source array set; According to the optimal light source array set, the sample object is illuminated and photographed to obtain a multi-light source image sample subset, wherein each multi-light source image sample in the multi-light source image sample subset is marked with the real defect description text and the optimal light source array corresponding to the multi-light source image sample; A subset of multi-light source image samples corresponding to all sample objects in the sample object set is output to obtain a multi-light source image sample set, and defect masks are configured for the multi-light source images in the multi-light source image sample set according to the real defect description text.

4. The method for detecting surface defects of multi-light source components according to claim 3, characterized in that: After obtaining the multi-light source image sample set, the method further includes: According to a preset data enhancement strategy, a data set preprocessing operation is performed on the multi-light source image sample set to obtain a preprocessed data set; According to a preset optimization strategy, the preprocessed data set is configured using a pre-built optimizer to obtain a multi-light source image sample set to be trained.

5. The multi-light source component surface defect detection method according to claim 4, characterized in that: The method of obtaining a multi-light source surface defect recognition model including an image encoder, a text prompt encoder and a mask decoder includes: Build an image encoder using the pre-built patch embedding layer, position embedding layer, transformer encoder layer, and neck layer. Build a text prompt encoder using the pre-built pre-processing layers, input embedding layers, transformer encoder layers, and output embedding layers. Constructing a converter block using pre-built bidirectional attention blocks and multi-head attention, constructing an upsampling block using pre-built deconvolution layers, GELU activation and normalization layers, constructing an output super network block using a preset number of standard multi-layer perceptron layers, and constructing a prediction block using one of the standard multi-layer perceptron layers; Constructing a mask decoder using the transformer block, the upsampling block, the output supernetwork block, and the prediction block; A multi-light source surface defect recognition model is constructed based on the image encoder, text prompt encoder and mask decoder.

6. The method for detecting surface defects of multi-light source components according to claim 5, characterized in that: The method of performing transfer learning on the multi-light source surface defect recognition model using the multi-light source image sample set according to the network model parameters to obtain a fine-tuned multi-light source surface defect recognition model includes: According to a preset structural training strategy, a training target of the multi-light source surface defect recognition model is identified, and an adapter is configured for the training target to obtain an adaptation model; Using the multi-light source image sample set, and according to the network model parameters, performing a training operation based on a training target on the adaptation model; When the image encoder, the text prompt encoder and the mask decoder are all trained, a trained multi-light source surface defect recognition model is obtained.

7. The method for detecting surface defects of multi-light source components according to claim 6, characterized in that: The configuring the adapter for the training target includes: When the training target is the image encoder, deploying two adapters on each pre-built transformer encoder in the image encoder; When the training target is the text prompt encoder, deploying two adapters on each pre-built transformer encoder in the text prompt encoder; When the training target is the mask decoder, three adapters are deployed on the first dual-head attention block on each pre-built transformer encoder in the mask decoder.

8. A multi-light source component surface defect detection device, characterized in that: The device comprises: An image acquisition module, used to illuminate the object to be detected by using a pre-built multi-angle dome light source, and to photograph the object to be detected by using a pre-built camera device under the illumination state, to obtain a set of object images, wherein each object image in the set of object images is marked with a defect text description and a light source array corresponding to the multi-angle dome light source; The defect recognition module is used to use a pre-trained multi-light source surface defect recognition model to perform a feature extraction operation on the object image set to obtain component texture features, text features and light source array features, and to perform a surface defect recognition operation on the component texture features, text features and light source array features to obtain the object defects of the object to be detected.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-light source component surface defect detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-light source component surface defect detection method as described in any one of claims 1 to 7 is implemented.

Images