Wheel hub defect detection method and device, electronic equipment and storage medium

Abstract

The application relates to a wheel hub defect detection method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a pre-trained defect detection model; wherein the defect detection model comprises an encoder provided with a Transformer structure, a multi-head attention mechanism is arranged in the Transformer structure, and the multi-head attention mechanism comprises initial weights; in response to a model adjustment instruction, a model adjustment plug-in is called to adjust the defect detection model; wherein the model adjustment plug-in is used for adjusting the initial weights in the multi-head attention mechanism according to the training of a small sample data set corresponding to a new process; obtaining a wheel hub image to be detected, and processing the wheel hub image to be detected through the adjusted defect detection model to obtain a defect detection result of the wheel hub image to be detected. Through the method, the problem that a detection model is difficult to quickly adapt to a new production process after the production process is improved is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a hub defect detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In the field of industrial manufacturing, especially in the production process of automobile hubs, defect detection is a key link to ensure product quality and safety. In recent years, with the continuous development of computer vision technology, detection models based on deep learning technology have shown great capabilities in industrial vision detection.

[0003] Before using the detection model for defect detection, a plurality of hub defect pictures need to be collected to train the detection model. After the detection model is trained and tested, the detection model is put into production.

[0004] However, in actual production, as the production process is improved and updated, the corresponding detection model also needs to be upgraded and iterated. Each time the detection model is upgraded, new process / new defect sample data needs to be collected to retrain the detection model. However, due to the small amount of sample data that can be obtained in the early stage of the new production process, the detection model is difficult to quickly adapt to the new production process. SUMMARY

[0005] The present application provides a hub defect detection method, device, electronic device, and storage medium to solve the technical problem that the detection model is difficult to quickly adapt to the new production process after the production process is improved and updated.

[0006] In a first aspect, the present application provides a hub defect detection method, comprising:

[0007] obtaining a pre-trained defect detection model; wherein the defect detection model comprises an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism comprises initial weights for processing input vectors into feature representations;

[0008] in response to a model adjustment instruction, calling a model adjustment plug-in to adjust the defect detection model to obtain an adjusted defect detection model; wherein the model adjustment plug-in is used to adjust at least one of the initial weights in the multi-head attention mechanism according to the training of a small sample data set corresponding to a new process;

[0009] obtaining a hub image to be detected, and processing the hub image to be detected by the adjusted defect detection model to obtain a defect detection result of the hub image to be detected.

[0010] In an embodiment of the present application, the initial weights include a weight matrix for generating a query (Q) vector, a weight matrix for generating a key (K) vector, a weight matrix for generating a value (V) vector, and / or a weight matrix for performing an output linear transformation after attention calculation in the multi-head attention mechanism.

[0011] In an embodiment of the present application, in response to a model adjustment instruction, a model adjustment plug-in is called to adjust the defect detection model to obtain an adjusted defect detection model, including:

[0012] The learnable parameters in the model adjustment plug-in are initialized;

[0013] The defect detection model calling the model adjustment plug-in is trained by using the small sample dataset to determine the learnable parameters;

[0014] The defect detection model is adjusted based on the learnable parameters and the initial weights; wherein the processing result of the multi-head attention mechanism on the input vector after adjustment is defined by the following formula:

[0015] ;

[0016] h is the processing result of the multi-head attention mechanism on the input vector, is the initial weight, x is the input vector, is an adjustment result generated by the model adjustment plug-in and the learnable parameters acting on the input vector together.

[0017] In an embodiment of the present application, the learnable parameters include a set of pre-set candidate ranks corresponding to a plurality of pairs of low-rank matrices and a gating weight corresponding to each of the candidate ranks, and the adjustment result is represented as:

[0018] ;

[0019] is the gating weight corresponding to the i-th candidate rank, is a pair of low-rank matrices corresponding to the i-th candidate rank, and a is a pre-set scaling factor.

[0020] In an embodiment of the present application, the defect detection model calling the model adjustment plug-in is trained by using the small sample dataset to determine the learnable parameters, including:

[0021] In the training iteration process, a plurality of pairs of low-rank matrices are updated; and,

[0022] respectively, and updating the gating weights by a Softmax function based on the sensitivities; wherein the sensitivity is expressed as:

[0023] ;

[0024] is the sensitivity corresponding to the i-th candidate rank, and L is the loss function.

[0025] In an embodiment of the present application, the initialization of the learnable parameters in the model adjustment plug-in includes:

[0026] the low-rank matrix in the pair of low-rank matrices corresponding to each candidate rank is initialized as a zero matrix; and the low-rank matrix in the pair of low-rank matrices corresponding to each candidate rank is initialized as a zero matrix; and

[0027] the low-rank matrix in the pair of low-rank matrices corresponding to each candidate rank is initialized as a zero matrix; and initialized by a Gaussian distribution.

[0028] In an embodiment of the present application, before the model adjustment plug-in is called to adjust the defect detection model in response to a model adjustment instruction to obtain an adjusted defect detection model, the method further includes:

[0029] obtaining the small sample dataset; wherein the small sample dataset includes X-ray images of sample hub images corresponding to a new process, and each of the sample hub images is labeled with a corresponding defect category and a defect mask;

[0030] performing image enhancement processing on the small sample dataset; wherein the image enhancement processing includes Gaussian filtering and / or wavelet transform on the sample hub images.

[0031] In a second aspect, the present application provides a wheel hub defect detection device, which includes:

[0032] a first obtaining module configured to obtain a pre-trained defect detection model; wherein the defect detection model includes an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism includes initial weights for processing input vectors into feature representations;

[0033] The calling module is configured to call the model adjustment plug-in to adjust the defect detection model in response to a model adjustment instruction, to obtain an adjusted defect detection model; wherein the model adjustment plug-in is configured to adjust at least one initial weight in the multi-head attention mechanism according to training of a small sample data set corresponding to a new process.

[0034] The first processing module is configured to obtain a to-be-detected hub image, and process the to-be-detected hub image by using the adjusted defect detection model, to obtain a defect detection result of the to-be-detected hub image.

[0035] In a third aspect, the present application provides an electronic device, comprising: at least one communication interface; at least one bus connected with the at least one communication interface; at least one processor connected with the at least one bus; and at least one memory connected with the at least one bus, wherein the processor is configured to execute the method for detecting hub defects according to any one of the embodiments of the first aspect of the present application.

[0036] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions for executing the method for detecting hub defects according to any one of the embodiments of the first aspect of the present application.

[0037] Compared with the prior art, the above technical solution provided by the embodiments of the present application has the following advantages:

[0038] The method provided by the embodiments of the present application has the following advantages: when the production process is adjusted, the model adjustment plug-in is called to adjust the defect detection model, and the adjustment is based on a small sample data set corresponding to a new process, and the adjustment is applied to at least one initial weight in the multi-head attention mechanism. After the adjustment of the defect detection model is completed, the to-be-detected hub image is processed by using the adjusted defect detection model, and the defect detection of the to-be-detected hub image is completed.

[0039] According to the technical solution provided by the embodiments of the present application, the defect detection model is updated based on a small sample, so that the defect detection model can reduce the dependence on data when adapting to a new process. At the same time, only at least one initial weight in the multi-head attention mechanism is adjusted, and the defect detection model does not need to be modified to a large extent, so that the iteration and upgrading speed of the defect detection model is accelerated. In summary, the technical solution provided by the present application can quickly adapt the defect detection model to a new production process after the production process is improved and updated. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, further serve to explain the principles of the application.

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0042] One or more embodiments are illustrated by way of example in the drawings, which are not limiting of the embodiments and which do not constitute a complete description of all of the possible embodiments. Like numbers refer to like elements throughout the drawings, and no particular element is implied to have a same reference number in different drawings unless otherwise indicated. The drawings are not necessarily drawn to scale, and the emphasis is placed on illustrating the principles of the embodiments.

[0043] Figure 1 A flowchart of a wheel hub defect detection method provided by an embodiment of the present application;

[0044] Figure 2 A structural diagram of a defect detection model in a wheel hub defect detection method provided by an embodiment of the present application;

[0045] Figure 3 A structural diagram of a Transformer Block in a defect detection model in a wheel hub defect detection method provided by an embodiment of the present application;

[0046] Figure 4 A structural diagram of a wheel hub defect detection device provided by an embodiment of the present application;

[0047] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the scope of protection of the present application.

[0049] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, reference numerals and / or letters can be repeated in different examples in the present application. Such repetition is for the purpose of simplification and clarity, and does not in itself indicate a relationship between the various embodiments and / or settings being discussed.

[0050] In order to solve the technical problem that the detection model is difficult to quickly adapt to the new production process after the production process is improved and updated in the prior art, the present application provides a wheel hub defect detection method, device, electronic equipment and storage medium, which can realize the adjustment of the defect detection module and ensure that the defect detection model can quickly adapt to the new production process.

[0051] Figure 1 A flowchart of a wheel hub defect detection method provided by an embodiment of the present application is shown in Figure 1 The wheel hub defect detection method provided by the embodiment of the present application specifically includes the following steps:

[0052] S1: obtaining a pre-trained defect detection model; wherein the defect detection model includes an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism includes initial weights for processing input vectors into feature representations;

[0053] Specifically, the defect detection model is used for defect detection on a to-be-detected wheel hub image. Before the technical solution provided by the embodiment of the present application is executed, the defect detection model has been pre-trained. For a to-be-detected wheel hub image input into the defect detection model, the defect detection model outputs a defect detection result of the to-be-detected wheel hub image.

[0054] In a feasible embodiment of the present application, the defect detection model can be specifically a deep learning model based on a SegFormer architecture.

[0055] Figure 2 A structural diagram of the defect detection model in the wheel hub defect detection method provided by the embodiment of the present application is shown in Figure 2 The structure of the defect detection model is described.

[0056] In the embodiment, the pre-trained defect detection model is a deep learning model based on a SegFormer architecture. The SegFormer model mainly consists of an encoder (Encoder) and a decoder (Decoder).

[0057] The encoder of the defect detection model is used to extract multi-level features of the input hub image to be detected. The encoder of the defect detection model is provided with a Transformer structure, which specifically represents four cascaded Transformer Blocks (e.g., Transformer Block1-4 in Figure 2 , i.e., the first to fourth Transformer blocks), and each Transformer Block processes a feature map of a different scale to capture context information of the image at different resolutions.

[0058] The input hub image to be detected is first subjected to an Over Patch Merging operation to divide the image into a plurality of patches and preliminarily process the patches into input feature vectors of the first Transformer Block. The Over Patch Merging operation reduces the spatial resolution by merging adjacent patches and expands the channel dimension, thereby constructing a hierarchical feature representation. The core of the operation is to concatenate local features and compress the channel number through linear transformation, achieving a spatial compression effect similar to the pooling operation in a convolutional neural network (CNN), but retaining more semantic information.

[0059] The output feature vectors of each Transformer Block (i.e., Stage1-4 features in Figure 2 , i.e., the first to fourth stage features) are subjected to an Over Patch Merging operation to downsample the width and height, and then input into the next Transformer Block. Meanwhile, the output feature vectors of the layer are also sent to the subsequent decoder part.

[0060] Figure 3 The structure of the Transformer Block in the defect detection model in the hub defect detection method provided by the embodiments of the present application is shown in Figure 3 . Each Transformer Block is provided with a multi-head attention mechanism (Multi-Head Attention) and a feed-forward neural network (FFN) inside.

[0061] The input features are first calculated in parallel by a multi-head self-attention mechanism to obtain multiple sets of attention weights, capturing long-range dependencies in the sequence. The multi-head self-attention mechanism is configured to process the input vectors into initial weights for feature representation, which are learned during pre-training of the defect detection model to capture rich low-level and high-level semantic features of images. The input vectors are processed by the initial weights, and the multi-head self-attention mechanism outputs corresponding feature representations. Subsequently, the output of the multi-head attention mechanism is fused with the original input through a residual connection (Add) operation and enters the FFN for nonlinear transformation. The result is again stacked with the previous stage features through a residual connection, forming a double residual connection structure, and finally completing the feature output of a Transformer Block. The Transformer Block outputs the feature representations.

[0062] The decoder is responsible for decoding the multi-scale feature maps extracted by the encoder into pixel-level defect segmentation masks. Referring to Figure 2 The decoder mainly consists of two MLP (Multi-Layer Perceptron) blocks. First, the feature maps output by the four different levels of Transformer Blocks from the encoder are sampled to a unified resolution (e.g., 1 / 4 of the original resolution) by the first MLP block (MLP1), and the channel numbers of each feature map are uniformly adjusted (e.g., to 256) using a 1x1 convolution operation. Then, these processed feature vectors are concatenated and input into the second MLP block (MLP2). The last layer of MLP2 generates a feature map with a channel number equal to the number of defect classes through a 1x1 convolution layer, and finally obtains the defect segmentation mask of the hub image to be detected.

[0063] S2: In response to the model adjustment instruction, a model adjustment plug-in is called to adjust the defect detection model to obtain an adjusted defect detection model. The model adjustment plug-in is used to adjust at least one initial weight in the multi-head attention mechanism according to the training of the small sample dataset corresponding to the new process.

[0064] Specifically, when receiving the model adjustment instruction, the model adjustment plug-in is called to adjust the defect detection model, and the adjusted defect detection model is obtained. In one specific example, the model adjustment instruction can be triggered manually by the user, for example, when the user learns that the current hub production process has changed, the user actively inputs the model adjustment instruction to instruct the defect detection model to adjust. In another specific example, the model adjustment instruction can also be automatically triggered by the production parameters related to the production process.

[0065] The model adjustment plug-in is configured to adjust at least one initial weight in the multi-head attention mechanism based on training of the small sample dataset. Specifically, when receiving a model adjustment instruction, the model adjustment plug-in is activated to act on the multi-head attention mechanism of the encoder in the defect detection model, and the initial weight in the multi-head attention mechanism is adjusted.

[0066] To ensure that the small sample dataset can clearly reflect the characteristics of the sample hub images corresponding to the new process, the small sample dataset is subjected to image enhancement processing. In an embodiment of the present application, before the model adjustment plug-in is called to adjust the defect detection model, the following steps are included:

[0067] The small sample dataset is obtained, wherein the small sample dataset includes a plurality of X-ray images of sample hub images corresponding to the new process, and each sample hub image is labeled with a corresponding defect category and a defect mask.

[0068] The small sample dataset is subjected to image enhancement processing, wherein the image enhancement processing includes Gaussian filtering and / or wavelet transform of the sample hub images.

[0069] Specifically, the small sample dataset is an image dataset corresponding to a new production process, which includes a plurality of X-ray images of sample hub images corresponding to the new process, and each sample hub image is labeled with a corresponding defect category and a defect mask.

[0070] As a specific example, in the production process of the hub, the hub is scanned, and the mechanical arm is precisely clamped and rotated at multiple angles to ensure that the surface and internal structure of the hub can be scanned in all directions, obtaining X-ray images of sample hub images of the hub. The number of sample hub images obtained at various angles for one hub is mainly determined according to the model and size of the hub.

[0071] Through the above embodiment, the noise of the sample hub image is suppressed by Gaussian filtering, and the details of the sample hub image are enhanced by wavelet transform, so that the sample hub images in the small sample dataset can clearly show the hub produced by the new production process.

[0072] In an embodiment of the present application, the initial weight includes a weight matrix for generating a query (Q) vector, a weight matrix for generating a key (K) vector, a weight matrix for generating a value (V) vector, and / or a weight matrix for performing output linear transformation after attention calculation in the multi-head attention mechanism.

[0073] Specifically, since the multi-head attention mechanism processes sequence vectors, the feature map of the image is first divided into multiple patches, and each patch represents a vector in the sequence. The multi-head attention mechanism performs three independent linear transformation matrices Q (query), K (key), V (value) vectors are generated, That is, the image bottom layer and high layer semantic features obtained during pre-training are included, which affect the defect detection result of the hub image to be detected by the defect detection model, and serve as initial weights.

[0074] In an embodiment of the present application, in response to a model adjustment instruction, a model adjustment plug-in is called to adjust the defect detection model to obtain an adjusted defect detection model, including:

[0075] The learnable parameters in the model adjustment plug-in are initialized;

[0076] The defect detection model calling the model adjustment plug-in is trained by a small sample data set to determine the learnable parameters;

[0077] Adjust the defect detection model based on the learnable parameters and the initial weights; wherein the processing result of the input vector by the multi-head attention mechanism after adjustment is defined by the following formula:

[0078] ;

[0079] h is the processing result of the input vector by the multi-head attention mechanism, is the initial weight, x is the input vector, is the adjustment result generated by the input vector jointly acted on by the model adjustment plug-in and the learnable parameters.

[0080] Specifically, when adjusting the defect detection model, only the initial weight in the multi-head attention mechanism is adjusted. The defect detection model calling the model adjustment plug-in is trained based on a small sample data set to determine the learnable parameters for adjusting the initial weight, so as to adjust the initial weight by the learnable parameters and change the processing result of the input vector by the multi-head attention mechanism.

[0081] It can be understood that the model adjustment plug-in determines the learnable parameters based on the training of the small sample data set, so that the defect detection model can pay attention to the relevant features of the sample hub image in the new production process, and the adjusted defect detection model is adapted to the new production process.

[0082] In an embodiment of the present application, the learnable parameters include a plurality of candidate ranks respectively corresponding to a plurality of pairs of low-rank matrices and a gating weight corresponding to each candidate rank, and the adjustment result is represented as:

[0083] ;

[0084] is the gating weight corresponding to the i-th candidate rank, a pair of low-rank matrices corresponding to the i-th candidate rank, and a is a preset scaling factor.

[0085] Specifically, a set of preset candidate ranks are maintained in the model adjustment plug-in. Each candidate rank in the set of candidate ranks corresponds to a pair of low-rank matrices and a gating weight.

[0086] By maintaining a set of candidate ranks and associating the candidate ranks with low-rank matrices and gating weights that can be dynamically adjusted, the model adjustment plug-in gives the defect detection model the ability to adjust the rank value of the low-rank matrix (e.g., the weight matrix corresponding to the initial weight) when the model adjustment plug-in acts on the defect detection model, so that the defect detection model can select the optimal rank according to the actual situation.

[0087] The scaling factor is used to balance the update amplitude of the learnable parameters on the initial weight. In an available embodiment of the present application, a = 1.

[0088] After the model adjustment plug-in is called, the learnable parameters in the model adjustment plug-in are first initialized. Specifically, a set of preset candidate ranks , the low-rank matrices corresponding to each candidate rank , and the gating weights corresponding to the candidate ranks are initialized, and these learnable parameters are first set to preset values.

[0089] In an available embodiment of the present application, initializing the learnable parameters in the model adjustment plug-in includes:

[0090] Initializing the low-rank matrix in the pair of low-rank matrices corresponding to each candidate rank to a zero matrix; and initializing the low-rank matrix in the pair of low-rank matrices corresponding to each candidate rank by a Gaussian distribution.

[0091] After the initialization of the learnable parameters is completed, the prepared small sample dataset is input into the defect detection model configured with the model adjustment plug-in to train the defect detection model. During the training process, the initial weight is kept frozen, and only the learnable parameters are updated. The update of the learnable parameters occurs in the backpropagation stage in the training process.

[0092] In an available embodiment of the present application, the defect detection model calling the model adjustment plug-in is trained by the small sample dataset to determine the learnable parameters, including:

[0093] During the training iteration process, the pairs of low-rank matrices are updated; and

[0094] The sensitivity of each candidate rank pair to the loss function is calculated respectively, and the gating weight is updated based on the sensitivity through a Softmax function; wherein the sensitivity is represented as:

[0095] ;

[0096] is the sensitivity corresponding to the i-th candidate rank, and L is the loss function.

[0097] Specifically, when training the defect detection model calling the model adjustment plug-in through the small sample data set, in the back propagation stage in the training iteration process, the gradient (i.e. partial derivative) of the loss function to the low-rank matrix and is calculated layer by layer through the chain rule, and the parameter value in the low-rank matrix at each training iteration is constantly solved based on the gradient using the gradient descent method, completing the update of multiple pairs of low-rank matrices.

[0098] It can be understood that in the process of updating the low-rank matrices and , most of the parameters (such as initial weights, biases, etc.) of the defect detection model calling the model adjustment plug-in remain frozen, and only the low-rank matrices and are adjusted to optimize the low-rank matrices and to achieve fine-tuning.

[0099] At the same time, for each candidate rank, the sensitivity of the candidate rank to the loss function is calculated, and the gating weight is updated based on the sensitivity, so that the candidate rank with high sensitivity can obtain a larger gating weight.

[0100] It can be understood that the sensitivity actually describes the attention of the current adjusted initial weight to the input features of a certain level, and the gating weight corresponding to the candidate rank is determined based on the sensitivity, so that the initial weight is adjusted, which can dynamically adjust the contribution of the low-rank matrix of different candidate ranks in the update of the defect detection model.

[0101] S3: obtaining the to-be-detected hub image, and processing the to-be-detected hub image through the adjusted defect detection model to obtain a defect detection result of the to-be-detected hub image;

[0102] Specifically, after the adjustment of the defect detection model is completed, the to-be-detected hub image is processed using the adjusted defect detection model, thereby obtaining a defect detection result of the to-be-detected hub image, and completing the defect detection of the to-be-detected hub image.

[0103] In an embodiment of the present application, the defect detection result of the hub image to be detected specifically includes a defect segmentation mask and a defect category labeled on the defect segmentation mask.

[0104] The method provided in the embodiment of the present application can call a model adjustment plug-in to adjust the defect detection model when the production process is adjusted. The adjustment is based on a small sample data set corresponding to the new process, and the adjustment is applied to at least one initial weight in the multi-head attention mechanism. After the adjustment of the defect detection model is completed, the defect detection model is processed by the adjusted defect detection model to complete the defect detection of the hub image to be detected.

[0105] The technical solution provided in the embodiment of the present application updates the defect detection model based on a small sample, so that the defect detection model can reduce the dependence on data when adapting to a new process. At the same time, only at least one initial weight in the multi-head attention mechanism is adjusted, and the defect detection model does not need to be adjusted to a large extent, which accelerates the iteration and upgrading speed of the defect detection model. In summary, the technical solution provided in the present application can quickly adapt the defect detection model to the new production process after the production process is improved and updated.

[0106] In some actual production scenarios, a new defect category appears on the production line of the hub production process. At this time, the defect detection model can be updated to enable the defect detection model to recognize the new defect category.

[0107] In this embodiment, in response to the model adjustment instruction, the encoder part of the defect detection model is frozen, the decoder part of the defect detection model is adjusted, and the output layer channel number of the decoder part is increased, so that the defect detection model can recognize the new defect category.

[0108] The technical solution provided in the present application is described based on the specific implementation steps of each embodiment. The technical solution can include the following steps when implemented:

[0109] S01, collect and prepare a small sample data set corresponding to a new production process:

[0110] For the hub produced by the new production process, a small amount (for example, 20 pictures for each defect) of image samples containing representative defects are collected. These images can be X-ray images or visible light images.

[0111] The collected new sample images are preprocessed and labeled. If the image is an X-ray image, it is first converted into a grayscale image, then Gaussian filtering can be used to remove noise, and then wavelet transform is used to enhance details, so as to facilitate subsequent labeling and model training.

[0112] Precise labeling of the defect area in the preprocessed image generates a mask for training.

[0113] S02, load a pre-trained defect detection model:

[0114] Load a defect detection model trained on a large number of old process hub data or other related X-ray defect detection tasks (such as X-ray defect images of automobile knuckles, automobile shells, and other metal parts) as a base model. The weights in the multi-head attention mechanism in this model are the initial weights .

[0115] S03, configure the model adjustment plug-in:

[0116] Initialize the model adjustment plug-in, including:

[0117] Select the multi-head attention mechanism (e.g., Q, K, V projection layer and output layer) in the defect detection model encoder to which the model adjustment plug-in is to be applied;

[0118] Set a set of candidate ranks ;

[0119] Initialize the low-rank matrix in each pair of low-rank matrices corresponding to each candidate rank to a zero matrix; and initialize the low-rank matrix in each pair of low-rank matrices corresponding to each candidate rank through a Gaussian distribution ; ;

[0120] Set the initial values of the scaling factor α and the learnable gating weight .

[0121] S04, execute the adjustment of the defect detection model:

[0122] Input the prepared small sample dataset into the defect detection model configured with the model adjustment plug-in for training;

[0123] During training, the initial weights of the defect detection model are mostly frozen;

[0124] When backpropagating, calculate the gradient of the loss function with respect to the low-rank matrix and layer by layer through the chain rule, and based on the gradient, constantly solve the parameter values in the low-rank matrix at each training iteration using the gradient descent method to update the multiple pairs of low-rank matrices;

[0125] Update the gating weight according to the sensitivity of the loss function to each candidate rank ;

[0126] After the updating is completed, the adjustment of the initial weight for processing the input vector h of the multi-head attention mechanism is adjusted from to .

[0127] S05, deploy the defect detection model after the adjustment is completed:

[0128] The hub image to be processed is processed by using the defect detection model after the adjustment is completed, and a defect detection result is obtained.

[0129] Figure 4 The structure diagram of the hub defect detection device provided by the embodiment of the application corresponds to the method embodiment, and the embodiment of the application further provides a hub defect detection device. Referring to Figure 4 , the device comprises:

[0130] The first acquisition module 401 is configured to acquire a pre-trained defect detection model. The defect detection model comprises an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism comprises initial weights for processing an input vector into a feature representation.

[0131] The calling module 402 is configured to, in response to a model adjustment instruction, call a model adjustment plug-in to adjust the defect detection model to obtain an adjusted defect detection model. The model adjustment plug-in is configured to adjust at least one initial weight in the multi-head attention mechanism according to training of a small sample data set corresponding to a new process.

[0132] The first processing module 403 is configured to acquire a hub image to be detected, and process the hub image to be detected by using the defect detection model after the adjustment to obtain a defect detection result of the hub image to be detected.

[0133] In a feasible embodiment of the application, the initial weights comprise a weight matrix for generating a query (Q) vector, a weight matrix for generating a key (K) vector, a weight matrix for generating a value (V) vector, and / or a weight matrix for performing output linear transformation after attention calculation in the multi-head attention mechanism.

[0134] In a feasible embodiment of the application, the calling module 402 comprises:

[0135] The initialization unit is configured to initialize learnable parameters in the model adjustment plug-in.

[0136] The training unit is configured to train the defect detection model calling the model adjustment plug-in by using the small sample data set to determine the learnable parameters.

[0137] an adjusting unit configured to adjust the defect detection model based on the learnable parameters and the initial weights, wherein a processing result of the multi-head attention mechanism on the input vector is defined by the following formula:

[0138] ;

[0139] h is the processing result of the multi-head attention mechanism on the input vector, is the initial weight, x is the input vector, is an adjustment result generated by the model adjustment plug-in and the learnable parameters acting on the input vector.

[0140] In an embodiment of the present application, the learnable parameters include a plurality of pairs of low-rank matrices corresponding to a plurality of preset candidate ranks respectively, and a gating weight corresponding to each candidate rank, and the adjustment result is represented by the following formula:

[0141] ;

[0142] is the gating weight corresponding to the i-th candidate rank, is a pair of low-rank matrices corresponding to the i-th candidate rank, and a is a preset scaling factor.

[0143] In an embodiment of the present application, the training unit includes:

[0144] an updating subunit configured to update the plurality of pairs of low-rank matrices during a training iteration process; and

[0145] a calculating subunit configured to calculate a sensitivity of each candidate rank to the loss function respectively, and update the gating weight through a Softmax function based on the sensitivity, wherein the sensitivity is represented by the following formula:

[0146] ;

[0147] is the sensitivity corresponding to the i-th candidate rank, and L is the loss function.

[0148] In an embodiment of the present application, the initialization unit includes:

[0149] a first initialization subunit configured to initialize a low-rank matrix in a pair of low-rank matrices corresponding to each candidate rank to a zero matrix; and a second initialization subunit configured to initialize a low-rank matrix in a pair of low-rank matrices corresponding to each candidate rank to a matrix initialized by a Gaussian distribution.

[0150] a second initialization subunit configured to initialize a low-rank matrix in a pair of low-rank matrices corresponding to each candidate rank to a matrix initialized by a Gaussian distribution.

[0151] ​In an embodiment of the present application, the device further comprises:

[0152] The second acquisition module is configured to acquire a small sample dataset, wherein the small sample dataset comprises X-ray images of a plurality of sample hub images corresponding to a new process, and each sample hub image is labeled with a corresponding defect category and a defect mask.

[0153] The second processing module is configured to perform image enhancement processing on the small sample dataset, wherein the image enhancement processing comprises Gaussian filtering and / or wavelet transform on the sample hub images.

[0154] As shown in Figure 5 The present application provides an electronic device, which comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504,

[0155] The memory 503 is configured to store a computer program.

[0156] In an embodiment of the present application, the processor 501 is configured to execute the program stored in the memory 503 to implement the steps of the wheel hub defect detection method provided in any one of the preceding method embodiments, for example comprising:

[0157] acquiring a pre-trained defect detection model, wherein the defect detection model comprises an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism comprises initial weights for processing input vectors into feature representations;

[0158] In response to a model adjustment instruction, a model adjustment plug-in is called to adjust the defect detection model to obtain an adjusted defect detection model, wherein the model adjustment plug-in is configured to adjust at least one initial weight in the multi-head attention mechanism according to training of the small sample dataset corresponding to the new process;

[0159] acquiring a to-be-detected hub image, and processing the to-be-detected hub image through the adjusted defect detection model to obtain a defect detection result of the to-be-detected hub image.

[0160] The present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the wheel hub defect detection method provided in any one of the preceding method embodiments.

[0161] The apparatus embodiments described above are only illustrative, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the embodiments or some parts of the embodiments.

[0163] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order in which they are described, unless specifically indicated as such. It is also to be understood that additional or alternative steps can be employed.

[0164] The above description is merely illustrative of the application and should not be taken as limiting. Numerous modifications and variations underlying the general principles of the applications can be made by those of ordinary skill in the art without departing from the spirit or scope of the application. Therefore, the application is not to be limited to the embodiments described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of detecting defects in a wheel hub, characterized by, The method comprises: obtaining a pre-trained defect detection model; wherein the defect detection model comprises an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism comprises initial weights for processing input vectors into feature representations; in response to a model adjustment instruction, calling a model adjustment plug-in to adjust the defect detection model to obtain an adjusted defect detection model; wherein the model adjustment plug-in is used to adjust at least one of the initial weights in the multi-head attention mechanism according to the training of a small sample data set corresponding to a new process; obtaining a to-be-detected hub image, and processing the to-be-detected hub image through the adjusted defect detection model to obtain a defect detection result of the to-be-detected hub image; wherein, in response to a model adjustment instruction, calling a model adjustment plug-in to adjust the defect detection model to obtain an adjusted defect detection model, comprising: initializing the learnable parameters in the model adjustment plug-in; training the defect detection model calling the model adjustment plug-in through the small sample data set to determine the learnable parameters; adjusting the defect detection model based on the learnable parameters and the initial weights; wherein the processing result of the multi-head attention mechanism on the input vector after adjustment is defined by the following formula: h = W0x + K adj (x); h is a processing result of the multi-head attention mechanism on the input vector, W0is the initial weight, x is the input vector, K adj (x) is an adjustment result generated by the model adjustment plug-in and the learnable parameters acting on the input vector; The learnable parameters include a plurality of pairs of low-rank matrices corresponding to a set of preset candidate ranks {r1, r2,..., r n} respectively and a gating weight corresponding to each of the candidate ranks, and the adjustment result is represented as: P ri The gating weight corresponding to the i-th candidate rank is denoted as (B ri A ri The pair of low-rank matrices corresponding to the i-th candidate rank is denoted as (A 2. The method of claim 1, wherein, The initial weights include a weight matrix for generating a query Q vector, a weight matrix for generating a key K vector, a weight matrix for generating a value V vector, and / or a weight matrix for performing output linear transformation after attention calculation in the multi-head attention mechanism.

3. The method of claim 1, wherein, Training the defect detection model calling the model adjustment plug-in through the small sample data set to determine the learnable parameters, comprising: updating a plurality of pairs of low-rank matrices during the training iteration process; and, respectively calculating the sensitivity of each candidate rank to the loss function, and updating the gating weight through the Softmax function based on the sensitivity; wherein the sensitivity is represented as: s ri the sensitivity corresponding to the i-th candidate rank, L is the loss function.

4. The method of claim 1, wherein, initializing the learnable parameters in the model adjustment plug-in, comprising: a low-rank matrix B in a pair of low-rank matrices corresponding to each of the candidate ranks ri initialized to a zero matrix; and, A low-rank matrix A in a pair of low-rank matrices corresponding to each of the candidate ranks ri Initialization is performed by a Gaussian distribution.

5. The method of claim 1, wherein, Before the method further comprises: obtaining the small sample data set; wherein the small sample data set comprises X-ray images of a plurality of sample hub images corresponding to a new process, and each of the sample hub images is labeled with a corresponding defect category and a defect mask; performing image enhancement processing on the small sample data set; wherein the image enhancement processing comprises Gaussian filtering and / or wavelet transform on the sample hub image.

6. A wheel defect detection apparatus characterized by comprising: The device comprises: The first acquisition module is configured to acquire a pre-trained defect detection model; wherein the defect detection model comprises an encoder provided with a Transformer structure, the Transformer structure is provided with a multi-head attention mechanism, and the multi-head attention mechanism comprises initial weights for processing an input vector into a feature representation; The calling module is configured to, in response to a model adjustment instruction, call a model adjustment plug-in to adjust the defect detection model to obtain an adjusted defect detection model; wherein the model adjustment plug-in is configured to adjust at least one initial weight in the multi-head attention mechanism according to training of a small sample data set corresponding to a new process; The first processing module is configured to acquire a to-be-detected hub image, and process the to-be-detected hub image by using the adjusted defect detection model to obtain a defect detection result of the to-be-detected hub image; The calling module comprises: An initialization unit configured to initialize learnable parameters in the model adjustment plug-in; A training unit configured to train the defect detection model calling the model adjustment plug-in by using a small sample data set to determine the learnable parameters; An adjustment unit configured to adjust the defect detection model based on the learnable parameters and the initial weights; wherein a processing result of the multi-head attention mechanism on the input vector is defined by the following formula: h = W0x + K adj (x); h is the processing result of the multi-head attention mechanism on the input vector, W0is the initial weight, x is the input vector, K adj (x) is the adjustment result generated by the joint action of the model adjustment plug-in and the learnable parameters on the input vector; The learnable parameters include a plurality of pairs of low-rank matrices corresponding to a set of preset candidate ranks {r1, r2,..., r n} respectively, and a gating weight corresponding to each candidate rank, and the adjustment result is represented as: P ri is the gating weight corresponding to the i-th candidate rank, (B ri A ri is a pair of low-rank matrices corresponding to the i-th candidate rank, and a is a preset scaling factor.

7. An electronic device, comprising: comprise: at least one communication interface; at least one bus connected with the at least one communication interface; at least one processor connected with the at least one bus; at least one memory connected with the at least one bus, wherein the processor is configured to implement the method of any one of claims 1-5.

8. A computer storage medium, characterized in that The computer executable instructions are stored in the memory, and the computer executable instructions are used to execute the method of any one of claims 1-5.

Images