Network training method and device, hub detection method and device, equipment, medium and product

By extracting the three-layer features of the hub image and generating differential features, optimizing the abnormal detection neural network parameters, the problem of poor robustness for different models in hub detection is solved, and the detection effect of high accuracy and low leakage detection rate is achieved.

CN119399166APending Publication Date: 2025-02-07CHINA MOBILE COMM LTD RES INST +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411515327.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-29
Publication Date
2025-02-07

AI Technical Summary

Technical Problem

The prior art requires a large number of abnormal sample training in hub detection, and is limited by image acquisition equipment and algorithms, resulting in poor robustness to different hub models.

Method used

By obtaining the training set of sample hub images, the three intermediate layers of features are extracted to generate a normal feature sample library, the three intermediate layers of features are compared with the normal features for differential feature generation, and the abnormal detection neural network parameters are optimized to improve detection accuracy.

Benefits of technology

It achieves high accuracy and extremely low leakage detection rate when only a small number of qualified samples are used, saving data and time costs for algorithm research and development and iterative upgrades, and improving the robustness of different wheel hub models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119399166A_ABST
    Figure CN119399166A_ABST
Patent Text Reader

Abstract

The invention provides a network training method and device, a hub detection method and device, equipment, a medium and a product. The method comprises the following steps: acquiring a training set of sample hub images; the middle three-layer features of each hub image in the training set are extracted, a normal feature sample library is generated, and the middle three-layer features are three convolutional layers after a full link layer and an activation layer are removed; comparing the middle three-layer features with normal features in a normal feature sample library to generate differential features; obtaining a prediction result according to the difference features and the middle three-layer features, wherein the prediction result comprises whether the wheel hub is a suspected abnormal wheel hub or not and an abnormal area of the suspected abnormal wheel hub; and optimizing parameters of the anomaly detection neural network according to the prediction result to obtain a trained anomaly detection neural network. According to the method, only a small number of qualified samples are utilized, more effective feature representation is obtained, high accuracy and extremely low omission ratio are achieved, and the data and time cost of algorithm research and development and iterative upgrading is saved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data computing technology, and in particular, to a method, device, equipment, medium and product for network training and wheel hub detection. Background Art

[0002] Industrial vision inspection is a key link in intelligent manufacturing. At present, there are problems in industrial vision inspection, such as fragmented application scenarios, differentiated customer needs, complex imaging environment, scarce data samples, diversified defect patterns, and discrete data distribution. These problems lead to challenges such as difficulty in large-scale promotion, difficulty in high-definition imaging, difficulty in learning models, and difficulty in centralized data management.

[0003] In the auto parts industry, taking the wheel hub production process as an example, manual quality inspection is currently commonly used to detect surface defects on the wheel hub after painting, and there is generally a certain rate of missed detection after product delivery. The academic and industrial circles can generally only detect fixed types of defects on the front of the wheel hub, and still lack a full inspection solution for all surfaces of the wheel hub and actual production line products. In the algorithm dimension, the defects vary greatly and samples are scarce, resulting in limited model training data and high fitting difficulty. In the product dimension, there are many workpiece models and new models are launched quickly, which poses challenges to the speed and cost of iterative upgrades after the business goes online. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, equipment, medium and product for network training and wheel hub detection to solve the problem that the existing training requires a large number of abnormal samples and is limited by the image acquisition equipment and the algorithm itself, resulting in poor robustness to different wheel hub models.

[0005] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for training an anomaly detection neural network, comprising:

[0007] Obtain a training set of sample wheel hub images;

[0008] Extracting the middle three-layer features of each wheel hub image in the training set to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer;

[0009] Compare the intermediate three-layer features with normal features in a normal feature sample library to generate differential features; obtain prediction results based on the differential features and the intermediate three-layer features, the prediction results including whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub;

[0010] Optimize the parameters of the anomaly detection neural network according to the prediction results to obtain a trained anomaly detection neural network.

[0011] Optionally, the training set includes: a normal wheel hub image sample set, and an abnormal wheel hub image sample set generated by a pseudo-abnormal generation method based on the normal wheel hub image sample set.

[0012] Optionally, the obtaining of a training set of sample wheel hub images includes:

[0013] Taking sample panoramic images of a plurality of sample hubs, and extracting sample feature information of the sample panoramic images;

[0014] The sample feature information is matched with the preset feature point of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the angle between the sample wheel hub and the preset feature point of the initial position is calculated, and the sample wheel hub is rotated to reach the preset shooting point of the sample wheel hub;

[0015] Sample images of each surface of the sample hub at preset shooting points of the sample hub are photographed to obtain a training set of sample hub images.

[0016] Optionally, before the sample feature information is matched with the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the method further includes:

[0017] Panoramic pictures of multiple models of wheel hubs are obtained, and feature information of the panoramic pictures of the multiple models of wheel hubs is extracted, the feature information including feature points and feature descriptors, a feature point library and a feature descriptor library are obtained, and shooting points are preset according to the structural characteristics of each wheel hub model and the feature information to obtain the wheel hub model library.

[0018] Optionally, the step of comparing the intermediate three-layer features with normal features in a normal feature sample library to generate differential features may also include:

[0019] Extracting the middle three-layer features of each wheel hub image in the training set to generate an abnormal feature sample library;

[0020] A data triplet including the features in the intermediate three-layer features, the normal feature sample library and the abnormal feature sample library is constructed, and the intermediate three-layer features are optimized so that the distance between the intermediate three-layer features and the normal feature sample library is smaller than the distance between the intermediate three-layer features and the abnormal feature sample library.

[0021] Optionally, obtaining a prediction result according to the differential feature and the intermediate three-layer features includes:

[0022] The intermediate three-layer features are re-encoded according to the differential features to obtain re-encoded features, and the re-encoded features are input into a decoder to obtain a prediction result.

[0023] Optionally, optimizing the parameters of the abnormality detection neural network according to the predicted abnormal wheel hub and abnormal area to obtain a trained abnormality detection neural network includes:

[0024] The parameters of the anomaly detection neural network are optimized using the absolute error loss function, the focal loss function and the prediction result to obtain a trained anomaly detection neural network.

[0025] In a second aspect, an embodiment of the present invention provides a method for detecting wheel hub defects, comprising:

[0026] Acquire target images of multiple surfaces of the wheel hub to be inspected;

[0027] Input the target image into the abnormality detection neural network as described in any one of the first aspects, the abnormality detection neural network extracts the middle three-layer features of the target image of multiple surfaces of the wheel hub to be detected, and generates a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer; compare the middle three-layer features with the normal features in the normal feature sample library to generate differential features; obtain a detection result based on the differential features and the middle three-layer features, the detection result is whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub, and output the abnormal area of ​​the suspected abnormal wheel hub as a suspected defect area;

[0028] The suspected defect area is input into a classification neural network, and the classification neural network outputs the probability that the suspected defect area is a normal area. When the probability is less than a preset probability threshold, it is an abnormal image.

[0029] Optionally, obtaining target images of various surfaces of the wheel hub to be inspected includes:

[0030] Taking a panoramic image of the wheel hub to be inspected, and extracting target feature information of the panoramic image;

[0031] The target feature information is matched with the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the angle between the wheel hub to be detected and the preset feature points of the initial position is calculated, and the wheel hub to be detected is rotated to reach the preset shooting point position of the wheel hub to be detected;

[0032] The target images of each surface of the wheel hub to be inspected at the preset shooting points of the wheel hub to be inspected are captured.

[0033] In a third aspect, an embodiment of the present invention provides a training device for an anomaly detection neural network, comprising:

[0034] A first acquisition module is used to acquire a training set of sample wheel hub images;

[0035] The first processing module is used to extract the middle three-layer features of each wheel hub image in the training set to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer;

[0036] The second processing module is used to compare the intermediate three-layer features with the normal features in the normal feature sample library to generate differential features; obtain a prediction result according to the differential features and the intermediate three-layer features, and the prediction result includes whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub;

[0037] The first optimization module is used to optimize the parameters of the anomaly detection neural network according to the prediction result to obtain a trained anomaly detection neural network.

[0038] Optionally, the training set includes: a normal wheel hub image sample set, and an abnormal wheel hub image sample set generated by a pseudo-abnormal generation method based on the normal wheel hub image sample set.

[0039] Optionally, the first acquisition module includes:

[0040] A first processing submodule is used to capture sample panoramic images of a plurality of sample hubs and extract sample feature information of the sample panoramic images;

[0041] The second processing submodule is used to correspond the sample feature information to the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, calculate the angle between the sample wheel hub and the preset feature points of the initial position, and rotate the sample wheel hub to reach the preset shooting point position of the sample wheel hub;

[0042] The third processing submodule is used to take sample images of each surface of the sample hub at preset shooting points of the sample hub to obtain a training set of sample hub images.

[0043] Optionally, also include:

[0044] The third processing module is used to obtain panoramic pictures of multiple models of wheel hubs and extract feature information of the panoramic pictures of the multiple models of wheel hubs, wherein the feature information includes feature points and feature descriptors, thereby obtaining a feature point library and a feature descriptor library, and presetting shooting points according to the structural characteristics of each wheel hub model and the feature information to obtain the wheel hub model library.

[0045] Optionally, also include:

[0046] An extraction module, used to extract the middle three-layer features of each wheel hub image in the training set and generate an abnormal feature sample library;

[0047] The second optimization module is used to construct a data triplet including the features in the intermediate three-layer features, the normal feature sample library and the abnormal feature sample library, and optimize the intermediate three-layer features so that the distance between the intermediate three-layer features and the normal feature sample library is smaller than the distance between the intermediate three-layer features and the abnormal feature sample library.

[0048] Optionally, the second processing module includes:

[0049] The re-encoding submodule is used to re-encode the intermediate three-layer features according to the differential features to obtain re-encoded features, and input the re-encoded features into the decoder to obtain prediction results.

[0050] Optionally, the first optimization module includes:

[0051] The optimization submodule is used to optimize the parameters of the anomaly detection neural network using the absolute error loss function and the focal loss function and the prediction result to obtain a trained anomaly detection neural network.

[0052] In a fourth aspect, an embodiment of the present invention provides a device for detecting wheel hub defects, comprising:

[0053] A third acquisition module is used to acquire target images of multiple surfaces of the wheel hub to be inspected;

[0054] A fourth processing module, for inputting the target image into an abnormality detection neural network as described in any one of the first aspects, wherein the abnormality detection neural network extracts the intermediate three-layer features of the target image of the multiple surfaces of the wheel hub to be detected, and generates a normal feature sample library, wherein the intermediate three-layer features are three convolutional layers after removing the full connection layer and the activation layer; comparing the intermediate three-layer features with the normal features in the normal feature sample library to generate differential features; obtaining a detection result based on the differential features and the intermediate three-layer features, wherein the detection result is whether it is a suspected abnormal wheel hub and an abnormal area of ​​the suspected abnormal wheel hub, and outputting the abnormal area of ​​the suspected abnormal wheel hub as a suspected defect area if the detection result is a suspected abnormal wheel hub;

[0055] The fifth processing module is used to input the suspected defect area into a classification neural network, and the classification neural network outputs the probability that the suspected defect area is a normal area. When the probability is less than a preset probability threshold, it is an abnormal image.

[0056] Optionally, the third acquisition module includes:

[0057] A first shooting module is used to shoot a panoramic image of the wheel hub to be inspected and extract target feature information of the panoramic image;

[0058] The sixth processing module is used to correspond the target feature information to the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, calculate the angle between the wheel hub to be detected and the preset feature points of the initial position, and rotate the wheel hub to be detected to reach the preset shooting point position of the wheel hub to be detected;

[0059] The second shooting module is used to shoot target images of various surfaces of the wheel hub to be tested at preset shooting points of the wheel hub to be tested.

[0060] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the training method for anomaly detection neural network as described in any one of the first aspect, or implements the steps in the training method for anomaly detection neural network as described in any one of the second aspect.

[0061] In a sixth aspect, an embodiment of the present invention provides a readable storage medium storing a program or instruction on the readable storage medium, which, when executed by a processor, implements the steps in the training method for anomaly detection neural network as described in any one of the first aspect, or implements the steps in the training method for anomaly detection neural network as described in any one of the second aspect.

[0062] In a seventh aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the training method for anomaly detection neural network as described in any one of the first aspect, or implement the steps in the training method for anomaly detection neural network as described in any one of the second aspect.

[0063] In the present invention, a training set of sample wheel hub images is obtained; the middle three-layer features of each wheel hub image in the training set are extracted to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full link layer and the activation layer; the middle three-layer features are compared with the normal features in the normal feature sample library to generate differential features; a prediction result is obtained based on the differential features and the middle three-layer features, and the prediction result includes whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub; by comparing the middle three-layer features with the normal features in the normal feature sample library, differential features are generated, thereby obtaining a more effective feature representation, achieving high accuracy and extremely low missed detection rate; and the parameters of the abnormal detection neural network are optimized according to the prediction results to obtain a trained abnormal detection neural network. A small number of qualified samples are used to obtain a more effective feature representation, achieve high accuracy and extremely low missed detection rate, and save the data and time costs of algorithm development and iterative upgrades, solving the existing problem of requiring a large number of abnormal sample training and poor robustness to different wheel hub models due to limitations of image acquisition equipment and the algorithm itself. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0065] Figure 1 is a flow chart of a training method for anomaly detection neural network provided by an embodiment of the present invention;

[0066] Figure 2 is a schematic diagram of the structure of an anomaly detection neural network provided by an embodiment of the present invention;

[0067] Figure 3 It is a structural schematic diagram of a differential attention module of an anomaly detection neural network provided by an embodiment of the present invention;

[0068] Figure 4 is a flow chart of a method for detecting wheel hub defects provided by an embodiment of the present invention;

[0069] Figure 5 is a flow chart of another method for detecting wheel hub defects provided by an embodiment of the present invention;

[0070] Figure 6 It is a structural schematic diagram of a training device for anomaly detection neural network provided by an embodiment of the present invention;

[0071] Figure 7It is a structural schematic diagram of a wheel hub defect detection device provided by an embodiment of the present invention;

[0072] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Please refer to Figure 1 , an embodiment of the present invention provides a training method for an anomaly detection neural network, comprising:

[0075] Step 11: Obtain a training set of sample wheel images;

[0076] In the embodiment of the present invention, optionally, the training set includes: a normal wheel hub image sample set, and an abnormal wheel hub image sample set generated by a pseudo-abnormal generation method based on the normal wheel hub image sample set.

[0077] In an embodiment of the present invention, a training set of sample wheel hub images is obtained, and an abnormal wheel hub image sample set is generated according to the normal wheel hub image sample set by using an existing pseudo-anomaly generation method. Only normal wheel hub image samples are used to train the anomaly detection neural network, thereby achieving high accuracy and extremely low missed detection rate while using only a small number of qualified samples, greatly saving data and time costs for algorithm development and iterative upgrades.

[0078] Step 12: extracting the middle three-layer features of each wheel image in the training set to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer;

[0079] In the embodiment of the present invention, ResNet18 is used as the encoder for extracting the middle three layers of features, and end-to-end semantic segmentation is performed to detect and locate abnormal areas; for details, please refer to Figure 2Resnet18 extracts features from the three middle layers after removing the full-link layer and the activation layer, and constructs a normal feature sample library (Normal Bank) and an abnormal feature sample library (DefectBank) respectively; the differential attention module (Differential Attention) integrated after Resnet18 makes full use of the difference information of the features in the input normal feature sample library, and re-encodes the extracted features of the three middle layers, so as to obtain a more effective feature representation, achieving high accuracy and extremely low missed detection rate.

[0080] Step 13: Compare the intermediate three-layer features with the normal features in the normal feature sample library to generate differential features; obtain prediction results based on the differential features and the intermediate three-layer features, and the prediction results include whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub;

[0081] In the embodiment of the present invention, optionally, obtaining a prediction result according to the differential feature and the intermediate three-layer feature includes:

[0082] The intermediate three-layer features are re-encoded according to the differential features to obtain re-encoded features, and the re-encoded features are input into a decoder to obtain a prediction result.

[0083] In the embodiment of the present invention, a differential attention module (DifferentialAttention) is integrated after Resnet18, and the structure of the differential attention module is as follows: Figure 3 As shown, the middle three-layer feature IF is used to calculate the mean square error between the middle three-layer feature IF and the normal features in the normal feature sample library NormalBank, and the difference information of the features in the input normal feature sample library is fully utilized to generate differential features (Differential Feature), and the differential features are combined with the original features through the differential attention module, the extracted middle three-layer features are re-encoded, and the re-encoded features are input into the decoder to generate a prediction result mask to obtain the prediction result to identify anomalies. The re-encoded features that integrate the differential information are more discriminative and accurate, thereby effectively improving the accuracy of the anomaly detection network.

[0084] Step 14: Optimize the parameters of the anomaly detection neural network according to the prediction results to obtain a trained anomaly detection neural network.

[0085] In the embodiment of the present invention, optionally, optimizing the parameters of the abnormality detection neural network according to the predicted abnormal wheel hub and abnormal area to obtain the trained abnormality detection neural network includes:

[0086] The parameters of the anomaly detection neural network are optimized using the absolute error loss function, the focal loss function and the prediction result to obtain a trained anomaly detection neural network.

[0087] In the embodiment of the present invention, in order to enhance the closeness between the prediction result and the true value, the absolute error (L1) loss function and the focus (Lf) loss function are used to optimize the parameters of the anomaly detection neural network to obtain the trained anomaly detection neural network. The total loss function is l all =λ l1 *L l1 +λ f *L f +λ t *L t Among them, L t To optimize the data triplet of the intermediate three-layer features, λ l1 , f and λ t are hyperparameters, which can be set to 0.5, 0.4, and 0.1 respectively according to actual conditions. Among them, the absolute error (L1) loss function effectively retains the edge information in the segmented image, while the focus (Lf) loss function alleviates the problem of unbalanced distribution of training data, enabling the model to focus on challenging samples and improve the accuracy of abnormal segmentation.

[0088] In an embodiment of the present invention, a training set of sample wheel hub images is obtained, and the training set includes: a normal wheel hub image sample set, and an abnormal wheel hub image sample set generated by a pseudo-anomaly generation method based on the normal wheel hub image sample set; only a small number of qualified samples are used to save data and time costs for algorithm development and iterative upgrades; the middle three-layer features of each wheel hub image in the training set are extracted to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full link layer and the activation layer; the middle three-layer features are compared with normal features in the normal feature sample library to generate differential features; according to the differential features, the middle three-layer features are feature re-encoded to obtain re-encoded features, and the re-encoded features are input into a decoder to obtain a prediction result, and the prediction result includes whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub; by comparing the middle three-layer features with the normal features in the normal feature sample library, differential features are generated, thereby obtaining a more effective feature representation, achieving high accuracy and extremely low missed detection rate; and according to the prediction results, the parameters of the abnormality detection neural network are optimized to obtain a trained abnormality detection neural network. By using a small number of qualified samples, a more effective feature representation can be obtained, achieving high accuracy and extremely low missed detection rate, and saving data and time costs for algorithm development and iterative upgrades. This solves the existing problem of requiring a large number of abnormal samples for training and having poor robustness to different wheel models due to limitations of image acquisition equipment and the algorithm itself.

[0089] In the embodiment of the present invention, optionally, the step of obtaining a training set of sample wheel hub images includes:

[0090] Taking sample panoramic images of a plurality of sample hubs, and extracting sample feature information of the sample panoramic images;

[0091] The sample feature information is matched with the preset feature point of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the angle between the sample wheel hub and the preset feature point of the initial position is calculated, and the sample wheel hub is rotated to reach the preset shooting point of the sample wheel hub;

[0092] Sample images of each surface of the sample hub at preset shooting points of the sample hub are photographed to obtain a training set of sample hub images.

[0093] In the embodiment of the present invention, to obtain a sample panoramic image of a sample wheel hub, firstly, a panoramic photo of the wheel hub is taken based on the initial position of the robot arm, and the feature information of the panoramic image is extracted using the ORB algorithm, including: feature points (kp i ) and feature descriptors (des i ); then, through the template matching algorithm, the most similar panoramic photo of the wheel hub in the wheel hub model library is used as the model of the wheel hub, (wherein, the acquisition of the wheel hub model library includes: acquiring panoramic pictures of multiple models of wheel hubs, and extracting feature information of the panoramic pictures of the multiple models of wheel hubs, the feature information includes: feature points and feature descriptors, and obtaining a feature point library and a feature descriptor library, which are KP = {kp1, kp2, kp3, ...}, DES = {des1, des2, des3, ...}, respectively, and according to the structural characteristics of each wheel hub model and the feature information, the shooting is preset point, and obtain the wheel hub model library); then find out the kp1 and des1 corresponding to the initial point, determine the matching feature points through kp1 and des1, and calculate the angle between the sample hub and the initial position through projection transformation according to the matching feature points, use a turntable to rotate the sample hub to reach the initial shooting point, complete the shooting of sample images of each surface of the sample hub at the preset shooting point of the sample hub, and obtain a training set of sample hub images, so that it can be compatible with all the wheel hub models that have been stored in the library, dynamically and flexibly support many models of workpieces, support covering complex workpiece structures, and meet high detection beats.

[0094] In the embodiment of the present invention, optionally, the step of comparing the intermediate three-layer features with normal features in a normal feature sample library to generate differential features may also include:

[0095] Extracting the middle three-layer features of each wheel hub image in the training set to generate an abnormal feature sample library;

[0096] A data triplet including the features in the intermediate three-layer features, the normal feature sample library and the abnormal feature sample library is constructed, and the intermediate three-layer features are optimized so that the distance between the intermediate three-layer features and the normal feature sample library is smaller than the distance between the intermediate three-layer features and the abnormal feature sample library.

[0097] In the embodiment of the present invention, in order to enhance the ability of the anomaly detection network to distinguish between normal samples and abnormal samples and establish a more accurate decision boundary for such samples, a contrastive learning method is used to optimize the distance between the intermediate three-layer feature (IF) and the normal feature sample (N) library and the abnormal feature sample (A) library, which is conducive to closer proximity between similar features and ensure greater separation between different categories of features, thereby improving the distinction between normal samples and abnormal samples and improving the accuracy of anomaly detection. The formula of the data triple is: L t =max(d(IF,N)-d(IF,A)+margin,0); where d is the cosine distance; margin is a hyperparameter generally set to 0.5.

[0098] The anomaly detection network is optimized by using a contrastive learning method to calculate the distance between the prediction result and the normal feature sample library and the abnormal feature sample library, so that the anomaly detection network can better distinguish normal and abnormal samples and estimate the decision boundary more accurately.

[0099] Please refer to Figure 4 , an embodiment of the present invention provides a method for detecting wheel hub defects, comprising:

[0100] Step 41: Acquire target images of multiple surfaces of the wheel hub to be inspected;

[0101] In the embodiment of the present invention, optionally, the acquiring target images of various surfaces of the wheel hub to be inspected includes:

[0102] Taking a panoramic image of the wheel hub to be inspected, and extracting target feature information of the panoramic image;

[0103] The target feature information is matched with the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the angle between the wheel hub to be detected and the preset feature points of the initial position is calculated, and the wheel hub to be detected is rotated to reach the preset shooting point position of the wheel hub to be detected;

[0104] The target images of each surface of the wheel hub to be inspected at the preset shooting points of the wheel hub to be inspected are captured.

[0105] In the embodiment of the present invention, please refer to Figure 5To obtain the sample panoramic image of the wheel hub to be inspected, firstly, a panoramic photo of the wheel hub is taken based on the initial position of the robot arm, and the feature information of the panoramic image is extracted using the ORB algorithm, including: feature points (kp i ) and feature descriptors (des i );

[0106] Then, the most similar panoramic photo of the wheel hub in the wheel hub model library is used as the model of the wheel hub through the template matching algorithm (wherein, the acquisition of the wheel hub model library includes: acquiring panoramic pictures of multiple models of wheel hubs, and extracting feature information of the panoramic photos of the multiple models of wheel hubs, the feature information includes: feature points and feature descriptors, and obtaining a feature point library and a feature descriptor library, which are KP={kp1, kp2, kp3,...}, DES={des1, des2, des3,...} respectively, and the shooting points are preset according to the structural characteristics of each wheel hub model and the feature information to obtain the wheel hub model library);

[0107] Then find out kp1 and des1 corresponding to the initial point, determine the matching feature points through kp1 and des1, and calculate the angle between the wheel hub to be detected and the initial position through projection transformation according to the matching feature points, and use the turntable to rotate the wheel hub to reach the initial shooting point;

[0108] Finally, images of the various surfaces of the wheel hub to be inspected at the preset shooting points of the wheel hub to be inspected are taken to obtain a training set of images of the wheel hub to be inspected, so that it can be compatible with all wheel hub models that have been put into the warehouse, dynamically and flexibly support many models of workpieces, support covering complex workpiece structures, and meet high inspection rhythms.

[0109] Step 42: input the target image into the abnormality detection neural network as described in any one of the first aspects, the abnormality detection neural network extracts the middle three-layer features of the target image of multiple surfaces of the wheel hub to be detected, and generates a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer; compare the middle three-layer features with the normal features in the normal feature sample library to generate differential features; obtain a detection result based on the differential features and the middle three-layer features, the detection result is whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub, and the abnormal area of ​​the suspected abnormal wheel hub with the detection result as the suspected abnormal wheel hub is output as the suspected defect area;

[0110] In the embodiment of the present invention, please refer to Figure 5 The anomaly detection neural network uses ResNet18 as the encoder for the middle three-layer feature extraction, and performs end-to-end semantic segmentation to detect and locate abnormal areas; for details, please refer to Figure 2Resnet18 extracts features from the three middle layers after removing the full-link layer and the activation layer, and constructs a normal feature sample library (Normal Bank) and an abnormal feature sample library (Defect Bank) respectively; the differential attention module (Differential Attention) integrated after Resnet18 makes full use of the difference information of the features in the input normal feature sample library, and re-encodes the extracted features of the three middle layers, so as to obtain a more effective feature representation, achieving high accuracy and extremely low missed detection rate.

[0111] The differential attention module (Differential Attention) integrated after Resnet18 has the following structure: Figure 3 As shown, the middle three-layer feature IF is used to calculate the mean square error between the middle three-layer feature IF and the normal features in the normal feature sample library NormalBank, and the difference information of the features in the input normal feature sample library is fully utilized to generate differential features (Differential Feature), and the differential features are combined with the original features through the differential attention module, the extracted middle three-layer features are re-encoded, and the re-encoded features are input into the decoder to generate a prediction result mask to obtain the prediction result to identify anomalies. The re-encoded features that integrate the differential information are more discriminative and accurate, thereby effectively improving the accuracy of the anomaly detection network.

[0112] Step 43: Input the suspected defect area into a classification neural network, and the classification neural network outputs the probability that the suspected defect area is a normal area. When the probability is less than a preset probability threshold, it is an abnormal image.

[0113] In the embodiment of the present invention, please refer to Figure 5 , Resnet-50 is used as the classification neural network, and the output of the classification neural network is the probability that the regional image is the suspected defect area and the normal area. By using the classification neural network to further determine whether the image contains defects, the detection accuracy and extremely low missed detection rate are improved, and the data and time costs of algorithm development and iterative upgrades are saved, which solves the existing problems of requiring a large number of abnormal sample training and poor robustness to different wheel models due to the limitations of image acquisition equipment and the algorithm itself.

[0114] Please refer to Figure 6 , an embodiment of the present invention provides a training device for anomaly detection neural network, comprising:

[0115] A first acquisition module 61 is used to acquire a training set of sample wheel hub images;

[0116] The first processing module 62 is used to extract the middle three-layer features of each wheel image in the training set to generate a normal feature sample library, wherein the middle three-layer features are three convolution layers after removing the full connection layer and the activation layer;

[0117] The second processing module 63 is used to compare the intermediate three-layer features with the normal features in the normal feature sample library to generate differential features; obtain a prediction result according to the differential features and the intermediate three-layer features, and the prediction result includes whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub;

[0118] The first optimization module 64 is used to optimize the parameters of the anomaly detection neural network according to the prediction result to obtain a trained anomaly detection neural network.

[0119] In the embodiment of the present invention, optionally, the training set includes: a normal wheel hub image sample set, and an abnormal wheel hub image sample set generated by a pseudo-abnormal generation method based on the normal wheel hub image sample set.

[0120] In the embodiment of the present invention, optionally, the first acquisition module includes:

[0121] A first processing submodule is used to capture sample panoramic images of a plurality of sample hubs and extract sample feature information of the sample panoramic images;

[0122] The second processing submodule is used to correspond the sample feature information to the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, calculate the angle between the sample wheel hub and the preset feature points of the initial position, and rotate the sample wheel hub to reach the preset shooting point position of the sample wheel hub;

[0123] The third processing submodule is used to take sample images of each surface of the sample hub at preset shooting points of the sample hub to obtain a training set of sample hub images.

[0124] In the embodiment of the present invention, optionally, the following further comprises:

[0125] The third processing module is used to obtain panoramic pictures of multiple models of wheel hubs and extract feature information of the panoramic pictures of the multiple models of wheel hubs, wherein the feature information includes feature points and feature descriptors, thereby obtaining a feature point library and a feature descriptor library, and presetting shooting points according to the structural characteristics of each wheel hub model and the feature information to obtain the wheel hub model library.

[0126] In the embodiment of the present invention, optionally, the method further includes:

[0127] An extraction module, used to extract the middle three-layer features of each wheel hub image in the training set and generate an abnormal feature sample library;

[0128] The second optimization module is used to construct a data triplet including the features in the intermediate three-layer features, the normal feature sample library and the abnormal feature sample library, and optimize the intermediate three-layer features so that the distance between the intermediate three-layer features and the normal feature sample library is smaller than the distance between the intermediate three-layer features and the abnormal feature sample library.

[0129] In the embodiment of the present invention, optionally, the second processing module includes:

[0130] The re-encoding submodule is used to re-encode the intermediate three-layer features according to the differential features to obtain re-encoded features, and input the re-encoded features into the decoder to obtain prediction results.

[0131] In the embodiment of the present invention, optionally, the first optimization module includes:

[0132] The optimization submodule is used to optimize the parameters of the anomaly detection neural network using the absolute error loss function and the focal loss function and the prediction result to obtain a trained anomaly detection neural network.

[0133] The training device for abnormality detection neural network provided by the embodiment of the present invention can realize Figure 1 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0134] Please refer to Figure 7 , an embodiment of the present invention provides a device for detecting wheel hub defects, comprising:

[0135] The third acquisition module 71 is used to acquire target images of multiple surfaces of the wheel hub to be inspected;

[0136] The fourth processing module 72 is used to input the target image into the abnormality detection neural network as described in any one of the first aspects, the abnormality detection neural network extracts the middle three-layer features of the target image of the multiple surfaces of the wheel hub to be detected, and generates a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer; compare the middle three-layer features with the normal features in the normal feature sample library to generate differential features; obtain a detection result based on the differential features and the middle three-layer features, the detection result is whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub, and output the abnormal area of ​​the suspected abnormal wheel hub as a suspected defect area;

[0137] The fifth processing module 73 is used to input the suspected defect area into a classification neural network, and the classification neural network outputs the probability that the suspected defect area is a normal area. When the probability is less than a preset probability threshold, it is an abnormal image.

[0138] In the embodiment of the present invention, optionally, the third acquisition module includes:

[0139] A first shooting module is used to shoot a panoramic image of the wheel hub to be inspected and extract target feature information of the panoramic image;

[0140] The sixth processing module is used to correspond the target feature information to the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, calculate the angle between the wheel hub to be detected and the preset feature points of the initial position, and rotate the wheel hub to be detected to reach the preset shooting point position of the wheel hub to be detected;

[0141] The second shooting module is used to shoot target images of various surfaces of the wheel hub to be tested at preset shooting points of the wheel hub to be tested.

[0142] The wheel hub defect detection device provided by the embodiment of the present invention can achieve Figure 4 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0143] An embodiment of the present invention provides an electronic device 80, see Figure 8 As shown, Figure 8 It is a principle block diagram of an electronic device 80 according to an embodiment of the present invention, including a processor 81, a memory 82, and a program or instruction stored in the memory 82 and executable on the processor 81. When the program or instruction is executed by the processor, the steps in the training method for an abnormality detection neural network described in any one of the present invention are implemented, or the steps in the method for wheel hub defect detection described in any one of the present invention are implemented.

[0144] An embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the training method for an abnormality detection neural network described in any one of the present invention are implemented, or the steps of the method for wheel hub defect detection described in any one of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0145] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 or Figure 4 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0146] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0147] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0148] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0149] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0151] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A training method for anomaly detection neural network, characterized in that: include: Obtain a training set of sample wheel hub images; Extracting the middle three-layer features of each wheel hub image in the training set to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer; Compare the intermediate three-layer features with normal features in a normal feature sample library to generate differential features; obtain prediction results based on the differential features and the intermediate three-layer features, the prediction results including whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub; Optimize the parameters of the anomaly detection neural network according to the prediction results to obtain a trained anomaly detection neural network.

2. The training method for anomaly detection neural network according to claim 1, characterized in that: The training set includes: a normal wheel hub image sample set, and an abnormal wheel hub image sample set generated by a pseudo-abnormal generation method based on the normal wheel hub image sample set.

3. The training method for anomaly detection neural network according to claim 1, characterized in that: The step of obtaining a training set of sample wheel hub images comprises: Taking sample panoramic images of a plurality of sample hubs, and extracting sample feature information of the sample panoramic images; The sample feature information is matched with the preset feature point of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the angle between the sample wheel hub and the preset feature point of the initial position is calculated, and the sample wheel hub is rotated to reach the preset shooting point of the sample wheel hub; Sample images of each surface of the sample hub at preset shooting points of the sample hub are photographed to obtain a training set of sample hub images.

4. The training method for anomaly detection neural network according to claim 3, characterized in that: Before the sample feature information is matched with the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the method further includes: Panoramic pictures of multiple models of wheel hubs are obtained, and feature information of the panoramic pictures of the multiple models of wheel hubs is extracted, the feature information including feature points and feature descriptors, a feature point library and a feature descriptor library are obtained, and shooting points are preset according to the structural characteristics of each wheel hub model and the feature information to obtain the wheel hub model library.

5. The training method for anomaly detection neural network according to claim 1, characterized in that: The step of comparing the intermediate three-layer features with normal features in a normal feature sample library to generate differential features may also include: Extracting the middle three-layer features of each wheel hub image in the training set to generate an abnormal feature sample library; A data triplet including the features in the intermediate three-layer features, the normal feature sample library and the abnormal feature sample library is constructed, and the intermediate three-layer features are optimized so that the distance between the intermediate three-layer features and the normal feature sample library is smaller than the distance between the intermediate three-layer features and the abnormal feature sample library.

6. The training method for anomaly detection neural network according to claim 1, characterized in that: The obtaining of prediction results according to the differential features and the intermediate three-layer features includes: The intermediate three-layer features are re-encoded according to the differential features to obtain re-encoded features, and the re-encoded features are input into a decoder to obtain a prediction result.

7. The training method for anomaly detection neural network according to claim 1, characterized in that: The step of optimizing the parameters of the abnormality detection neural network according to the predicted abnormal wheel hub and abnormal area to obtain a trained abnormality detection neural network includes: The parameters of the anomaly detection neural network are optimized using the absolute error loss function, the focal loss function and the prediction result to obtain a trained anomaly detection neural network.

8. A method for detecting wheel hub defects, characterized in that: include: Acquire target images of multiple surfaces of the wheel hub to be inspected; The target image is input into the abnormality detection neural network as described in any one of claims 1 to 7, and the abnormality detection neural network extracts the intermediate three-layer features of the target image of multiple surfaces of the wheel hub to be detected, and generates a normal feature sample library, wherein the intermediate three-layer features are three convolutional layers after removing the full connection layer and the activation layer; the intermediate three-layer features are compared with the normal features in the normal feature sample library to generate differential features; a detection result is obtained according to the differential features and the intermediate three-layer features, and the detection result is whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub, and the abnormal area of ​​the suspected abnormal wheel hub with the detection result being output as a suspected defect area; The suspected defect area is input into a classification neural network, and the classification neural network outputs the probability that the suspected defect area is a normal area. When the probability is less than a preset probability threshold, it is an abnormal image.

9. The method for detecting wheel hub defects according to claim 8, characterized in that: The step of acquiring target images of various surfaces of the wheel hub to be inspected comprises: Taking a panoramic image of the wheel hub to be inspected, and extracting target feature information of the panoramic image; The target feature information is matched with the preset feature points of the initial position of the most similar wheel hub model matched in the pre-stored wheel hub model library, the angle between the wheel hub to be detected and the preset feature points of the initial position is calculated, and the wheel hub to be detected is rotated to reach the preset shooting point position of the wheel hub to be detected; The target images of each surface of the wheel hub to be inspected at the preset shooting points of the wheel hub to be inspected are captured.

10. A training device for anomaly detection neural network, characterized in that: include: A first acquisition module is used to acquire a training set of sample wheel hub images; The first processing module is used to extract the middle three-layer features of each wheel hub image in the training set to generate a normal feature sample library, wherein the middle three-layer features are three convolutional layers after removing the full connection layer and the activation layer; The second processing module is used to compare the intermediate three-layer features with the normal features in the normal feature sample library to generate differential features; obtain a prediction result according to the differential features and the intermediate three-layer features, and the prediction result includes whether it is a suspected abnormal wheel hub and the abnormal area of ​​the suspected abnormal wheel hub; The first optimization module is used to optimize the parameters of the anomaly detection neural network according to the prediction result to obtain a trained anomaly detection neural network.

11. A wheel hub defect detection device, characterized in that: include: A third acquisition module is used to acquire target images of multiple surfaces of the wheel hub to be inspected; A fourth processing module, for inputting the target image into an abnormality detection neural network as described in any one of claims 1 to 7, wherein the abnormality detection neural network extracts the intermediate three-layer features of the target image of multiple surfaces of the wheel hub to be detected, and generates a normal feature sample library, wherein the intermediate three-layer features are three convolutional layers after removing the full connection layer and the activation layer; comparing the intermediate three-layer features with the normal features in the normal feature sample library to generate differential features; obtaining a detection result based on the differential features and the intermediate three-layer features, wherein the detection result is whether it is a suspected abnormal wheel hub and an abnormal area of ​​the suspected abnormal wheel hub, and outputting the abnormal area of ​​the suspected abnormal wheel hub as a suspected defect area if the detection result is a suspected abnormal wheel hub; The fifth processing module is used to input the suspected defect area into a classification neural network, and the classification neural network outputs the probability that the suspected defect area is a normal area. When the probability is less than a preset probability threshold, it is an abnormal image.

12. An electronic device, characterized in that: The method comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the training method for anomaly detection neural network as described in any one of claims 1 to 7, or implements the steps in the method for wheel hub defect detection as described in any one of claims 8 or 9.

13. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps in the training method of the abnormality detection neural network as described in any one of claims 1 to 7 are implemented, or the steps in the method of wheel hub defect detection as described in any one of claims 8 or 9 are implemented.

14. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps in the training method of an abnormality detection neural network as described in any one of claims 1 to 7, or implement the steps in the method of wheel hub defect detection as described in any one of claims 8 or 9.