Method and device for detecting subtle defects on open set product surfaces based on machine haptics

Abstract

The application provides a kind of open set product surface subtle defect detection method and equipment based on machine haptics, belongs to product surface defect intelligent detection field, comprising: collecting the haptic signal of different categories of products and constructing data set;Construct open set texture suppression model, it includes preprocessing module, hierarchical reverse attention module and post-processing module;Construct open set defect detection model, it includes self-fusion data enhancement module, label smoothing module, supervised contrast learning module of soft label and lightweight projection head module;Using machine haptics method collects the haptic signal of product to be measured, first by open set texture suppression model to suppress background texture interference, then through open set defect detection model to the image after filtering background texture is distinguished, and then the specific detection result is obtained.The application realizes the open set product surface subtle defect detection across categories, effectively avoids the false alarm and missed detection of surface subtle defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent detection of product surface defects, and in particular to a method and device for open-set product surface subtle defect detection based on machine haptics. BACKGROUND

[0002] Intelligent detection of product surface defects plays a strategic and fundamental role in the field of intelligent manufacturing. In the process of modern manufacturing development, high-precision surface defect recognition technology can not only effectively ensure product quality stability and avoid engineering safety risks such as mechanical property degradation caused by surface defects (such as stiffness attenuation caused by surface cracks in metal materials), but also provide data-driven decision-making basis for dynamic optimization of the production process by establishing a mapping relationship between defect characteristics and process parameters. Therefore, product surface defect detection technology has very important application value and application prospect.

[0003] Machine haptic sensing can obtain detailed surface three-dimensional topography through flexible touch, thereby capturing micron-level surface defects invisible to the naked eye, providing a new solution for surface subtle defect recognition in precision manufacturing processes. However, existing haptic detection methods still face some open-set challenges in industrial application scenarios, which are as follows: (1) In the flexible customization production mode, the open-set uncertainty of product surface textures of different batches or models leads to overlapping areas between unknown normal textures and defect patterns, which easily causes false alarm and misjudgment; (2) The open-set heterogeneity of defect categories among different product categories makes it difficult for pre-trained classification models to effectively represent unknown defect patterns of new products, which has a risk of missing detection.

[0004] Although current open-set detection methods based on deep learning have been applied in specific industrial scenarios, there are obvious limitations in their technical path. Existing research mainly focuses on the solution of a single sub-problem: one type of method realizes adaptive modeling of background appearance through open-set texture suppression strategy, but lacks the ability to handle open defects; another type of method focuses on the construction of open-set defect recognition mechanism, but fails to effectively cope with dynamic changes in product textures. This one-dimensional technical path leads to the inability of existing methods to meet the cross-domain adaptability requirements of detection systems, resulting in serious performance degradation in cross-category open-set detection scenarios, and further causing potential safety production risks and product quality risks. Therefore, how to realize open-set product surface subtle defect detection based on machine haptics is an engineering problem that needs to be solved urgently. SUMMARY

[0005] The present application provides an open-set product surface subtle defect detection method based on machine haptics, which aims to realize cross-category open-set product surface subtle defect detection by constructing a two-stage framework of open-set background suppression and open-set defect recognition based on deep learning technology.

[0006] In a first aspect, the present invention provides a method for detecting minute defects on the surface of open-collection products based on machine tactile sensing, comprising:

[0007] S1. Use a robotic arm to hold a tactile sensor to collect tactile signals from different product categories with different background textures and defect patterns, and build a dataset.

[0008] S2. Construct the first-stage open-set texture suppression model, which includes a preprocessing module, a hierarchical inversion attention module, and a postprocessing module. The preprocessing module is used to extract the initial hierarchical features. The hierarchical inversion attention module inverts attention layer by layer from deep to shallow, forcing the model to relearn more texture information. The postprocessing module uses a segmentation method to suppress background texture pixels and completes the suppressed pixels.

[0009] S3. Construct the second-stage open-set defect detection model, which includes a self-fusion data augmentation module, a label smoothing module, a soft-label supervised contrastive learning module, and a lightweight projection head module. The self-fusion data augmentation module is used to obtain boundary negative samples with high similarity and low attribution. Label smoothing is used to mitigate the surge in open-space risk caused by overconfidence in traditional one-hot encoding during the prediction stage. The soft-label supervised contrastive learning module improves the discriminative ability of defect patterns by maximizing inter-class dissimilarity and intra-class compactness. The lightweight projection head module is used to obtain classification... logit And based on percentile statistics, a category-specific rejection threshold is constructed;

[0010] S4. Use machine tactile methods to collect tactile signals of the product under test. First, use an open set texture suppression model to suppress background texture interference. Then, use an open set defect detection model to judge the image after filtering out background texture, and then obtain specific detection results.

[0011] The present invention provides a method for detecting minute defects on the surface of open-set products based on machine tactile sensing, characterized in that a pre-trained preprocessing module extracts the original image in an open-set texture suppression model. Multi-scale feature maps at different depths Then input the hierarchical inverted attention module.

[0012] According to the present invention, an open-set product surface micro-defect detection method based on machine tactile sensing is provided, wherein the hierarchical inversion attention module of the open-set texture suppression model includes four recursively executed stages: fusion, inversion, correction, and update.

[0013] During the fusion stage, features from different input scales are first unified to the scale of the shallowest feature map, then pixel-level fusion is performed, followed by intensity normalization of global pixels to obtain a preliminary localization map. ;

[0014] In the reverse stage, the last layer of the First, the spatial dimension is aligned to Then, all pixel intensities are reversed to generate the reverse attention map ;

[0015] In the correction stage, the inverse attention weight Element-level multiplication is performed with the current layer feature And adaptive feature enhancement is achieved through a learnable convolution kernel to obtain the corrected feature ;

[0016] In the update stage, the deep positioning clues And the corrected feature of the current layer are fused through a residual connection to obtain the updated positioning map of the current layer ;

[0017] The above process is repeated layer by layer from deep to shallow until the prediction mask is output at the shallowest layer .

[0018] According to the open set product surface subtle defect detection method based on machine touch provided by the application, the output of each layer of the hierarchical reverse attention module of the open set texture suppression model is subjected to a depth supervision mechanism.

[0019] According to the open set product surface subtle defect detection method based on machine touch provided by the application, the post-processing module of the open set texture suppression model applies To the original image Then, all zero-intensity pixels are re-assigned to the average pixel intensity of all samples in the data set to obtain an image filtered of background texture .

[0020] According to the open set product surface subtle defect detection method based on machine touch provided by the application, the self-fusion data enhancement module of the open set defect detection model generates high-quality negative samples through Body rotation and superposition operations, combined with the binary mask output by the open set texture suppression model .

[0021] According to the open set product surface subtle defect detection method based on machine touch provided by the application, the label smoothing module of the open set defect detection model uses a smoothing coefficient To construct a soft label with information entropy constraint The supervised contrast learning module of the soft label of the open set defect detection model divides positive and negative examples through the measurement value of cosine similarity. ​

[0022] The application provides a machine tactile-based open-set product surface subtle defect detection method. The light-weight projection head module of the open-set defect detection model adopts a three-layer feedforward neural network architecture to map a representation vector of a sample to a target category. .

[0023] The application provides a machine tactile-based open-set product surface subtle defect detection method, which further comprises the following steps of dynamically constructing a category-specific confidence set for storing logit values of all correctly classified samples in an open-set defect detection model training process.

[0024] The application provides a machine tactile-based open-set product surface subtle defect detection method, which further comprises the following steps of setting a confidence percentile of a category-specific confidence set as a default rejection threshold of the category based on a preset confidence parameter after completing an open-set defect detection model training stage. k k

[0025] In a second aspect, the application provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the machine tactile-based open-set product surface subtle defect detection method according to any one of the above aspects when executing the program.

[0026] In a third aspect, the application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program implements the steps of the machine tactile-based open-set product surface subtle defect detection method according to any one of the above aspects when executed by a processor.

[0027] Compared with the prior art, the machine tactile-based open-set product surface subtle defect detection method provided by the application has the following beneficial effects:

[0028] The application provides a machine tactile-based open-set product surface subtle defect detection method, which realizes cross-category open-set product surface subtle defect detection by constructing a two-stage framework of open-set background suppression and open-set defect recognition based on deep learning technology, effectively avoids false alarms caused by the uncertainty of different product surface textures and missed detection caused by defect category heterogeneity, and thus effectively guarantees product quality.

[0029] The application provides an open-set texture suppression model, which realizes open-set suppression of all-mode texture interference by iteratively enhancing defect saliency and dynamically adjusting background texture influence through a cascading architecture.

[0030] ​​​​The open set defect detection model is provided, different defect mode distinguishing features are captured through self-fusion data enhancement, label smoothing and soft label contrast learning, and high-precision identification of known defect modes and unknown defect modes is realized. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0032] Figure 1 is one of the flowcharts of the open set product surface subtle defect detection method based on machine haptics provided by the present application;

[0033] Figure 2 is the second flowchart of the open set product surface subtle defect detection method based on machine haptics provided by the present application;

[0034] Figure 3 is a schematic diagram of the haptic signal acquisition process and the data set construction process provided by the present application;

[0035] Figure 4 is a schematic diagram of the constructed open set texture suppression model structure provided by the present application;

[0036] Figure 5 is a schematic diagram of the constructed open set defect detection model structure provided by the present application;

[0037] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0039] It should be noted that in the description of the embodiments of the present application, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent in such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. The above terms can be understood in their specific meaning in the present application according to the specific circumstances.

[0040] The following will be described in combination with Figures 1-6 The method for detecting fine defects on the surface of open set products based on machine haptics provided by the embodiments of the present application is described.

[0041] Figure 1 is one of the flowcharts of the method for detecting fine defects on the surface of open set products based on machine haptics provided by the present application, Figure 2 is the second flowchart of the method for detecting fine defects on the surface of open set products based on machine haptics provided by the present application, including but not limited to the following steps:

[0042] S1, using a mechanical arm to hold a haptic sensor to collect haptic signals of different categories of products with different background textures and defect patterns, and constructing a data set.

[0043] Figure 3 is a schematic diagram of the process of collecting haptic signals and constructing a data set provided by the present application, as Figure 3 shown, using a mechanical arm to hold a vision-based haptic sensor to traverse the surface to be tested of the sample, collect haptic signals of different categories of products with different background textures and defect patterns, and mark the pixel-level label of each haptic sample by manual marking method. After marking is completed, the marking result is reviewed by a domain expert, and a haptic signal data set with pixel-level marking is constructed.

[0044] S2, constructing an open set texture suppression model in the first stage, which includes a preprocessing module, a hierarchical reverse attention module and a post-processing module.

[0045] Figure 4 is a schematic diagram of the structure of the constructed open set texture suppression model provided by the present application, as Figure 4As shown, a first-stage open-set texture suppression model is constructed based on deep learning technology. The preprocessing module extracts hierarchical preliminary features; the hierarchical inversion attention module inverts attention layer by layer from deep to shallow, forcing the model to relearn more texture information to highlight the salience of defect areas; the post-processing module uses a segmentation method to suppress background texture pixels and completes the suppressed pixels. Specifically:

[0046] (1) The preprocessing module of the open set texture suppression model adopts the Res2Net network architecture with a depth of 5 layers. It is pre-trained using the publicly available steel surface defect detection dataset NEU. After training, the classification layer is discarded, and only the feature extraction layer is retained as the preprocessing module.

[0047] (2) The pre-trained preprocessing module extracts the original image. Multi-scale feature maps of layers 3, 4, and 5 Then input the hierarchical inverted attention module.

[0048] (3) The hierarchical inversion attention module includes four recursively executed phases: fusion, inversion, correction, and update.

[0049] During the fusion stage, features from different input scales are first unified to the scale of the shallowest feature map, then pixel-level fusion is performed, followed by intensity normalization of global pixels to obtain a preliminary localization map. At this point, the deepest layer i=5, therefore the initial localization map is actually as follows: :

[0050]

[0051] During the reversal phase, the upper layer... First it will be aligned to The spatial dimension is then reversed, and all pixel intensities are inverted to generate an inverted attention map. :

[0052]

[0053] In the formula This is a downsampling operation; the superscript indicates the scaling factor. It is the sigmoid function. It is a one-dimensional matrix. This operation establishes a negation space for defect regions in the form of probability complements, forcing the model to focus on texture regions ignored by the initial prediction.

[0054] During the correction phase, the inverse attention weights will be... With current layer features Perform Hadamard product and use learnable convolution kernels. Adaptive feature enhancement is implemented to obtain the modified feature :

[0055]

[0056] In the update stage, the deep layer positioning clues are fused through the residual connection to obtain the updated positioning map of the current layer and the modified feature of the current layer . :

[0057]

[0058] The operation builds a self-deep-to-shallow layer recursive mechanism, so that the original positioning reference is gradually updated through layer-by-layer enhancement, and guides the next layer modification until the final prediction result is output .

[0059] The output of each level of the above process is subjected to depth supervision. The loss function is set as the weighted loss of Dice loss and BCEloss , so as to maintain the stability of pixel-level classification and the robustness of region overlap matching.

[0060] (4) The output prediction mask is first applied to the original image through pixel-by-pixel multiplication , and then all zero-intensity pixels are re-assigned to the average pixel intensity of all samples in the data set. The adaptive thresholding process eliminates high-frequency texture components through spatial domain filtering, and enhances the contrast difference between the defect area and the uniform background by means of intensity normalization strategy. The output normalized image provides a high-quality sample for subsequent defect recognition.

[0061] S3, a second-stage open-set defect detection model is constructed, which includes a self-fusion data enhancement module, a label smoothing module, a supervised contrast learning module of soft label and a lightweight projection head module.

[0062] Based on deep learning technology, a second-stage open-set defect detection model is constructed, Figure 5 which is a structure diagram of the constructed open-set defect detection model provided by the present application, as shown in Figure 5are shown, where self-fusion data augmentation is used to obtain boundary negative samples with high similarity and low attribution; label smoothing is used to alleviate the risk of open space risk surge caused by the overconfidence of traditional one-hot encoding in the prediction stage; the supervised contrast learning module of soft label improves the discriminability of defect mode by maximizing the difference between classes and the compactness within classes; the lightweight projection head is used to obtain the classification logit, and the rejection threshold of the category is constructed based on the percentile statistics, so as to accurately divide the decision boundary of the closed set feature space and the open space. Specifically:

[0063] (1) The self-fusion data augmentation module generates high-quality negative samples by combining the rotation and superposition operations of the original image and the binary mask output by the open set texture suppression model . For a sample , its corresponding negative sample is defined as:

[0064]

[0065] In the formula, N represents the total number of closed set classes, is the rotation angle , and the affine transformation is randomly sampled from a uniform distribution . The generation process of this operation is based on the inherent characteristics of the same sample, only uses local transformation to distort the key defect mode, and retains the original contour, thereby avoiding the introduction of additional open space risk in the global fusion process of the classic Mixup method, so that the negative sample has the properties of low attribution and high similarity. Since the background mode has been unified to a constant value, the generation process will not introduce new background noise, so it has good adaptability.

[0066] (2) The label smoothing module constructs a soft label with information entropy constraint :

[0067]

[0068] In the formula, is the smoothing factor, is the total number of classes containing negative samples. This operation reduces the prediction confidence variance of the model for known classes, alleviating the distribution shift problem between training and testing. At the same time, it reserves the lower bound of the margin for subsequent closed space and open space feature boundary learning.

[0069] (3) The supervised contrast learning module of soft label divides positive and negative examples by the measurement value of cosine similarity, which naturally compatible with one-hot. For a pair of samples and , the difference between their soft label vectors is calculated using cosine similarity and as the loss of the supervised contrastive learning module with soft labels :

[0070]

[0071]

[0072] where is the representation vector of the sample , is a hyper-parameter to control the similarity of representation vectors. Thanks to the inherent property of cosine similarity, if and belong to the same class, , otherwise , the loss function will degenerate into the standard supervised contrastive loss, thus achieving the compatibility of hard and soft labels.

[0073] (4) The lightweight projection head module adopts a 3-layer feedforward neural network architecture, and its core function is to map the representation vector of the sample to the target class . The cross-entropy loss function based on soft labels can be defined as:

[0074]

[0075] where represents the predicted probability of the sample's real class k, corresponds to the prediction probability distribution of non-target classes. In the model training process, the system dynamically constructs a class-specific confidence set to store the logit values of all correctly classified samples. After the training stage is completed, the system sets the percentile in as the default rejection threshold of class k based on the pre-set confidence parameter .

[0076] (5) In the inference stage, if the of the sample is , it will be classified as type k, otherwise it will be rejected as an unknown type.

[0077] S4, collect the tactile signal of the product to be tested using the machine tactile method, first suppress the background texture interference by the open set texture suppression model, then distinguish the image filtered by the background texture through the open set defect detection model, and then obtain the specific detection result.

[0078] It can be understood that the tactile signal of the product to be tested can be collected using the same machine tactile method as S2.

[0079] The following is a specific embodiment:

[0080] In order to verify the actual application effect of the present application, the present application takes 304 steel plate as the base material, simulates different surface textures through sand blasting process, and simulates typical defect patterns using micro-processing technology. Then the tactile signal of the product to be tested is collected using the machine tactile method and the HUST-Tac data set is made. Finally, the verification is carried out on the data set, and the common defect detection algorithm is compared.

[0081] HUST-Tac consists of 2844 samples, which are divided into four superclasses according to the surface texture pattern, and each superclass contains five subclasses representing different defect types. The texture superclass is marked as smooth, #150, #120 and #80 from the smoothest to the roughest, wherein the "#" symbol represents the particle size used in the sand blasting process during the manufacturing of rough texture. The defect subclass includes protrusion, pit, scratch, crack and normal (representing a defect-free surface). In order to effectively simulate the defect detection in an open environment, the normal, protrusion, pit and scratch samples of the smooth texture class are included in the training set, while the samples with gradient texture and the samples with crack defect are considered as the attributes of new products and are only used in the test set. A stratified random sampling strategy is adopted to divide the samples into training and test subsets in a ratio of 9:1, thereby ensuring the generalization ability of the model and reducing the risk of overfitting or local optimum in the parameter optimization process. Table 1 provides detailed information of the data set, wherein "-" represents an empty set and " / " represents a parallel relationship.

[0082] Table 1 HUST-Tac data set

[0083]

[0084] The performance of the open set texture suppression model is tested on the data set, and different evaluation indicators are adopted to comprehensively evaluate the model performance, including precision, recall, harmonic index F1 score of precision and recall, mean absolute error, intersection over union, and enhanced matching standard .

[0085] Among them, the F1 score reflects the overall discrimination ability of the model to the defect target from a macroscopic perspective, the mean absolute error and the intersection over union reveal the precision and positioning accuracy of the pixel-level prediction from a microscopic perspective, and the enhanced matching standard The overall and local accuracy of the defect foreground and texture background segmentation results are evaluated while considering pixel-level matching and image-level statistical information. The average of five experiments is taken for each quantitative index to avoid randomness, and the experimental results are summarized in Table 2, and the arrow direction represents better.

[0086] Table 2 Quantitative indicators of open set texture suppression model

[0087]

[0088] Through the quantitative evaluation analysis of Table 2, it can be seen that the open set texture suppression model proposed in the present application is significantly better than the existing method in various performance indicators. Further analysis shows that DDNet lacks multi-scale feature fusion and pre-positioning mechanism, resulting in insufficient defect boundary capture capability, and thus causing significant missed detection phenomenon. Although SINet-V2 solves the pre-positioning problem through region search strategy, the primary texture enhancement module it uses cannot provide more advanced information, resulting in a sharp drop in performance in complex texture patterns. In contrast, the open set texture suppression model proposed in the present application innovatively introduces a hierarchical reverse attention module to realize progressive optimization of feature maps, and ultimately maintains stable detection performance in complex texture samples.

[0089] Then the image filtered by the open set texture suppression model is further used to test the performance of the open set defect detection model, and a multi-dimensional evaluation system is constructed for the open set defect detection task to systematically quantify the comprehensive performance of the model in known class recognition and unknown sample detection. Among them, the closed set accuracy evaluates the closed set structural risk, the area under the ROC curve of unknown class evaluates the open space risk, the comprehensive accuracy and macro F1 score jointly measure the synergistic effect of known class discrimination and unknown class rejection, and the threshold-independent macro ROC curve area index evaluates the feature separability of the model in all classes. The average of five experiments is taken for each quantitative index to avoid randomness, and the experimental results are summarized in Table 3, and the arrow direction represents better.

[0090] Table 3 Quantitative indicators of open set defect detection model

[0091]

[0092] Through the quantitative evaluation analysis of Table 3, it can be known that the open set defect detection model proposed in the present research is significantly better than the existing method in various performance indicators. Further analysis can know that OVRNs and ConOSR improve the separability through the comparison of deep features, but the lack of negative sample generation mechanism of known class boundary leads to fuzzy decision boundary. In contrast, the open set defect detection model proposed in the present application introduces a boundary-aware negative sample generation mechanism to construct adversarial negative samples with low attribution and high similarity attributes, and designs a soft label contrast learning strategy to realize fine-grained feature decoupling, and finally achieves optimal intra-class tightness and inter-class separation, so as to obtain the optimal performance. It should be noted here that the input in this experiment stage is the image filtered by the open set texture suppression model, and if the open set texture suppression stage is not shared by the existing method, the performance of the existing method will be further reduced.

[0093] In summary, the above experimental results verify the superiority of the machine tactile based open set product surface micro-defect detection method in the cross-product defect detection task, and highlight its effectiveness and application potential as a cross-category open set product surface micro-defect detection method.

[0094] The machine tactile based open set product surface micro-defect detection method provided by the present application has the following beneficial effects compared with the prior art:

[0095] The present application provides a machine tactile based open set product surface micro-defect detection method, which has the following beneficial effects compared with the prior art:

[0096] The present application provides a machine tactile based open set product surface micro-defect detection method, which has the following beneficial effects compared with the prior art:

[0097] The present application provides a machine tactile based open set product surface micro-defect detection method, which has the following beneficial effects compared with the prior art:

[0098] Figure 6 is a structural schematic diagram of an electronic device provided by the present application, such as Figure 6As shown, the electronic device can include a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 complete communications with each other through the communications bus 640. The processor 610 can invoke a logic instruction in the memory 630 to execute the machine-tactile-based open-set product surface subtle defect detection method.

[0099] In addition, the logic instruction in the memory 630 described above can be implemented in the form of a software function unit and sold or used as an independent product when used, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0100] In another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the machine-tactile-based open-set product surface subtle defect detection method provided by the above-mentioned embodiments.

[0101] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

[0102] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A machine-tactile based open-set product surface micro-defect detection method, characterized in that, Comprise: S1, using a mechanical arm to hold a tactile sensor to collect tactile signals of different categories of products with different background textures and defect patterns, and construct a data set; S2, construct the first stage of open set texture suppression model, the model includes preprocessing module, hierarchical reverse attention module and post-processing module; wherein the preprocessing module is used to extract hierarchical preliminary features; the hierarchical reverse attention module reverses the attention from deep to shallow layer by layer, forcing the model to relearn more texture information; the post-processing module uses segmentation method to suppress background texture pixels, and fills in the pixels after being suppressed; the preprocessing module of the open set texture suppression model adopts Res2Net network architecture, which is pre-trained using the public steel surface defect detection data set, and after training, the classification layer is discarded, only the feature extraction layer is reserved as the preprocessing module; S2 specifically comprises: The preprocessing module of the open set texture suppression model adopts Res2Net network architecture with a depth of 5 layers; the public steel surface defect detection data set is used for pre-training, and after training, the classification layer is discarded, only the feature extraction layer is reserved as the preprocessing module; A pre-trained preprocessing module extracts the original image of 3,4,5 layers of multi-scale feature maps and then inputs a hierarchical reverse attention module The hierarchical reverse attention module includes four recursive execution stages of fusion, reverse, correction and update: In the fusion stage, the input features of different scales are first unified to the scale of the shallowest feature map, then pixel-level fusion is performed, and then the intensity of the global pixel is normalized to obtain a preliminary positioning map Therefore, the preliminary positioning map is actually : In the reverse phase, the previous layer's will first be aligned to the spatial dimensions of the , and then all pixel intensities will be reversed to generate the reverse attention map : wherein is a down-sampling operation, the superscript is a scaling factor, is a sigmoid function, is an all-one matrix; In the correction stage, the reverse attention weight is performed Hadamard product with the current layer feature , and the adaptive feature enhancement is realized through the learnable convolution kernel , and the corrected feature is obtained : In the update stage, the deep-level localization cues are fused through a residual connection and the modified features of the current layer to obtain the updated localization map of the current layer : To make the original positioning reference By layer-by-layer reinforcement, step-by-step updating, and guiding the next layer to correct until the final prediction result is output ; S3, build the open set defect detection model of the second stage, the model contains self-fusion data enhancement module, label smoothing module, soft label supervised contrast learning module and lightweight projection head module;Wherein, the self-fusion data enhancement is used to obtain the boundary negative sample with high similarity and low attribute;Label smoothing is used to alleviate the open space risk surge caused by the excessive self-confidence of traditional one-hot encoding in the prediction stage;The soft label supervised contrast learning module improves the discriminant ability of the defect mode by maximizing the difference between classes and the compactness within the class;The lightweight projection head module is used to obtain the classification logit , and based on the percentile statistics, the category-specific rejection threshold is constructed; S4, collect the tactile signal of the product to be tested using the machine tactile method, first suppress the background texture interference by the open set texture suppression model, then filter the background texture of the image by the open set defect detection model, and then get the specific detection result.

2. The machine-tactile based open-set product surface micro-defect detection method of claim 1, wherein, The output of each layer of the hierarchical reverse attention module of the open set texture suppression model is subjected to a depth supervision mechanism.

3. The machine-tactile based open-set product surface micro-defect detection method of claim 2, wherein, The post-processing module of the open set texture suppression model filters out the background texture by multiplying each pixel of the original image by the background texture suppression function , and then reassigning all zero-intensity pixels to the mean pixel intensity of all samples in the dataset, resulting in an image with the background texture filtered out .

4. The machine-tactile based open-set product surface micro-defect detection method of claim 3, wherein, The self-fusion data enhancement module of the open set defect detection model is used to The body rotation and superposition operation is combined with the binary mask output by the open set texture suppression model Generate high-quality negative samples.

5. The machine-tactile based open-set product surface micro-defect detection method of claim 4, wherein, The label smoothing module of the open set defect detection model uses a smoothing coefficient Constructing soft labels with information entropy constraints ; The supervised contrast learning module of the soft label of the open set defect detection model divides positive and negative examples by the measurement value of cosine similarity.

6. The machine-tactile based open-set product surface micro-defect detection method of claim 5, wherein, The lightweight projection head module of the open set defect detection model adopts a 3-layer feedforward neural network architecture to map the representation vector of the sample to the target category .

7. The machine-tactile based open-set product surface micro-defect detection method of claim 6, wherein, Also include: In the training process of the open set defect detection model, a class-specific confidence set is dynamically constructed to store the logit values of all correctly classified samples.

8. The machine-tactile based open-set product surface micro-defect detection method of claim 7, wherein, Also include: After the open set defect detection model training phase is completed, based on a preset confidence parameter , the category k with the highest confidence is set as the default rejection threshold. The confidence percentile of the category with the highest confidence is set as the default rejection threshold. The confidence percentile of the category k with the highest confidence is set as the default rejection threshold.

Images