Water purifier outer surface defect detection method based on deep learning

Abstract

The invention relates to the technical field of water purifier outer surface defect detection, and provides a water purifier outer surface defect detection method based on deep learning, and the method comprises the steps: constructing a water purifier outer surface multi-modal defect data set comprising a traditional image and an optical image; constructing a defect detection model based on the water purifier outer surface multi-modal defect data set; inputting a traditional image in the multi-modal defect data set of the outer surface of the water purifier into the high-precision defect classification model for training, and outputting traditional image features; inputting an optical image training set in the multi-modal defect data set of the outer surface of the water purifier into an anti-interference optical defect detection model for training, and outputting optical image features; inputting the traditional image features and the optical image features into a multi-modal sensing network to obtain a trained defect detection model; evaluating the trained defect detection model by using the verification set to obtain a verified defect detection model; and detecting the outer surface defects of the water purifier by using the verified defect detection model.

Description

Technical Field

[0001] This invention relates to the field of water purifier surface defect detection technology, and in particular to a deep learning-based method for detecting water purifier surface defects. Background Technology

[0002] As common household and industrial heating devices, water purifiers often suffer from various types of physical damage and manufacturing defects on their outer surfaces during production, transportation, and use, such as scratches, dents, cracks, rust, and poor welding. These defects not only affect the product's appearance and lifespan but may also reflect potential problems with internal processes or materials. Therefore, high-precision defect detection of water purifier outer surfaces has become a key step in ensuring product quality and improving manufacturing standards.

[0003] Currently, industrial inspection mainly relies on manual visual inspection or traditional image processing-based methods to identify defects on the surface of water purifiers. However, manual inspection is inefficient and highly subjective, making it difficult to meet the demands of large-scale, rapid production inspections. Traditional image processing methods, on the other hand, have limited ability to identify defects in complex backgrounds, low contrast, and small dimensions, easily leading to missed or false detections, and thus failing to meet high industrial quality standards.

[0004] Current technologies generally rely on a single RGB image as input, lacking the ability to perceive the complex optical information of water purifier surfaces in multiple dimensions. This is especially true when faced with defects such as reflective interference, similar textures, and minute cracks, resulting in insufficient robustness in detection. Due to the limitations of RGB images in terms of material representation and spatial perception, single-modal input cannot fully capture the multi-layered structural information of defect areas, thus limiting the detection performance of the model. Summary of the Invention

[0005] To address the aforementioned shortcomings, the present invention aims to propose a deep learning-based method for detecting defects on the outer surface of water purifiers. This method addresses the problems of insufficient ability to identify minute defects and poor robustness to complex lighting and surface interference in practical applications of existing single-image-modal-based methods for detecting defects on the outer surface of water purifiers. Furthermore, it solves the problem that existing technologies fail to fully utilize the complementary characteristics between multiple image modalities during defect detection, resulting in difficulties in ensuring detection accuracy and stability under various backgrounds with multiple types, scales, and interference.

[0006] To achieve this objective, the present invention adopts the following technical solution:

[0007] A method for detecting defects on the outer surface of a water purifier based on deep learning, the method comprising the following steps:

[0008] S1: Construct a multimodal defect dataset for the outer surface of a water purifier, including both traditional and optical images;

[0009] S2: Based on the multimodal defect dataset of the water purifier's outer surface from step S1, construct a defect detection model. The defect detection model includes a high-precision defect classification model based on a hierarchical attention mechanism, an anti-interference optical defect detection model oriented towards few-shot learning, and a multimodal perception network.

[0010] S3: Input traditional images from the multimodal defect dataset of the water purifier's outer surface into the high-precision defect classification model for training, and output traditional image features;

[0011] S4: Input the optical image training set from the multimodal defect dataset on the outer surface of the water purifier into the anti-interference optical defect detection model for training, and output the optical image features;

[0012] S5: Input traditional image features and optical image features into a multimodal perception network to obtain a trained defect detection model;

[0013] S6: Use the validation set to evaluate the trained defect detection model and obtain the validated defect detection model;

[0014] S7: Use the validated defect detection model to detect defects on the outer surface of the water purifier.

[0015] Preferably, step S1 includes the following steps:

[0016] S11: Acquire RGB images of the outer surface of the water purifier from frontal, upward, and oblique angles to obtain traditional images. Perform grayscale normalization on the traditional images and enhance the response of small defect areas in the traditional images through brightness equalization and edge enhancement algorithms to obtain traditional enhanced images.

[0017] S12: Perform standardized cropping and data augmentation processing, including rotation, scaling and color perturbation, on traditional enhanced images to obtain standard training image samples, and perform expert annotation of defect regions on the standard training image samples to construct a traditional image defect sample set.

[0018] S13: Acquire structured light interference images of the outer surface of the water purifier to obtain optical images, perform geometric distortion correction and image alignment on the optical images, and remove specular reflection interference through reflectivity normalization and bright spot suppression algorithms to obtain enhanced optical images;

[0019] S14: Perform illumination equalization and high-frequency detail restoration processing on the enhanced optical image, and enhance the surface weak defect texture features through a local contrast enhancement algorithm to obtain the final optical image. Subsequently, construct an optical image defect sample set through expert annotation.

[0020] S15: Construct training data for the traditional image defect sample set and the optical image defect sample set respectively, and divide them into training sets and validation sets for training the traditional image detection model and the optical image detection model according to the ratio;

[0021] The outputs of the traditional image detection model and the optical image detection model are used as inputs to the multimodal perception network.

[0022] Preferably, step S3 includes the following sub-steps:

[0023] S31: Use a deep convolutional backbone network to perform preliminary feature extraction on traditional images to obtain multi-level spatial semantic features;

[0024] S32: Input the backbone features of the traditional image into the multi-resolution module, and use convolution kernels of different scales to extract the features at multiple scales. Use 3×3, 5×5 and 7×7 convolution kernels to extract different texture and edge features;

[0025] S33: In the resolution module, residual connections and attention weight adaptive mechanisms are introduced to fuse and enhance the multi-scale features of step S32;

[0026] S34: Input the output of the resolution module to the global attention aggregation module, model the relationship between any pixels based on the non-local attention mechanism, and calculate the attention map to further weight the global semantics;

[0027] S35: Simultaneously input the output of the resolution module into the local attention aggregation module, and use a local window partitioning strategy and spatial coordinate guidance mechanism to extract the discrimination features of local micro-defect regions;

[0028] S36: The features output by the global attention aggregation module and the local attention aggregation module are fused together. Combining the global and local attention features of traditional images, a classifier is introduced to output the final defect category result. Preferably, step S3 further includes the following step: constructing the loss function of the traditional image detection model by introducing multi-branch feature alignment loss, attention consistency loss, and cross-entropy classification loss, and minimizing this loss function. The calculation formula of the loss function of the traditional image detection model is as follows:

[0029] L total =α·L align +β·L attn +γ·L cls ;

[0030] Where α, β, and γ are hyperparameters used to adjust the importance of each part of the loss;

[0031] L alignThe multi-branch feature alignment loss is used to constrain the consistency of the output features of the global attention aggregation module and the local attention aggregation module in the semantic space. Its calculation formula is as follows:

[0032]

[0033] in, and Let N represent the feature vectors output by the global attention aggregation module and the local attention aggregation module for the i-th sample, respectively, where N is the total number of samples.

[0034] L attn L represents the loss of attentional consistency. attn L is used to maintain the consistency of spatial attention distribution of the main features after processing by the multi-resolution module, the global attention aggregation module, and the local attention aggregation module. cls L represents the cross-entropy loss. cls The prediction accuracy used to supervise the final defect category satisfies the following relationship:

[0035]

[0036] Where C is the total number of defect categories, y i For real category labels, To predict probabilities.

[0037] Preferably, in step S2, the anti-interference optical defect detection model includes a multimodal feature encoding module, a modal attention fusion module, a compression adapter module, and a fast compression adapter module;

[0038] Step S4 includes the following steps:

[0039] S41: Acquire RGB images of the outer surface of the water purifier, and obtain phase map, polarization map and multispectral map of the outer surface of the water purifier through structured light module, polarization imaging system and multi-band spectral imaging device, and construct a multimodal view group together with RGB images;

[0040] S42: Input the multimodal view group into the shared convolutional encoder to extract deep-level defect features from the physical images of each viewpoint;

[0041] S43: Input the multimodal features output by the shared convolutional encoder into the modal attention fusion module, and fuse them according to the weight information of phase, polarization and spectral map to generate a unified multimodal semantic feature representation;

[0042] S44: Input the multimodal semantic feature representation into the compression adapter module for dimensionality compression and semantic alignment, while preserving the key discriminative information of the defective region;

[0043] S45: Input the output of step S44 into the fast compression adapter module and further optimize the inference path using a lightweight residual structure.

[0044] Preferably, step S4 includes:

[0045] A multi-task loss function for the anti-interference optical defect detection model is constructed by introducing modal reconstruction loss, modal consistency loss, and few-sample discrimination loss, and then minimized. The calculation formula for the multi-task loss function is as follows:

[0046] L total =λ1·L recon +λ2·L consis +λ3·L fewshot ;

[0047] Where λ1, λ2, and λ3 are hyperparameters used to balance the importance of each loss term;

[0048] L recon L represents the modal reconstruction loss. recon The original information distribution of the constraint modal input can be restored after processing by the compression adapter module, specifically in the form of:

[0049]

[0050] Among them, X m This represents the original input features of the m-th mode. This represents the reconstructed modal output, where M is the total number of modes;

[0051] L consis L represents the modal consistency loss. consis Used to enhance the cooperative distribution of modalities in the shared semantic space and suppress feature shifts caused by physical interference, L consis Satisfying the relational expression:

[0052]

[0053] Among them, F i F j These represent the intermediate semantic features output by different modal encoders;

[0054] L fewshot L represents the small sample discrimination loss. fewshot To enhance the discrimination ability of the compression adapter module in scenarios with scarce samples, it is trained using prototype loss or improved cross-entropy loss from metric learning.

[0055] Preferably, the multimodal perception network includes a multimodal input module, a dual-branch backbone network, a fusion module, a feature pyramid network, and a classification and regression head module;

[0056] Step S5 includes the following sub-steps:

[0057] S51: Acquire an RGB image of the water purifier's outer surface. The multimodal perception network simultaneously receives data input from multiple heterogeneous images, including RGB image features I. RGB Structured light 3D topography features I SL Polarization imaging features I pol Phase Reconstruction Map Features I Phase With multi-band spectral features I Spec The RGB image and the fused optical image are then input into the dual-branch backbone network, respectively.

[0058] The dual-branch backbone network consists of two ResNet-18 backbone networks sharing weights. This dual-branch backbone network is used to extract feature representations of different modalities, outputting multi-scale feature maps in each branch.

[0059] F RGB =ResNet(I RGB ),F Opt =ResNet(I Opt );

[0060] Each feature map F includes three output layers, namely: and

[0061] S52: The multi-scale feature maps output by the dual-branch backbone network are input into the fusion module, which includes a channel switching module and a spatial attention module, wherein:

[0062] The channel switching module receives feature maps from two branches, performs channel fusion processing, and outputs channel-enhanced feature maps:

[0063]

[0064] The enhanced feature map is input into the spatial attention module to generate a spatial attention map and reallocate spatial weights, resulting in the fused feature map:

[0065]

[0066] S53: Input the fused feature map into the feature pyramid network to construct multi-scale feature maps for semantic enhancement.

[0067]

[0068] in Outputting fused semantic features at different scales;

[0069] S54: Perform defect identification and localization on the multi-scale feature maps output by the feature pyramid network, and output the classification results. Cls Output of location regression results Reg :

[0070] Output Cls =ClsHead(F FPN Output Reg =RegHead(F FPN ).

[0071] Preferably, in step S5, during the training of the multimodal perception classification network, the following steps are also included:

[0072] A total loss function is constructed by introducing modal collaborative constraint loss, edge-aware consistency loss, and defect recognition joint loss. This total loss function is then minimized to optimize the network's multimodal feature representation ability, defect region edge perception ability, and classification and localization accuracy. The formula for calculating the total loss function is as follows:

[0073] L total =μ1·L comodal +μ2·L edge +μ3·L joint ;

[0074] Where μ1, μ2, and μ3 represent the weight hyperparameters indicating the importance of the loss term;

[0075] Modal cooperative constraint loss L comodal To improve the consistency of representation between RGB and optical images at higher-order feature layers, a mutual information maximization strategy is employed for optimization, defined as follows:

[0076] L comodal =-I(F rgb ,F opt );

[0077] Wherein, represents the mutual information metric between RGB image features and optical image features;

[0078] Edge-aware consistency loss L edge To enhance the network's ability to perceive the structure of defect edge regions, it is calculated jointly using structural similarity and gradient difference metrics, and is defined as follows:

[0079]

[0080] in, Represents the Sobel operator. and These are the edge regions of the predicted map and the labeled map, respectively;

[0081] The joint loss for defect identification includes a combination of Focal loss and smoothed L1 loss. This joint loss is used to simultaneously perform defect classification and location regression, and satisfies the following relationship:

[0082]

[0083] Preferably, step S6 further includes the following step:

[0084] The performance of the trained defect detection model is evaluated using detection precision, recall, mean precision, and defect localization error.

[0085] One of the above technical solutions has the following advantages or beneficial effects:

[0086] This invention constructs a multimodal defect dataset comprising both traditional and optical images, providing a complementary data foundation for the model. Traditional images capture macroscopic structure and texture information, while optical images reveal microscopic details and hidden defects, together overcoming the limitations of single RGB images in material representation and spatial perception. Secondly, a defect detection model is designed, comprising a high-precision defect classification model, an anti-interference optical defect detection model, and a multimodal perception network. A collaborative mechanism addresses the differentiated perception needs of various defect types. The high-precision classification model focuses on extracting contextual features from traditional images, the anti-interference model robustly optimizes against noise and interference in optical images, and the multimodal network serves as the fusion hub. Subsequently, traditional and optical images are input into their respective models for independent training, ensuring that each modality's features are fully optimized without interference, avoiding feature confusion. Next, the multimodal perception network fuses the two types of features, leveraging cross-modal complementarity to enhance the comprehensive perception capability of complex defects, overcoming the problem of incomplete single-modal information. The trained model is evaluated and optimized using a validation set, forming a stable and reliable defect detection model, ultimately achieving intelligent detection of defects on the outer surface of water purifiers. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0088] Figure 1 This is a flowchart of a deep learning-based method for detecting defects on the outer surface of a water purifier, provided in an embodiment of the present invention. Detailed Implementation

[0089] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0090] In this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0091] A deep learning-based method for detecting defects on the outer surface of water purifiers, such as... Figure 1 In a preferred embodiment of the present invention, the method for detecting defects on the outer surface of a water purifier includes the following steps:

[0092] S1: Construct a multimodal defect dataset for the outer surface of a water purifier, including both traditional and optical images;

[0093] Traditional images, acquired using industrial cameras, reflect the macroscopic structure and texture information of the water purifier's outer surface. Optical images, acquired using polarized or structured light, can reveal microscopic details and hidden defects. All image data, after unified annotation, enhancement, and standardization, form a multimodal dataset, providing a rich and high-quality sample foundation for subsequent model training. The multimodal defect dataset refers to a collection integrating various types of image data, used to comprehensively describe the diverse characteristics of defects on the water purifier's outer surface.

[0094] S2: Construct a defect detection model based on the multimodal defect dataset of the water purifier's outer surface from step S1. The defect detection model includes a high-precision defect classification model based on a hierarchical attention mechanism, an anti-interference optical defect detection model oriented towards few-shot learning, and a multimodal perception network.

[0095] It should be noted that the high-precision defect classification model is a deep learning-based model structure. Its core lies in using a hierarchical attention mechanism to extract and classify image features. By focusing on key regions in the image at different levels, it improves the accuracy of defect classification. The anti-interference optical defect detection model is mainly optimized for noise and interference in optical images. It employs a few-shot learning strategy, enabling effective training with a limited number of samples, thereby improving the model's detection performance under complex optical conditions. The multimodal perception network is responsible for fusing traditional image features with optical image features. Through a specific network structure design, it fully utilizes the complementary information of the two modalities, enhancing the model's overall perception capability of defects on the outer surface of the water purifier.

[0096] Understandably, the purpose of constructing a defect detection model containing three sub-models is to fully leverage the advantages of each model to solve different types of defect detection problems. The high-precision defect classification model utilizes a hierarchical attention mechanism to focus on key regions in traditional images, extracting rich semantic features to achieve high-precision classification of macroscopic defects. The anti-interference optical defect detection model combines few-shot learning and anti-interference training methods to effectively address the problems of optical images being greatly affected by noise in complex environments and the scarcity of samples, improving the detection capability for microscopic and hidden defects. The multimodal perception network, as a fusion module, integrates features from two modalities. Through cross-modal feature fusion, it overcomes the problem of incomplete information from a single modality, enhancing the model's comprehensive perception capability for multiple types and scales of defects. The effect of step S2 is the construction of a fully functional and powerful defect detection model, enabling it to adapt to the complex and diverse defect detection needs of the water purifier's outer surface.

[0097] For example, in a specific implementation case, the high-precision defect classification model uses ResNet-50 as the backbone network, adds a multi-resolution module, and performs weighted fusion of features at different scales. The anti-interference optical defect detection model is built on MobileNetV2, uses a prototype network to achieve few-shot learning, and combines data augmentation methods such as random noise addition and contrast enhancement for anti-interference training. The multimodal perception network adopts a dual-branch backbone structure, with each branch receiving traditional image features and optical image features respectively. Features are stitched together through a channel switching module, and a multi-scale feature map is constructed using a feature pyramid network. Finally, the defect category and location information are output through a classification and regression head module.

[0098] S3: Input the traditional images from the multimodal defect dataset on the outer surface of the water purifier into the high-precision defect classification model for training, and output the traditional image features;

[0099] In step S3, the high-precision defect classification model adopts a multi-level attention fusion structure, combining dilated convolution and channel attention mechanisms to more accurately capture and distinguish minute defects in traditional images, and outputs feature representations with semantic discrimination capabilities.

[0100] Specifically, the purpose of inputting traditional images into the high-precision defect classification model for training is to enable the model to learn defect-related features from traditional images, thereby achieving accurate classification of macroscopic defects on the outer surface of water purifiers. The high-precision defect classification model extracts features layer by layer from the image using a deep convolutional neural network, initially acquiring multi-level spatial semantic features of the image. Based on this, using a hierarchical attention mechanism, the model can automatically focus on key regions in the image, strengthening the learning and representation of defect features. In this way, the model can more accurately capture macroscopic structural defect information in traditional images, providing strong support for subsequent defect classification. This enables the high-precision defect classification model to extract effective features from traditional images, laying the foundation for subsequent defect classification tasks.

[0101] S4: Input the optical image training set from the multimodal defect dataset on the outer surface of the water purifier into the anti-interference optical defect detection model for training, and output the optical image features;

[0102] Specifically, the optical image training set is input into the anti-interference optical defect detection model for training to obtain optical feature output. In this embodiment, the anti-interference model introduces data augmentation strategies such as image contrast enhancement and color perturbation, and adopts a few-shot classification mechanism based on a prototype network to improve the model's learning ability and robustness in environments with insufficient samples and interference.

[0103] S5: Input traditional image features and optical image features into a multimodal perception network to obtain a trained defect detection model;

[0104] Specifically, traditional image feature output and optical feature output are input into a multimodal perception network to obtain a trained defect detection model. In this embodiment, the multimodal perception network integrates a cross-modal attention mechanism and a residual feature alignment structure, and achieves synergistic enhancement of the two types of image features through a dynamic weighting mechanism, thereby improving the overall recognition accuracy of the model for complex defects on the outer surface of the water purifier.

[0105] S6: Use the validation set to evaluate the trained defect detection model and obtain the validated defect detection model;

[0106] Specifically, the trained model is evaluated using a validation set to obtain a validated detection model. In this embodiment, the evaluation metrics include accuracy, recall, F1 score, and mAP (mean Average Precision). Performance testing on the validation set ensures that the model has good generalization ability and stability, and the model parameters can be further fine-tuned and optimized based on the evaluation results.

[0107] S7: Use the verified defect detection model to detect defects on the outer surface of the water purifier.

[0108] Specifically, when the predictive performance of the water purifier outer surface defect detection model is assessed to be stable, the model can be used to predict defects on the water purifier outer surface, and the defect detection results are relatively accurate.

[0109] Preferably, step S1 includes the following steps:

[0110] S11: Acquire RGB images of the outer surface of the water purifier from frontal, upward, and oblique angles to obtain traditional images. Perform grayscale normalization on the traditional images and enhance the response of small defect areas in the traditional images through brightness equalization and edge enhancement algorithms to obtain traditional enhanced images.

[0111] By acquiring RGB images of the water purifier's outer surface from multiple angles (front view, upward view, and oblique side view), we can comprehensively obtain the macroscopic structure and texture information of the water purifier's outer surface from different perspectives. This ensures that the acquired traditional images can cover the features of various areas of the water purifier's outer surface, providing comprehensive visual information for subsequent defect detection. Gray-level normalization processing unifies the gray-level values ​​of the image to a specific range, eliminating gray-level differences between different images caused by factors such as lighting conditions and camera response, improving image consistency and comparability. Brightness equalization aims to adjust the brightness distribution of the image, enhance image contrast, and make details in the image more clearly visible, helping to highlight small defect areas. Edge enhancement algorithms further highlight the boundary difference between small defects and surrounding normal areas by enhancing the contour information of object edges in the image, thereby enhancing the response of small defect areas in traditional images, improving the subsequent model's ability to identify defects, and obtaining traditional enhanced images that are more conducive to defect detection.

[0112] RGB images are color images composed of red, green, and blue color channels. They can intuitively reflect the appearance, color, and texture information of an object. In the detection of defects on the outer surface of water purifiers, RGB images provide rich visual cues, helping to identify defects such as color changes, scratches, and dents. Gray-level normalization is the process of linearly or non-linearly mapping the gray values ​​of an image to a specified interval (such as [0,1] or [0,255]). Its function is to eliminate gray-level differences between images, allowing different images to be compared and processed on the same gray-level scale, thus improving the stability and reliability of image processing algorithms. Brightness equalization is a technique for adjusting the brightness distribution of an image, usually achieved through methods such as histogram equalization. It can enhance the contrast of an image, making the details in dark and bright areas clearer, thus making it easier to find defects hidden in shadows or overly bright areas. Edge enhancement algorithms are a class of algorithms used to highlight edge information in images, such as the Sobel operator and the Laplacian operator. Edges in an image usually correspond to the outline of an object or the boundary between different regions. In defect detection, many defects (such as cracks and scratches) will be manifested on edge features. Therefore, edge enhancement can more clearly display the outline of these defects and improve the detectability of defects.

[0113] S12: Perform standardized cropping and data augmentation processing, including rotation, scaling and color perturbation, on the traditional enhanced image to obtain standard training image samples, and perform expert annotation of defect regions on the standard training image samples to construct a traditional image defect sample set.

[0114] Standardized cropping and data augmentation of traditional enhanced images aim to further improve image quality and diversity, enhancing the model's generalization ability to defects of different scales and postures. Geometric transformations such as rotation, scaling, and translation can simulate various posture changes that water purifiers may undergo during actual production and use, enabling the model to adapt to defect detection needs from different perspectives. Color perturbation increases the color diversity of images by changing color parameters such as hue, saturation, and brightness, improving the model's robustness to color changes and avoiding performance degradation caused by lighting conditions or changes in object surface color. Data augmentation effectively expands the size of the training dataset, improves the model's generalization ability, and prevents overfitting. Expert annotation of defect regions on processed image samples provides accurate supervision signals for model training, enabling the model to learn the feature differences between defective and non-defective regions, thereby constructing a high-quality traditional image defect sample set and laying the foundation for subsequent training of traditional image detection models.

[0115] Standardized cropping refers to cropping images according to certain size and proportion requirements, ensuring all images have a uniform size and dimensions. This helps improve the efficiency of image processing and model training, and also facilitates standardized feature extraction and analysis by the model. Data augmentation is a method that expands the size and diversity of the training dataset through various image transformation techniques. Its role is to increase the model's adaptability and robustness to different image variations, improve the model's generalization ability, and reduce overfitting. Expert annotation refers to the precise annotation of defective regions in images by personnel with professional knowledge and experience. This typically uses bounding boxes, masks, etc., to identify the location and category of defects in the image. Expert annotation provides accurate supervision information for model training, enabling the model to learn the feature differences between defective and non-defective regions, thereby improving the model's detection accuracy and reliability.

[0116] S13: Acquire structured light interference images of the outer surface of the water purifier to obtain optical images, perform geometric distortion correction and image alignment on the optical images, and remove specular reflection interference through reflectivity normalization and bright spot suppression algorithms to obtain enhanced optical images;

[0117] Specifically, acquiring structured light interferometry images can obtain microscopic three-dimensional morphological information of the outer surface of a water purifier. This information is crucial for detecting microscopic defects (such as tiny pits, protrusions, and cracks) that are difficult to detect using traditional RGB images. Structured light interferometry imaging technology projects light with a specific interference fringe pattern onto the object's surface, then a camera collects the reflected light, and calculates the three-dimensional height information of the object's surface based on the distortion of the interference fringes, thus obtaining an optical image. However, the acquired optical images often suffer from geometric distortion and image misalignment, which can affect subsequent feature extraction and defect analysis. Therefore, geometric distortion correction and image alignment operations are necessary to ensure the geometric accuracy of the image and enable it to accurately reflect the three-dimensional morphology of the object's surface. Reflectivity normalization and bright spot suppression algorithms are used to remove specular reflection interference, which may obscure the true defect information of the object's surface, leading to misjudgment or missed detection. Reflectance normalization unifies the reflectance of an image to a specific range, reducing the impact of reflectance differences on defect detection; the bright spot suppression algorithm can effectively suppress bright spots in the image, highlighting the true texture and defect features of the object surface, thereby obtaining an enhanced optical image.

[0118] S14: Perform illumination equalization and high-frequency detail restoration processing on the enhanced optical image, and enhance the surface weak defect texture features through a local contrast enhancement algorithm to obtain the final optical image. Subsequently, construct an optical image defect sample set through expert annotation.

[0119] Illumination equalization and high-frequency detail restoration are performed on enhanced optical images to further optimize lighting conditions and detail features, making defect information in the image more prominent and clear. Illumination equalization enhances image contrast by adjusting the illumination distribution, making bright and dark areas more distinct and helping to discover defects hidden in shadows or overly bright areas. High-frequency detail restoration focuses on restoring high-frequency components in the image, such as edges and texture details, which is particularly important for detecting minute defects, as these defects often correspond to high-frequency variations in the image. Local contrast enhancement algorithms further enhance the texture features of weak surface defects by increasing the contrast of local areas in the image, making these weak features stand out from the surrounding background and improving the model's ability to detect weak defects. The result is a high-quality optical image, and an effective optical image defect sample set is constructed through expert annotation, providing strong support for training the optical image detection model.

[0120] S15: Construct training data for the traditional image defect sample set and the optical image defect sample set respectively, and divide them into training sets and validation sets for training the traditional image detection model and the optical image detection model according to the ratio;

[0121] The outputs of the traditional image detection model and the optical image detection model are used as inputs to the multimodal perception network.

[0122] Training data is constructed from both traditional and optical image defect sample sets, with each set divided into training and validation sets according to a specific ratio (e.g., a common 7:3 or 8:2 ratio). The training set is used to train both the traditional and optical image detection models, enabling them to learn feature representations and defect patterns in their respective modalities. The validation set is used to evaluate and monitor model performance during training, promptly identifying potential overfitting or underfitting issues, and adjusting hyperparameters and training strategies to ensure good generalization ability and stability. By rationally dividing the training and validation sets, limited sample data can be fully utilized, allowing for effective model optimization during training. This provides a reliable model foundation for subsequent multimodal perception network fusion, ensuring the performance and reliability of the entire defect detection system.

[0123] Preferably, step S3 includes the following sub-steps:

[0124] S31: Use a deep convolutional backbone network to perform preliminary feature extraction on traditional images to obtain multi-level spatial semantic features;

[0125] A deep convolutional backbone network is a deep neural network structure composed of multiple convolutional layers, pooling layers, etc., commonly including VGG and ResNet. It extracts image features through convolutional operations, while pooling operations are used to reduce feature dimensionality, decrease computational cost, and enhance the translation invariance of features. In step S31, the deep convolutional backbone network performs preliminary feature extraction on the traditional image to be processed, obtaining multi-level spatial semantic features that can characterize the image content. These features will serve as input to subsequent feature processing modules, providing basic data support for subsequent multi-scale feature extraction, attention mechanism processing, and other operations.

[0126] S32: Input the backbone features of the traditional image into the multi-resolution module, and use convolution kernels of different scales to extract the features at multiple scales. Use 3×3, 5×5 and 7×7 convolution kernels to extract different texture and edge features;

[0127] The Multi-Resolution Module (MRM) is a network structure used to extract features at different scales from feature maps. It is implemented by using convolutional kernels of different sizes. A convolutional kernel is a basic operational unit in a convolutional neural network, used to slide across the feature map and perform convolution operations to extract features at a specific scale. In step S32, the MRM performs multi-scale feature extraction on the backbone features of a traditional image, using convolutional kernels of different sizes (3×3, 5×5, and 7×7) to extract texture and edge features under different receptive fields. This enriches the scale information of the features, improves the model's ability to perceive defects of different sizes, and provides more comprehensive feature input for subsequent feature fusion and defect classification.

[0128] S33: In the resolution module, residual connections and attention weight adaptive mechanisms are introduced to fuse and enhance the multi-scale features of step S32;

[0129] Residual connections are a network architecture design that adds skip connections between convolutional layers, directly adding input features to output features. This alleviates the vanishing gradient problem in deep network training and facilitates the flow of information and gradients within deep networks. Adaptive attention weights are a method that dynamically adjusts feature weights based on feature importance, enabling the model to automatically focus on important feature regions and ignore less important information. In step S33, residual connections ensure that multi-scale features effectively transfer information during fusion, avoiding gradient vanishing; the adaptive attention weights mechanism weights multi-scale features, highlighting features related to defects, enhancing feature discriminativity, and improving feature fusion and enhancement effects. Different residual connection methods, such as full residual connections or partial residual connections, can be used, selected based on the network's depth and complexity.

[0130] S34: Input the output of the resolution module to the global attention aggregation module, model the relationship between any pixels based on the non-local attention mechanism, and calculate the attention map to further weight the global semantics;

[0131] The Global Attention Aggregation Module (GAAM) is a component that uses a non-local attention mechanism to perform global semantic modeling of features. The non-local attention mechanism captures global dependency information by calculating the relationships between different locations in the feature map, generating an attention weight matrix. In step S34, the GAAM module performs global semantic modeling on the features output by the multi-resolution module. It generates an attention map by calculating the relationships between arbitrary pixels, further weighting the global semantic features to highlight global structural information related to defects. This allows the model to better focus on macroscopic structural features in traditional images, improving its ability to identify macroscopic structural defects and enhancing its understanding and utilization of overall image information.

[0132] Attention maps can reflect the importance of global semantic information. By reweighting global semantic features, the model can pay more attention to global structural information related to defects, highlight macroscopic structural features in the image, thereby enhancing the ability to identify macroscopic structural defects and helping the model to understand the image content as a whole and grasp the relationship between defects and the overall structure.

[0133] S35: Simultaneously input the output of the resolution module into the local attention aggregation module, and use a local window partitioning strategy and spatial coordinate guidance mechanism to extract the discrimination features of local micro-defect regions;

[0134] The Local Attention Aggregation Module (LAAM) is a component focused on local feature extraction. It divides the feature map into multiple small windows using a local windowing strategy and aggregates features within each window. A spatial coordinate guidance mechanism utilizes the spatial location information of each pixel in the feature map to guide the model to focus on feature relationships within local regions, enhancing the expressive power of local features. In step S35, the LAAM module performs local feature extraction on the features output by the multi-resolution module. Through local windowing and spatial coordinate guidance, it focuses on local, subtle defect areas, extracting their discriminative features. This enables the model to accurately locate and identify local, minute defects (such as small scratches and dents) in traditional images, improving its perception of local details and providing more refined feature input for subsequent defect classification.

[0135] S36: The features output by the global attention aggregation module and the local attention aggregation module are fused together, and combined with the global and local attention features of traditional images, the classifier is introduced to output the final defect category result.

[0136] A classifier is a machine learning model used to map features to category labels. Common examples in deep learning include fully connected layers and support vector machines. In step S36, the classifier receives the fused global and local attention features, learns the mapping relationship between these features and defect categories, classifies defects in traditional images, and outputs the final defect category result. This completes the classification step in the defect detection task, providing users with specific defect type information.

[0137] Specifically, the global features output by the global attention aggregation module in step S34 and the local features output by the local attention aggregation module in step S35 are fused together. This combines the global and local attention features of traditional images to comprehensively represent the defect information in the image. The fused features are then input into a classifier. The classifier learns the mapping relationship between features and defect categories to classify defects in the image and outputs the final defect category result. This fusion method fully utilizes the overall structural information provided by global features and the detailed information provided by local features, making the model's defect identification more accurate and comprehensive, and improving the accuracy and reliability of defect classification.

[0138] Preferably, step S3 further includes the following step: constructing the loss function of the traditional image detection model by introducing multi-branch feature alignment loss, attention consistency loss, and cross-entropy classification loss, and minimizing the loss function, wherein the calculation formula of the loss function of the traditional image detection model is as follows:

[0139] L total =α·L align +β·L attn +γ·L cls ;

[0140] Where α, β, and γ are hyperparameters used to adjust the importance of each part of the loss;

[0141] L align The multi-branch feature alignment loss is used to constrain the consistency of the output features of the global attention aggregation module and the local attention aggregation module in the semantic space. Its calculation formula is as follows:

[0142]

[0143] in, and Let N represent the feature vectors output by the global attention aggregation module and the local attention aggregation module for the i-th sample, respectively, where N is the total number of samples.

[0144] L attn L represents the loss of attentional consistency. attn L is used to maintain the consistency of spatial attention distribution of the main features after processing by the multi-resolution module, the global attention aggregation module, and the local attention aggregation module. cls L represents the cross-entropy loss. cls The prediction accuracy used to supervise the final defect category satisfies the following relationship:

[0145]

[0146] Where C is the total number of defect categories, y i For real category labels, To predict probabilities.

[0147] Specifically, the multi-branch feature alignment loss aims to constrain the semantic space consistency of the output features of the global attention aggregation module and the local attention aggregation module, ensuring effective fusion of global and local features and avoiding conflicts and mismatches between features. The attention consistency loss maintains the consistency of the spatial attention distribution of the backbone features after processing by the multi-resolution module, the global attention aggregation module, and the local attention aggregation module, preventing the model's attention to key regions from shifting during feature extraction and processing, and ensuring the model continuously focuses on defect-related regions. The cross-entropy classification loss supervises the prediction accuracy of the final defect category. By minimizing the difference between the predicted probability and the true label, the model can accurately learn the feature representations of different defect categories, thereby improving the model's classification accuracy. During training, the importance of each loss component is balanced by adjusting the hyperparameters α, β, and γ, comprehensively considering feature alignment, attention consistency, and classification accuracy to achieve comprehensive model optimization, ultimately enabling the model to accurately identify and classify various defects in traditional images.

[0148] Preferably, in step S2, the anti-interference optical defect detection model includes a multimodal feature encoding module, a modal attention fusion module, a compression adapter module, and a fast compression adapter module;

[0149] Step S4 includes the following steps:

[0150] S41: Acquire RGB images of the outer surface of the water purifier, and obtain phase map, polarization map and multispectral map of the outer surface of the water purifier through structured light module, polarization imaging system and multi-band spectral imaging device, and construct a multimodal view group together with RGB images;

[0151] Specifically, RGB images of the water purifier's outer surface are acquired, while phase maps, polarization maps, and multispectral maps are acquired using a structured light module, a polarization imaging system, and a multi-band spectral imaging device, respectively. These images of different modalities reflect information about the water purifier's outer surface from multiple angles and physical characteristics. RGB images provide intuitive information about appearance, color, and texture; phase maps reflect the three-dimensional morphological features of the object's surface; polarization maps display the polarization characteristics of the object's surface, helping to distinguish different materials and detect defects under specular reflection; and multispectral maps provide the object's spectral response characteristics in different bands, aiding in the identification of defects based on material differences. Combining these images into a multimodal view set provides a more comprehensive and richer information foundation for subsequent feature extraction and fusion, enabling the model to comprehensively utilize features from multiple modalities for defect detection, improving detection accuracy and robustness.

[0152] Structured light modules are an active illumination technology that projects light in a specific pattern onto the surface of an object, and a camera collects the reflected light. Based on the distortion of the light, the three-dimensional shape information of the object is calculated. The phase map records the distortion information of the interference fringes on the object's surface, reflecting the object's microscopic three-dimensional shape features and can be used to detect minute shape changes and surface defects. Polarization imaging systems capture the polarization characteristics of reflected light from an object's surface. The polarization map displays the polarization information of the object's surface, offering unique advantages in distinguishing surface materials, detecting defects under specular reflection, and removing irrelevant reflection interference. Multi-band spectral imaging equipment can acquire spectral images of an object in multiple different bands. The multispectral map records the spectral response characteristics of the object at different wavelengths. Different materials and defects have different spectral responses at different bands, thus providing rich spectral information and helping to identify defects based on material differences. A multimodal view group is a data structure that combines RGB images, phase maps, polarization maps, and multispectral maps. It comprehensively reflects various physical properties of the object, providing a comprehensive information foundation for subsequent feature extraction and fusion. This allows the model to learn the object's features from multiple dimensions, improving the accuracy and robustness of defect detection.

[0153] S42: Input the multimodal view group into the shared convolutional encoder to extract deep-level defect features from the physical images of each viewpoint;

[0154] The shared convolutional encoder is a network structure for feature extraction from multimodal images. It consists of multiple convolutional layers, pooling layers, and activation functions, with images from different modalities sharing the same set of convolutional layer parameters. In step S42, the shared convolutional encoder performs unified feature extraction on the RGB images, phase maps, polarization maps, and multispectral maps in the multimodal view group. By sharing convolutional layer parameters, it ensures fairness and consistency in the feature extraction process for different modalities, enabling the extracted features to be effectively fused and compared in the same feature space, providing a basic feature representation for subsequent feature processing and defect detection.

[0155] S43: Input the multimodal features output by the shared convolutional encoder into the modal attention fusion module, and fuse them according to the weight information of phase, polarization and spectral map to generate a unified multimodal semantic feature representation;

[0156] Specifically, the multimodal features output from the shared convolutional encoder are input into the modal attention fusion module. This module fuses the features based on the weights of the phase map, polarization map, multispectral map, and RGB image. By calculating the importance weights of each modal feature and performing weighted fusion operations, the module highlights the modal features most contributing to defect detection and suppresses redundant or interfering information. The weights can be determined based on prior knowledge (such as the importance of different modal images for defect detection) or automatically learned through a learning algorithm. This approach generates a unified multimodal semantic feature representation, which integrates the advantages of multimodal images, enhancing the discriminative power of the features. This enables the model to more accurately identify and locate defects on the outer surface of the water purifier, improving the performance and reliability of defect detection.

[0157] S44: Input the multimodal semantic feature representation into the compression adapter module for dimensionality compression and semantic alignment, while preserving the key discriminative information of the defective region;

[0158] Specifically, the multimodal semantic feature representation generated in step S43 is input into the compression adapter module, which performs dimensionality compression and semantic alignment on the features. Dimensionality compression reduces the dimension of the feature vectors, removes redundant information, and lowers computational complexity, while retaining key discriminative information of the defect region. Semantic alignment ensures that features from different modalities are effectively fused and compared in the same semantic space, improving feature compatibility and consistency. In this way, a more compact and efficient feature representation can be obtained, providing optimized feature input for subsequent fast inference and defect detection, thereby improving the model's running efficiency and detection performance.

[0159] It's worth noting that the compression adapter module is a component used for dimensionality compression and semantic alignment of features. Dimensionality compression is the process of reducing the dimension of feature vectors and removing redundant information through dimensionality reduction algorithms (such as PCA, LDA, or 1×1 convolution in convolution operations). Semantic alignment is the process of effectively fusing and comparing features from different modalities within the same semantic space through feature transformations (such as linear transformations, normalization, etc.).

[0160] S45: Input the output of step S44 into the fast compression adapter module and further optimize the inference path using a lightweight residual structure.

[0161] Specifically, the output of step S44 is input into the fast compression adapter module, which further optimizes the features using a lightweight residual structure. By introducing residual connections, the lightweight residual structure accelerates feature propagation and processing without increasing computational burden, alleviating the gradient vanishing problem in deep network training and improving the model's convergence speed and stability. Simultaneously, the fast compression adapter module optimizes the inference path, reducing unnecessary computational operations and lowering the model's inference latency, ensuring that the model can quickly and efficiently complete defect detection tasks in practical applications, meeting the requirements of real-time detection.

[0162] Preferably, step S4 includes:

[0163] A multi-task loss function for the anti-interference optical defect detection model is constructed by introducing modal reconstruction loss, modal consistency loss, and few-sample discrimination loss, and then minimized. The calculation formula for the multi-task loss function is as follows:

[0164] L total =λ1·L tecon +λ2·L consis +λ3·L fewshot ;

[0165] Where λ1, λ2, and λ3 are hyperparameters used to balance the importance of each loss term;

[0166] L recon L represents the modal reconstruction loss. recon The original information distribution of the constraint modal input can be restored after processing by the compression adapter module, specifically in the form of:

[0167]

[0168] Among them, X m Represents the original input features of the m-th modality. This represents the reconstructed modal output, where M is the total number of modes;

[0169] Lconsis L represents the modal consistency loss. consis Used to enhance the cooperative distribution of modalities in the shared semantic space and suppress feature shifts caused by physical interference, L consis Satisfying the relational expression:

[0170]

[0171] Among them, F i F j These represent the intermediate semantic features output by different modal encoders;

[0172] L fewshot L represents the small sample discrimination loss. fewshot To enhance the discrimination ability of the compression adapter module in scenarios with scarce samples, it is trained using prototype loss or improved cross-entropy loss from metric learning.

[0173] Specifically, by introducing modal reconstruction loss (L recon Modal consistency loss (L) consis ) and small sample discriminant loss (L fewshot Constructing a multi-task loss function (L) for an anti-interference optical defect detection model total The model training and optimization process is guided by a loss function. Modality reconstruction loss aims to constrain the original information distribution of modal inputs after processing by the compression adapter module, ensuring that excessive original modal information is not lost during feature compression, thus maintaining feature integrity and reconstructability. Modality consistency loss strengthens the cooperative distribution of different modalities in a shared semantic space, suppressing feature shifts caused by physical interference (such as illumination changes and reflections), enabling better alignment and fusion of features from different modalities in the same semantic space, and improving feature stability and consistency. Few-shot discrimination loss enhances the discrimination capability of the compression adapter module in scenarios with scarce samples. It is trained using prototype loss in metric learning or improved cross-entropy loss, allowing the model to learn effective feature representations using limited labeled samples and quickly adapt to new defect types or detection tasks. During training, the importance of each loss component is balanced by adjusting hyperparameters λ1, λ2, and λ3, comprehensively considering feature reconstruction, consistency preservation, and few-shot discrimination to achieve comprehensive model optimization, ultimately enabling the model to accurately identify and classify various defects in optical images.

[0174] Preferably, the multimodal perception network includes a multimodal input module, a dual-branch backbone network, a fusion module, a feature pyramid network, and a classification and regression head module;

[0175] Step S5 includes the following sub-steps:

[0176] S51: Acquire an RGB image of the water purifier's outer surface. The multimodal perception network simultaneously receives data input from multiple heterogeneous images, including RGB image features I. RGB Structured light 3D topography features I SL Polarization imaging features I pol Phase Reconstruction Map Features I Phase With multi-band spectral features I Spec The RGB image and the fused optical image are then input into the dual-branch backbone network, respectively.

[0177] The dual-branch backbone network consists of two ResNet-18 backbone networks sharing weights. This dual-branch backbone network is used to extract feature representations of different modalities, outputting multi-scale feature maps in each branch.

[0178] F RGB =ResNet(I RGB ),F Opt =ResNet(I Opt );

[0179] Each feature map F includes three output layers, namely: and

[0180] S52: The multi-scale feature maps output by the dual-branch backbone network are input into the fusion module, which includes a channel switching module and a spatial attention module, wherein:

[0181] The channel switching module receives feature maps from two branches, performs channel fusion processing, and outputs channel-enhanced feature maps:

[0182]

[0183] The enhanced feature map is input into the spatial attention module to generate a spatial attention map and reallocate spatial weights, resulting in the fused feature map:

[0184]

[0185] S53: Input the fused feature map into the feature pyramid network to construct multi-scale feature maps for semantic enhancement.

[0186]

[0187] in Outputting fused semantic features at different scales;

[0188] S54: Perform defect identification and localization on the multi-scale feature maps output by the feature pyramid network, and output the classification results.Cls Output of location regression results Reg :

[0189] Output Cls =ClsHead(F FPN Output Reg =RegHead(F FPN ).

[0190] Specifically, the multimodal perception network aims to fuse image features from different modalities to achieve comprehensive and accurate detection of defects on the outer surface of water purifiers. The multimodal perception network includes a multimodal input module, a dual-branch backbone network, a fusion module, a feature pyramid network, and a classification and regression head module. The multimodal input module receives data input from multi-source heterogeneous images, such as RGB image features and structured light 3D topography features. These features reflect information about the outer surface of the water purifier from different angles and physical properties. The dual-branch backbone network consists of two ResNet-18 backbone networks with shared weights, extracting features from the RGB image and the fused optical image respectively, outputting multi-scale feature maps. The channel switching module and spatial attention module in the fusion module sequentially perform channel fusion and spatial weight redistribution on the feature maps from the two branches, enhancing the expressive power of the features. The feature pyramid network constructs multi-scale feature maps and performs semantic enhancement, enabling the model to focus on defect features at different scales. Finally, the classification and regression head module performs defect identification and localization based on the fused feature maps, outputting classification results and location regression results, achieving intelligent detection of defects on the outer surface of the water purifier.

[0191] Preferably, in step S5, during the training of the multimodal perception classification network, the following steps are also included:

[0192] A total loss function is constructed by introducing modal collaborative constraint loss, edge-aware consistency loss, and defect recognition joint loss. This total loss function is then minimized to optimize the network's multimodal feature representation ability, defect region edge perception ability, and classification and localization accuracy. The formula for calculating the total loss function is as follows:

[0193] L total =μ1·L comodal +μ2·L edge +μ3·L joint ;

[0194] Where μ1, μ2, and μ3 represent the weight hyperparameters indicating the importance of the loss term;

[0195] Modal cooperative constraint loss L comodal To improve the consistency of representation between RGB and optical images at higher-order feature layers, a mutual information maximization strategy is employed for optimization, defined as follows:

[0196] L comodal =-I(F rgb ,F opt );

[0197] Wherein, represents the mutual information metric between RGB image features and optical image features;

[0198] Edge-aware consistency loss L edge To enhance the network's ability to perceive the structure of defect edge regions, it is calculated jointly using structural similarity and gradient difference metrics, and is defined as follows:

[0199]

[0200] in, Represents the Sobel operator. and These are the edge regions of the predicted map and the labeled map, respectively;

[0201] The joint loss for defect identification includes a combination of Focal loss and smoothed L1 loss. This joint loss is used to simultaneously perform defect classification and location regression, and satisfies the following relationship:

[0202]

[0203] Specifically, in the process of training the multimodal perception classification network, a modal cooperative constraint loss L is introduced. comodal Edge-aware consistency loss L edge Joint loss L with defect identification joint Construct the total loss function L totalThe modal collaborative constraint loss aims to improve the consistency of representation between RGB and optical images at higher-order feature layers. By maximizing mutual information, it enables the features of the two modalities to better fuse and complement each other at the higher-order semantic level, enhancing the complementarity and synergistic effect of features. The edge-aware consistency loss is used to improve the network's ability to perceive the structure of defect edge regions. Combining structural similarity and gradient difference measures, it ensures that the model can accurately capture the detailed features of defect edges and improve the accuracy of defect localization. The defect recognition joint loss combines Focal loss and smoothed L1 loss to perform defect classification and location regression simultaneously, enabling the model to achieve good performance in both classification and localization tasks. Focal loss can effectively solve the class imbalance problem, making the model pay more attention to samples that are difficult to classify; smoothed L1 loss has good robustness to regression tasks and can accurately measure the difference between predicted bounding boxes and true bounding boxes. By reasonably setting hyperparameters μ1, μ2, and μ3 to balance the importance of each part of the loss, and comprehensively considering multiple aspects such as feature consistency, edge perception ability, and classification and localization accuracy, the model is fully optimized, ultimately enabling the model to accurately identify and locate defects on the outer surface of the water purifier.

[0204] Preferably, step S6 further includes the following step:

[0205] The performance of the trained defect detection model is evaluated using detection precision, recall, mean precision, and defect localization error.

[0206] Specifically, to comprehensively and objectively evaluate the performance of the trained defect detection model, four key metrics are used: detection precision, recall, mean precision, and defect localization error. Detection precision measures the proportion of correctly identified defect samples out of all predicted defect samples, reflecting the accuracy of the model's predictions. Recall measures the proportion of defect samples that the model can detect out of all actual defect samples, reflecting the completeness of the model's detection. Mean precision comprehensively considers the model's detection precision and recall across different defect categories, reflecting the overall performance balance of the model. Defect localization error measures the deviation between the predicted defect location and the actual defect location, evaluating the model's localization accuracy. These four metrics comprehensively quantify the model's detection performance from different perspectives, ensuring that the model can reliably identify and locate defects on the outer surface of water purifiers in practical applications.

[0207] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0208] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting defects on the outer surface of a water purifier based on deep learning, characterized in that, The method for detecting defects on the outer surface of a water purifier includes the following steps: S1: Construct a multimodal defect dataset for the outer surface of a water purifier, including both traditional and optical images; S2: Based on the multimodal defect dataset of the water purifier's outer surface from step S1, construct a defect detection model. The defect detection model includes a high-precision defect classification model based on a hierarchical attention mechanism, an anti-interference optical defect detection model oriented towards few-shot learning, and a multimodal perception network. S3: Input traditional images from the multimodal defect dataset of the water purifier's outer surface into the high-precision defect classification model for training, and output traditional image features; S4: Input the optical image training set from the multimodal defect dataset on the outer surface of the water purifier into the anti-interference optical defect detection model for training, and output the optical image features; S5: Input traditional image features and optical image features into a multimodal perception network to obtain a trained defect detection model; S6: Use the validation set to evaluate the trained defect detection model and obtain the validated defect detection model; S7: Use the validated defect detection model to detect defects on the outer surface of the water purifier.

2. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 1, characterized in that, Step S1 includes the following steps: S11: Acquire RGB images of the outer surface of the water purifier from frontal, upward, and oblique angles to obtain traditional images. Perform grayscale normalization on the traditional images and enhance the response of small defect areas in the traditional images through brightness equalization and edge enhancement algorithms to obtain traditional enhanced images. S12: Perform standardized cropping and data augmentation processing, including rotation, scaling and color perturbation, on traditional enhanced images to obtain standard training image samples, and perform expert annotation of defect regions on the standard training image samples to construct a traditional image defect sample set. S13: Acquire structured light interference images of the outer surface of the water purifier to obtain optical images, perform geometric distortion correction and image alignment on the optical images, and remove specular reflection interference through reflectivity normalization and bright spot suppression algorithms to obtain enhanced optical images; S14: Perform illumination equalization and high-frequency detail restoration processing on the enhanced optical image, and enhance the surface weak defect texture features through a local contrast enhancement algorithm to obtain the final optical image. Subsequently, construct an optical image defect sample set through expert annotation. S15: Construct training data for the traditional image defect sample set and the optical image defect sample set respectively, and divide them into training sets and validation sets for training the traditional image detection model and the optical image detection model according to the ratio; The outputs of the traditional image detection model and the optical image detection model are used as inputs to the multimodal perception network.

3. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 2, characterized in that, Step S3 includes the following sub-steps: S31: Use a deep convolutional backbone network to perform preliminary feature extraction on traditional images to obtain multi-level spatial semantic features; S32: Input the backbone features of the traditional image into the multi-resolution module, and use convolution kernels of different scales to extract the features at multiple scales. Use 3×3, 5×5 and 7×7 convolution kernels to extract different texture and edge features; S33: In the resolution module, residual connections and attention weight adaptive mechanisms are introduced to fuse and enhance the multi-scale features of step S32; S34: Input the output of the resolution module to the global attention aggregation module, model the relationship between any pixels based on the non-local attention mechanism, and calculate the attention map to further weight the global semantics; S35: Simultaneously input the output of the resolution module into the local attention aggregation module, and use a local window partitioning strategy and spatial coordinate guidance mechanism to extract the discrimination features of local micro-defect regions; S36: The features output by the global attention aggregation module and the local attention aggregation module are fused together, and combined with the global and local attention features of traditional images, the classifier is introduced to output the final defect category result.

4. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 3, characterized in that, Step S3 further includes the following steps: constructing the loss function of the traditional image detection model by introducing multi-branch feature alignment loss, attention consistency loss, and cross-entropy classification loss, and minimizing this loss function. The calculation formula of the loss function of the traditional image detection model is as follows: L total =α·L align +β·L attn +γ·L cls ; Where α, β, and γ are hyperparameters used to adjust the importance of each part of the loss; L align The multi-branch feature alignment loss is used to constrain the consistency of the output features of the global attention aggregation module and the local attention aggregation module in the semantic space. Its calculation formula is as follows: in, and Let N represent the feature vectors output by the global attention aggregation module and the local attention aggregation module for the i-th sample, respectively, where N is the total number of samples. L attn L represents the loss of attentional consistency. attn L is used to maintain the consistency of spatial attention distribution of the main features after processing by the multi-resolution module, the global attention aggregation module, and the local attention aggregation module. cls L represents the cross-entropy loss. cls The prediction accuracy used to supervise the final defect category satisfies the following relationship: Where C is the total number of defect categories, y i For real category labels, To predict probabilities.

5. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 1, characterized in that, In step S2, the anti-interference optical defect detection model includes a multimodal feature encoding module, a modal attention fusion module, a compression adapter module, and a fast compression adapter module; Step S4 includes the following steps: S41: Acquire RGB images of the outer surface of the water purifier, and obtain phase map, polarization map and multispectral map of the outer surface of the water purifier through structured light module, polarization imaging system and multi-band spectral imaging device, and construct a multimodal view group together with RGB images; S42: Input the multimodal view group into the shared convolutional encoder to extract deep-level defect features from the physical images of each viewpoint; S43: Input the multimodal features output by the shared convolutional encoder into the modal attention fusion module, and fuse them according to the weight information of phase, polarization and spectral map to generate a unified multimodal semantic feature representation; S44: Input the multimodal semantic feature representation into the compression adapter module for dimensionality compression and semantic alignment, while preserving the key discriminative information of the defective region; S45: Input the output of step S44 into the fast compression adapter module and further optimize the inference path using a lightweight residual structure.

6. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 5, characterized in that, Step S4 includes: A multi-task loss function for the anti-interference optical defect detection model is constructed by introducing modal reconstruction loss, modal consistency loss, and few-sample discrimination loss, and then minimized. The calculation formula for the multi-task loss function is as follows: L total =λ1·L recon +λ2·L consis +λ3·L fewshot ; Where λ1, λ2, and λ3 are hyperparameters used to balance the importance of each loss term; L recon L represents the modal reconstruction loss. recon The original information distribution of the constraint modal input can be restored after processing by the compression adapter module, specifically in the form of: Among them, X m This represents the original input features of the m-th mode. This represents the reconstructed modal output, where M is the total number of modes; L consis L represents the modal consistency loss. consis Used to enhance the cooperative distribution of modalities in the shared semantic space and suppress feature shifts caused by physical interference, L consis Satisfying the relational expression: Among them, F i F j These represent the intermediate semantic features output by different modal encoders; L fewshot L represents the small sample discrimination loss. fewshot To enhance the discriminative ability of the compression adapter module in scenarios with scarce samples, it is trained using prototype loss or improved cross-entropy loss from metric learning.

7. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 1, characterized in that, The multimodal perception network includes a multimodal input module, a two-branch backbone network, a fusion module, a feature pyramid network, and a classification and regression head module; Step S5 includes the following sub-steps: S51: Acquire an RGB image of the water purifier's outer surface. The multimodal perception network simultaneously receives data input from multiple heterogeneous images, including RGB image features I. RGB Structured light 3D topography features I SL Polarization imaging features I pol Phase Reconstruction Map Features I Phase With multi-band spectral features I Spec The RGB image and the fused optical image are then input into the dual-branch backbone network, respectively. The dual-branch backbone network consists of two ResNet-18 backbone networks sharing weights. This dual-branch backbone network is used to extract feature representations of different modalities, outputting multi-scale feature maps in each branch. F RGB =ResNet(I RGB ),F Opt =ResNet(I Opt ); Each feature map F includes three output layers, namely: and S52: The multi-scale feature maps output by the dual-branch backbone network are input into the fusion module, which includes a channel switching module and a spatial attention module, wherein: The channel switching module receives feature maps from two branches, performs channel fusion processing, and outputs channel-enhanced feature maps: The enhanced feature map is input into the spatial attention module to generate a spatial attention map and reallocate spatial weights, resulting in the fused feature map: S53: Input the fused feature map into the feature pyramid network to construct multi-scale feature maps for semantic enhancement. in Outputting fused semantic features at different scales; S54: Perform defect identification and localization on the multi-scale feature maps output by the feature pyramid network, and output the classification results. Cls Output of location regression results Reg : Output Cls =ClsHead(F FPN ),Output Reg =RegHead(F FPN )。 8. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 7, characterized in that, In step S5, the training of the multimodal perception classification network also includes the following steps: A total loss function is constructed by introducing modal collaborative constraint loss, edge-aware consistency loss, and defect recognition joint loss. This total loss function is then minimized to optimize the network's multimodal feature representation ability, defect region edge perception ability, and classification and localization accuracy. The formula for calculating the total loss function is as follows: L total =μ1·L comodal +μ2·L edge +μ3·L joint ; Where μ1, μ2, and μ3 represent the weight hyperparameters indicating the importance of the loss term; Modal cooperative constraint loss L comodal To improve the consistency of representation between RGB and optical images at higher-order feature layers, a mutual information maximization strategy is employed for optimization, defined as follows: L comodal =-I(F rgb ,F opt ); Wherein, represents the mutual information metric between RGB image features and optical image features; Edge-aware consistency loss L edge To enhance the network's ability to perceive the structure of defect edge regions, it is calculated jointly using structural similarity and gradient difference metrics, and is defined as follows: in, Represents the Sobel operator. and These are the edge regions of the predicted map and the labeled map, respectively; The joint loss for defect identification includes a combination of Focalloss and smoothing L1 loss. This joint loss is used to simultaneously perform defect classification and location regression, satisfying the following relationship:

9. The method for detecting defects on the outer surface of a water purifier based on deep learning according to claim 1, characterized in that, Step S6 also includes the following steps: The performance of the trained defect detection model is evaluated using detection precision, recall, mean precision, and defect localization error.

Images