A 3C structural part surface defect detection method based on a semantic memory network

Abstract

The present application relates to the technical fields of computer vision and industrial automation, and discloses a 3C structural part surface defect detection method based on a semantic memory network, which comprises the following steps: simultaneously collecting two-dimensional images and three-dimensional topographic point cloud data of the surface of a 3C structural part in a detection cabin; after preprocessing the point cloud into a depth image, obtaining a multi-scale two-dimensional texture feature map and a multi-scale three-dimensional topographic feature map; inputting the multi-scale feature map into a pre-constructed cross-modal semantic memory enhancement module to output enhanced multi-scale two-dimensional texture feature maps and three-dimensional topographic feature maps; then obtaining a multi-scale cross-modal fusion feature map through adaptive fusion; decoding the fusion feature map to generate a preliminary defect segmentation map, and finally obtaining a final defect detection map through heuristic optimization processing. The present application can effectively overcome the problems of high reflectivity and micro-defect detection, reduce the false detection rate, and is suitable for online detection of 3C structural parts.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial automation technology, and more specifically, to a method for detecting surface defects in 3C structural components based on semantic memory networks. Background Technology

[0002] In the manufacturing of 3C (computer, communication, and consumer electronics) products, the surface quality of precision structural components such as aluminum alloy frames, stainless steel camera rings, and glass covers is a key indicator determining the product's appearance, corrosion resistance, and assembly performance. Scratches, pinholes, and other defects may occur in 3C structural components during the manufacturing process.

[0003] While some research has applied deep learning to surface defect detection in recent years, most models remain limited to static feature extraction from a single modality (2D grayscale or color images). They rely solely on 2D texture information and cannot acquire three-dimensional morphological features such as the depth, height, and surface normals of defects. Furthermore, when dealing with 2D images of 3C structural components, the material properties of these components make it difficult to accurately distinguish between real defects and optical artifacts on their surface, leading to inaccurate detection results.

[0004] Therefore, it is necessary to provide a surface defect detection method for 3C structural components based on semantic memory networks, which aims to solve the problem that current technologies cannot accurately distinguish between real defects and optical artifacts on the surface of 3C structural components. Summary of the Invention

[0005] In view of this, the present invention proposes a surface defect detection method for 3C structural components based on semantic memory networks, aiming to solve the problem that current technologies cannot accurately distinguish between real defects and optical artifacts on the surface of 3C structural components.

[0006] This invention proposes a method for detecting surface defects in 3C structural components based on semantic memory networks, comprising the following steps: S100. A light-shielding detection chamber is mounted on the 3C structural component conveyor belt. The light-shielding detection chamber is equipped with a controllable light source array. When the 3C structural component enters the light-shielding detection chamber with the conveyor belt, under the illumination of the controllable light source array in a preset lighting mode, two-dimensional images and three-dimensional topographic point cloud data of the surface of the 3C structural component are collected simultaneously. S200. Preprocess the three-dimensional topography point cloud data to obtain a three-dimensional topography depth image; extract features from the two-dimensional image and the three-dimensional topography depth image to obtain a multi-scale two-dimensional texture feature map and a multi-scale three-dimensional topography feature map, respectively. S300. The multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are sent to the pre-constructed cross-modal semantic memory enhancement module to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map. S400. Adaptively fuse the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map to obtain a multi-scale cross-modal fusion feature map. S500: Decode the multi-scale cross-modal fusion feature map to generate a preliminary defect segmentation map; S600. Perform heuristic optimization processing on the preliminary defect segmentation map to obtain the final defect detection map.

[0007] Furthermore, when performing feature extraction on the two-dimensional image and the three-dimensional shape depth image to obtain multi-scale two-dimensional texture feature maps and multi-scale three-dimensional shape feature maps, the process includes: Based on the first two-dimensional convolutional neural network, feature extraction is performed on the two-dimensional image data to obtain a multi-scale two-dimensional texture feature map; Feature extraction is performed on the three-dimensional topographic depth image based on a second two-dimensional convolutional neural network to obtain a multi-scale three-dimensional topographic feature map; wherein... The second two-dimensional convolutional neural network has the same structure as the first two-dimensional convolutional neural network; The three-dimensional topographic feature map at each scale has the same spatial size as the two-dimensional texture feature map at the same scale.

[0008] Furthermore, the pre-constructed cross-modal semantic memory enhancement module includes a cross-modal semantic memory pool corresponding to each scale. Each memory pool stores several memory units, and each memory unit contains a two-dimensional defect texture prototype feature vector and a three-dimensional shape prototype feature vector semantically aligned with the two-dimensional defect texture prototype feature vector.

[0009] Furthermore, the construction of the cross-modal semantic memory enhancement module also includes a training phase: The real defect regions in the training images are pre-labeled, wherein the training images include two-dimensional training images and three-dimensional topographic depth training images corresponding to the two-dimensional training images; Feature extraction is performed on the training images to obtain multi-scale feature maps of two-dimensional training images and multi-scale feature maps of three-dimensional topographic depth training images; The labeled real defect pixel regions in the two-dimensional training image are downsampled according to the spatial resolution of the feature maps at each scale of the two-dimensional training image to obtain binary mask mapping maps at each scale of the two-dimensional training image. The labeled real defect pixel regions in the three-dimensional topography depth training image corresponding to the two-dimensional training image are downsampled according to the spatial resolution of the feature maps at each scale of the three-dimensional topography depth training image to obtain the binary mask mapping map at each scale of the three-dimensional topography depth training image. In the binary mask mapping, a value of 1 for each coordinate point indicates that the location belongs to the real defect area, and a value of 0 for each coordinate point indicates the background. For each scale of the binary mask mapping map of the two-dimensional training image, the corresponding feature vectors of the feature map of the corresponding scale of the two-dimensional training image are extracted from the coordinate points with a value of 1 to obtain the first feature vector set. The first feature vector set is then subjected to average pooling to obtain the candidate two-dimensional defect texture prototype feature vector. For each scale of the binary mask mapping map of the 3D topography depth training image, the corresponding feature vectors in the feature map of the corresponding scale of the 3D topography depth training image are extracted from the coordinate points with a value of 1 to obtain the second feature vector set. The second feature vector set is then subjected to average pooling to obtain the candidate 3D topography prototype feature vector. The candidate two-dimensional defect texture prototype feature vector and the corresponding candidate three-dimensional shape prototype feature vector are combined to obtain the candidate memory unit; Obtain the similarity between the candidate memory unit and the existing memory units in the memory pool, and determine the memory unit with the highest similarity. When the similarity is greater than a preset similarity threshold, the memory unit with the highest similarity is updated using an exponential moving average mechanism. When the similarity is less than a preset similarity threshold, the candidate memory unit is added to the memory pool as a new memory unit.

[0010] Furthermore, when the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are fed into a pre-constructed cross-modal semantic memory enhancement module to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map, the process includes: Two-dimensional texture feature vectors are extracted from two-dimensional texture feature maps at each scale for each spatial location, and three-dimensional shape feature vectors are extracted from three-dimensional shape feature maps at each scale for the same spatial location; wherein, the spatial location is each coordinate point on the two-dimensional texture feature map and the three-dimensional shape feature map at each scale. The two-dimensional texture feature vector is matched with all two-dimensional defect texture prototype feature vectors in the memory pool to obtain the best matching two-dimensional defect texture prototype feature vector and the first similarity. The three-dimensional shape feature vector corresponding to the two-dimensional texture feature vector is matched with all three-dimensional shape prototype feature vectors in the memory pool to obtain the best matching three-dimensional shape prototype feature vector and the second similarity. Based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity, the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are processed respectively to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map.

[0011] Furthermore, when processing the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map, the process includes: Based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity, it is determined whether a real defect exists at the corresponding spatial location; wherein, When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to different memory units, it is determined that there is no real defect at the corresponding spatial location. When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to the same memory unit, and the first similarity is greater than or equal to a preset first similarity threshold and the second similarity is greater than or equal to a preset second similarity threshold, it is determined that there is a real defect at the corresponding spatial location. When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to the same memory unit, and the first similarity is less than a preset first similarity threshold and / or the second similarity is less than a preset second similarity threshold, it is determined that there is no real defect at the corresponding spatial location. When it is determined that there is a real defect at the corresponding spatial location, the corresponding two-dimensional texture feature vector is enhanced based on the best-matched two-dimensional defect texture prototype feature vector, and the corresponding three-dimensional shape feature vector is enhanced based on the best-matched three-dimensional shape prototype feature vector. When it is determined that there is no real defect at the corresponding spatial location, the corresponding two-dimensional texture feature vector and three-dimensional shape feature vector are set to zero; After enhancing or setting all two-dimensional texture feature vectors in the multi-scale two-dimensional texture feature map to zero, the enhanced multi-scale two-dimensional texture feature map is obtained. After enhancing or setting all three-dimensional topographic feature vectors in the multi-scale three-dimensional topographic feature map to zero, the enhanced multi-scale three-dimensional topographic feature map is obtained.

[0012] Furthermore, when enhancing the corresponding two-dimensional texture feature vector based on the best-matched two-dimensional defect texture prototype feature vector, and when enhancing the corresponding three-dimensional shape feature vector based on the best-matched three-dimensional shape prototype feature vector, the process includes: Based on the best-matched two-dimensional defect texture prototype feature vector, a first scaling factor and a first translation factor are generated by a first multilayer perceptron, and a channel-level affine transformation is performed on the corresponding two-dimensional texture feature vector based on the first scaling factor and the first translation factor. Based on the best-matched 3D shape prototype feature vector, a second scaling factor and a second translation factor are generated through a second multilayer perceptron. A channel-level affine transformation is then performed on the corresponding 3D shape feature vector based on the second scaling factor and the second translation factor. The first multilayer perceptron and the second multilayer perceptron have the same structure.

[0013] Furthermore, when adaptively fusing the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map to obtain the multi-scale cross-modal fusion feature map, the process includes: The enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map are concatenated along the channel dimension, and then channel compression is performed through a 1×1 convolutional layer to obtain a multi-scale cross-modal fusion feature map.

[0014] Furthermore, the aforementioned method for detecting surface defects in 3C structural components based on semantic memory networks also includes the following steps: Based on the final defect detection image, it is determined whether there are defects on the surface of the 3C structural component. When it is determined that there are defects on the surface of the 3C structural component, a rejection instruction is output to the downstream sorting mechanism.

[0015] Furthermore, when determining whether there are defects on the surface of the 3C structural component based on the final defect detection image, the process includes: The total area containing defective pixels is obtained, and based on the total area, it is determined whether there are defects on the surface of the 3C structural component; wherein... When the total area is greater than a preset area threshold, it is determined that there is a defect on the surface of the 3C structural component; When the total area is less than or equal to a preset area threshold, it is determined that there are no defects on the surface of the 3C structural component.

[0016] Compared with existing technologies, the advantages of this invention are as follows: A light-shielding detection chamber is mounted on the 3C structural component conveyor belt. This chamber effectively shields against interference from external natural light and stray light from the workshop, ensuring a consistent imaging environment and avoiding the impact of lighting changes on defect identification. The light-shielding detection chamber is equipped with a controllable light source array, which can adaptively adjust the lighting mode (brightness, color temperature, angle) according to different materials and surface roughness, ensuring maximum highlighting of defect features and reducing interference from reflected light on the 3C structural component surface. When the 3C structural component enters the light-shielding detection chamber with the conveyor belt, under the illumination of the controllable light source array in a preset lighting mode, two-dimensional images and three-dimensional topographic point cloud data of the 3C structural component surface are simultaneously acquired. This provides spatiotemporally aligned multidimensional information for subsequent cross-modal fusion, offering a reliable data foundation. The 3D topographic point cloud data is preprocessed to obtain a 3D topographic depth image, converting the point cloud data into a depth map for subsequent feature extraction. Feature extraction is performed on the 2D image and the 3D topographic depth image to obtain multi-scale 2D texture feature maps and multi-scale 3D topographic feature maps, respectively. The multi-scale feature maps take into account both global semantics and local details, enabling the model to capture both large-scale scratches and pinholes. The multi-scale 2D texture feature maps and multi-scale 3D topographic feature maps are fed into a pre-constructed cross-modal semantic memory enhancement module to obtain enhanced multi-scale 2D texture feature maps and enhanced multi-scale 3D topographic feature maps. This approach combines the accuracy of 2D texture feature maps with the immunity to reflections in 3D topographic feature maps, accurately distinguishing between real defects and highly reflective artifacts, thus enabling accurate feature map enhancement. The enhanced multi-scale 2D texture feature maps and enhanced multi-scale 3D topographic feature maps are adaptively fused to obtain a multi-scale cross-modal fused feature map, which integrates complementary information and improves the clarity of defect boundaries. The multi-scale cross-modal fusion feature map is decoded to generate a preliminary defect segmentation map, ensuring that the generated preliminary defect segmentation map retains details of minute defects. Heuristic optimization processing is then performed on the preliminary defect segmentation map to obtain the final defect detection map. Further optimization of the segmentation results reduces isolated noise and improves the quality of the final defect detection map.

[0017] In summary, this invention overcomes the interference of high reflectivity artifacts by using a light-shielding chamber for acquisition, dual-path feature extraction, cross-modal memory enhancement, adaptive fusion, decoding, and post-processing to achieve pixel-level defect detection, significantly reducing false detections and missed detections, and meeting the online high-precision quality inspection requirements of 3C structural components. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a surface defect detection method for 3C structural components based on semantic memory networks provided in an embodiment of the present invention; Figure 2 The flowchart illustrates the processing of the cross-modal semantic memory enhancement module provided in this embodiment of the invention. Figure 3 This is a flowchart of the training phase of the cross-modal semantic memory enhancement module provided in an embodiment of the present invention; Figure 4 The logic flowchart for determining the surface of a 3C structural component is provided in an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey its scope to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] In some embodiments of this application, see Figure 1-4 As shown, this embodiment provides a method for detecting surface defects in 3C structural components based on semantic memory networks, including the following steps: S100. A light-shielding detection chamber is mounted on the 3C structural component conveyor belt. The light-shielding detection chamber is equipped with a controllable light source array. When the 3C structural component enters the light-shielding detection chamber with the conveyor belt, under the illumination of the controllable light source array in a preset lighting mode, two-dimensional images and three-dimensional topographic point cloud data of the surface of the 3C structural component are collected simultaneously.

[0021] In this embodiment, the 3C structural component to be inspected is a mobile phone back cover. A light-shielding inspection chamber is installed above the mobile phone back cover conveyor belt, and the inner wall of the chamber is coated with a black light-absorbing material. A multi-angle ring light source is installed at the top of the chamber. The light source is a controllable LED array, and its brightness, color temperature, and on / off timing can be adjusted independently. A 2D camera is used to acquire a two-dimensional image of the mobile phone back cover. The 2D camera is an industrial camera commonly used in this field. A 3D line laser profilometer is used to acquire the three-dimensional topographic point cloud data of the mobile phone back cover. The 2D camera and the 3D line laser profilometer are coaxially mounted to ensure that the image data captured by the two are spatially and temporally aligned.

[0022] S200. Preprocess the three-dimensional topography point cloud data to obtain a three-dimensional topography depth image; extract features from the two-dimensional image and the three-dimensional topography depth image to obtain a multi-scale two-dimensional texture feature map and a multi-scale three-dimensional topography feature map, respectively.

[0023] Specifically, when performing feature extraction on the two-dimensional image and the three-dimensional shape depth image to obtain multi-scale two-dimensional texture feature maps and multi-scale three-dimensional shape feature maps, the process includes: Based on the first two-dimensional convolutional neural network, feature extraction is performed on the two-dimensional image data to obtain a multi-scale two-dimensional texture feature map; Feature extraction is performed on the three-dimensional topographic depth image based on a second two-dimensional convolutional neural network to obtain a multi-scale three-dimensional topographic feature map; wherein... The second two-dimensional convolutional neural network has the same structure as the first two-dimensional convolutional neural network; The three-dimensional topographic feature map at each scale has the same spatial size as the two-dimensional texture feature map at the same scale.

[0024] In this embodiment, after preprocessing the 3D topographic point cloud data, the resulting 3D topographic depth image has the same resolution as the 2D image, both being 384×384. Two identical convolutional neural networks are used to obtain 2D texture feature maps and 3D topographic feature maps at four scales. Both convolutional neural networks can use the lightweight SMT-T architecture, which includes four stages: the first stage is a 7×7 convolution with a stride of 2, outputting a 192×192×64 feature map; the second stage uses two scale-aware modulation modules (SAM), outputting a 96×96×128 feature map; the third stage uses two SAMs and one multi-head self-attention module (MSA), outputting a 48×48×256 feature map; and the fourth stage uses two MSA modules, outputting a 24×24×512 feature map. This results in a 4-scale 2D texture feature map. and 4-scale three-dimensional topographic feature map .

[0025] It is understandable that using two convolutional neural networks with the same structure means that the two networks have the same receptive field, downsampling rate and number of feature map channels. This symmetrical design makes the two-dimensional and three-dimensional feature maps of the same scale naturally have a one-to-one spatial relationship, without the need for additional registration or interpolation operations, which greatly simplifies the implementation of subsequent cross-modal fusion. Second, the two networks can be trained independently (or jointly fine-tuned) to focus on the representation of texture information and geometric information respectively, avoiding modal interference caused by shared parameters.

[0026] S300. The multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are sent to the pre-constructed cross-modal semantic memory enhancement module to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map.

[0027] Specifically, the pre-constructed cross-modal semantic memory enhancement module includes a cross-modal semantic memory pool corresponding to each scale. Each memory pool stores several memory units, and each memory unit contains a two-dimensional defect texture prototype feature vector and a three-dimensional shape prototype feature vector semantically aligned with the two-dimensional defect texture prototype feature vector.

[0028] In this embodiment, four independent memory pools, denoted as M1 to M4, are established for the four scales mentioned above. The capacity of each memory pool is set to 200 memory units. Each memory unit is a structure containing two floating-point arrays: a two-dimensional prototype vector (the dimension is the same as the number of channels of the feature map of the corresponding scale, for example, the prototype dimension of M3 is 256) and a three-dimensional prototype vector (the dimension is also 256).

[0029] Understandably, multi-scale independent memory pools solve the problem of different feature dimensions at different scales—shallow memory pools store low-dimensional prototypes (e.g., 64-dimensional), while deep memory pools store high-dimensional prototypes (e.g., 512-dimensional), avoiding the ineffective similarity calculation caused by mixing vectors of different dimensions into the same pool. Secondly, each memory pool only serves its corresponding scale, making the retrieval process more efficient because the retrieval scope is limited to a reasonable dimensional space. Thirdly, the two-dimensional and three-dimensional prototypes in the memory unit are strictly aligned semantically, meaning that the two prototypes in the same memory unit describe the same type of defect (e.g., scratches) in different modalities. This alignment relationship is the core of the subsequent "cross-modal consistency verification"—only when the current two-dimensional feature matches the two-dimensional prototype, the current three-dimensional feature matches the three-dimensional prototype, and the two prototypes belong to the same memory unit, is it determined to be a real defect. This design fundamentally solves the problem of false detection of highly reflective artifacts, because reflective images appear as scratches in two dimensions, but their three-dimensional shape does not match the shape prototype of the real scratch, thus failing the consistency check.

[0030] Specifically, the construction of the cross-modal semantic memory enhancement module also includes a training phase: The real defect regions in the training images are pre-labeled, wherein the training images include two-dimensional training images and three-dimensional topographic depth training images corresponding to the two-dimensional training images; Feature extraction is performed on the training images to obtain multi-scale feature maps of two-dimensional training images and multi-scale feature maps of three-dimensional topographic depth training images; The labeled real defect pixel regions in the two-dimensional training image are downsampled according to the spatial resolution of the feature maps at each scale of the two-dimensional training image to obtain binary mask mapping maps at each scale of the two-dimensional training image. The labeled real defect pixel regions in the three-dimensional topography depth training image corresponding to the two-dimensional training image are downsampled according to the spatial resolution of the feature maps at each scale of the three-dimensional topography depth training image to obtain the binary mask mapping map at each scale of the three-dimensional topography depth training image. In the binary mask mapping, a value of 1 for each coordinate point indicates that the location belongs to the real defect area, and a value of 0 for each coordinate point indicates the background. For each scale of the binary mask mapping map of the two-dimensional training image, the corresponding feature vectors of the feature map of the corresponding scale of the two-dimensional training image are extracted from the coordinate points with a value of 1 to obtain the first feature vector set. The first feature vector set is then subjected to average pooling to obtain the candidate two-dimensional defect texture prototype feature vector. For each scale of the binary mask mapping map of the 3D topography depth training image, the corresponding feature vectors in the feature map of the corresponding scale of the 3D topography depth training image are extracted from the coordinate points with a value of 1 to obtain the second feature vector set. The second feature vector set is then subjected to average pooling to obtain the candidate 3D topography prototype feature vector. The candidate two-dimensional defect texture prototype feature vector and the corresponding candidate three-dimensional shape prototype feature vector are combined to obtain the candidate memory unit; Obtain the similarity between the candidate memory unit and the existing memory units in the memory pool, and determine the memory unit with the highest similarity. When the similarity is greater than a preset similarity threshold, the memory unit with the highest similarity is updated using an exponential moving average mechanism. When the similarity is less than a preset similarity threshold, the candidate memory unit is added to the memory pool as a new memory unit.

[0031] In this embodiment, all memory pools are initially empty before training begins. During training, when a new defect type is encountered, the corresponding 2D and 3D prototypes are paired and added to the memory pool of the corresponding scale. For existing defect types, the existing prototypes are updated using an exponential moving average. The dimensions of all 2D prototypes in the same memory pool remain consistent, and the dimensions of all 3D prototypes also remain consistent to ensure the validity of subsequent cosine similarity calculations. Taking scale 3 (feature map 48×48) as an example: First, for each image in the training set, a pixel-level mask of the defect is manually labeled (a binary image on the original 384×384 image). This mask is downsampled to 48×48 using nearest neighbor mapping to obtain mask mapping map N3 (48×48). Iterate through all points with coordinates (i,j) in N3. If N3(i,j)=1, extract a 256-dimensional feature vector from F2D_3(i,j) and collect all such vectors to form set S2D; simultaneously, extract the corresponding 256-dimensional vector from F3D_3(i,j) and collect it to form the first feature vector set S3D. Average all vectors in S2D element-wise to obtain a 256-dimensional candidate two-dimensional defect texture prototype feature vector p2d; similarly, average all vectors in S3D element-wise to obtain a candidate three-dimensional shape prototype feature vector p3d. Combine (p2d, p3d) into a candidate memory unit. Then, calculate the cosine similarity between this candidate unit and existing memory units in M3. Find the maximum similarity and its corresponding memory unit k. Set the similarity threshold to 0.85. If the maximum similarity is greater than 0.85, the memory cell with the highest similarity is updated using an exponential moving average mechanism; if the maximum similarity is less than or equal to 0.85, (p2d, p3d) is added as a new memory cell to M3. After training all images, the memory pool converges to a stable prototype of various defects.

[0032] Understandably, downsampling to obtain a binary mask map solves the alignment problem between the original icon annotation and feature maps of different scales, ensuring accurate extraction of candidate prototype positions. Extracting a set of feature vectors from all coordinate points with a value of 1 reflects the complete coverage of the defect region on the feature map, avoiding feature bias caused by only taking the center point. Average pooling of the feature vector set merges multiple feature vectors into a representative prototype, eliminating noise while preserving the essential features of the defect, and controlling the memory pool capacity to prevent it from exploding. Combining two-dimensional and three-dimensional candidate prototypes into candidate memory units ensures that the memory pool stores cross-modal association pairs. By calculating the similarity between candidate units and existing units, effective compression of repetitive defect knowledge is achieved—multiple redundant prototypes of the same type of defect are not stored; instead, a unique prototype is continuously optimized through exponential moving average. A preset similarity threshold allows the model to distinguish between "same defect type" (updated) and "new defect type" (added), thereby enabling dynamic evolution of the memory pool to adapt to multi-batch, multi-material production environments. This training method ensures that the memory pool is discriminative, compact, and adaptive.

[0033] Specifically, when the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are fed into a pre-constructed cross-modal semantic memory enhancement module to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map, the process includes: Two-dimensional texture feature vectors are extracted from two-dimensional texture feature maps at each scale for each spatial location, and three-dimensional shape feature vectors are extracted from three-dimensional shape feature maps at each scale for the same spatial location; wherein, the spatial location is each coordinate point on the two-dimensional texture feature map and the three-dimensional shape feature map at each scale. The two-dimensional texture feature vector is matched with all two-dimensional defect texture prototype feature vectors in the memory pool to obtain the best matching two-dimensional defect texture prototype feature vector and the first similarity. The three-dimensional shape feature vector corresponding to the two-dimensional texture feature vector is matched with all three-dimensional shape prototype feature vectors in the memory pool to obtain the best matching three-dimensional shape prototype feature vector and the second similarity. Based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity, the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are processed respectively to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map.

[0034] In this embodiment, taking scale 3 as an example, feature maps F2D_3 (48×48×256) and F3D_3 (48×48×256) have been obtained, along with a pre-trained M3 (capacity 200). For each coordinate (i,j) on the feature map, the two-dimensional texture feature vector v2d=F2D_3(i,j) (256-dimensional) and the three-dimensional shape feature vector v3d=F3D_3(i,j) (256-dimensional) are extracted. The cosine similarity between v2d and each of the two-dimensional prototypes p2d_k in M3 is calculated, and the highest similarity is taken as the first similarity. The corresponding feature vector is taken as the best-matching two-dimensional defect texture prototype feature vector. Similarly, the best-matching three-dimensional shape prototype feature vector and the second similarity are obtained.

[0035] It is understandable that each spatial location, i.e., each coordinate point on the feature map, signifies that the granularity of defect detection in this invention reaches the feature map level (corresponding to a local neighborhood of the original image), enabling fine point-by-point discrimination without missing minute defects. Independent retrieval of the two-dimensional and three-dimensional feature vectors allows the model to obtain the most similar memories from both texture and shape perspectives. These two matching results may be consistent (pointing to the same memory unit) or inconsistent (pointing to different memory units); it is precisely the inconsistency that reveals the existence of highly reflective artifacts. Recording the prototype and similarity score of the best match provides a complete basis for subsequent cross-modal consistency judgment. Since each scale has an independent memory pool, retrieval only needs to be performed in the memory pool of the current scale, and the computational cost is linearly related to the memory pool capacity, making it efficient and feasible in engineering.

[0036] Specifically, when processing the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map, the process includes: Based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity, it is determined whether a real defect exists at the corresponding spatial location; wherein, When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to different memory units, it is determined that there is no real defect at the corresponding spatial location. When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to the same memory unit, and the first similarity is greater than or equal to a preset first similarity threshold and the second similarity is greater than or equal to a preset second similarity threshold, it is determined that there is a real defect at the corresponding spatial location. When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to the same memory unit, and the first similarity is less than a preset first similarity threshold and / or the second similarity is less than a preset second similarity threshold, it is determined that there is no real defect at the corresponding spatial location. When it is determined that there is a real defect at the corresponding spatial location, the corresponding two-dimensional texture feature vector is enhanced based on the best-matched two-dimensional defect texture prototype feature vector, and the corresponding three-dimensional shape feature vector is enhanced based on the best-matched three-dimensional shape prototype feature vector. When it is determined that there is no real defect at the corresponding spatial location, the corresponding two-dimensional texture feature vector and three-dimensional shape feature vector are set to zero; After enhancing or setting all two-dimensional texture feature vectors in the multi-scale two-dimensional texture feature map to zero, the enhanced multi-scale two-dimensional texture feature map is obtained. After enhancing or setting all three-dimensional topographic feature vectors in the multi-scale three-dimensional topographic feature map to zero, the enhanced multi-scale three-dimensional topographic feature map is obtained.

[0037] In this embodiment, the preset first similarity threshold is 0.7, and the preset second similarity threshold is 0.6.

[0038] Understandably, if two best-matching prototypes belong to different memory units, or if they belong to the same unit but either similarity is below a threshold, they are considered to lack a true defect. This rule directly corresponds to the mathematical expression of "cross-modal consistency," completely solving the problem of 2D false detection in highly reflective scenes. For true defects, the matched prototype is used to enhance the current feature vector, amplifying the response of the defect region, which is beneficial for accurate segmentation by the subsequent decoder. For artifacts or background, the feature vector is set to zero, which is equivalent to directly "erasing" interference information in the feature space, preventing false detections from being passed to the final output. This processing is performed independently for each coordinate point, maintaining the spatial resolution. The final enhanced feature map has the same size as the original feature map and can be directly input into the fusion module. Although the binary operation of enhancement and zeroing is simple, it essentially implements a memory-based attention mechanism. Only regions verified by historical knowledge are retained; otherwise, they are filtered out, making it more robust and interpretable than traditional attention mechanisms.

[0039] Specifically, when enhancing the corresponding two-dimensional texture feature vector based on the best-matched two-dimensional defect texture prototype feature vector, and when enhancing the corresponding three-dimensional shape feature vector based on the best-matched three-dimensional shape prototype feature vector, the process includes: Based on the best-matched two-dimensional defect texture prototype feature vector, a first scaling factor and a first translation factor are generated by a first multilayer perceptron, and a channel-level affine transformation is performed on the corresponding two-dimensional texture feature vector based on the first scaling factor and the first translation factor. Based on the best-matched 3D shape prototype feature vector, a second scaling factor and a second translation factor are generated through a second multilayer perceptron. A channel-level affine transformation is then performed on the corresponding 3D shape feature vector based on the second scaling factor and the second translation factor. The first multilayer perceptron and the second multilayer perceptron have the same structure.

[0040] In this embodiment, the Multilayer Perceptron (MLP) structure is as follows: The first layer is fully connected, with an input dimension of 256 and an output dimension of 128, batch normalized and then connected to ReLU; the second layer is also fully connected, with an input dimension of 128 and an output dimension of 512. The 512 outputs are divided into two parts: the first 256 are used as a scaling factor γ, and the last 256 are used as a translation factor β. To prevent γ from being too large or too small, γ is scaled using a sigmoid function and then multiplied by 2 (so that γ is within the range of 0 to 2). When calculating the affine transformation, since v2d is a 256-dimensional vector, γ and β are also 256-dimensional, and are directly multiplied element-wise and then added. For the 3D part, another MLP with the same initialization is used, but the weights are not shared because the optimal modulation parameter distributions for 2D texture and 3D shape may be different. This MLP is jointly optimized with the classifier during training, and the loss function includes segmentation loss and memory regularization term.

[0041] Understandably, an MLP is a lightweight, fully connected network capable of learning a nonlinear mapping from prototype vectors to modulation parameters, allowing the modulation parameters to adapt to defect features of different scales and types. The scaling factor γ controls the amplification of each channel, and the translation factor β controls the offset of each channel. This channel-by-channel independent modulation is more expressive than global scaling, enabling targeted enhancement of defect-related channels and suppression of irrelevant channels. Using independent but structurally identical MLPs for two-dimensional and three-dimensional modulation reflects that while the modulation strategies for the two modes can differ, maintaining structural symmetry facilitates parameter sharing or transfer learning.

[0042] S400. Adaptively fuse the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map to obtain a multi-scale cross-modal fusion feature map.

[0043] Specifically, when adaptively fusing the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map to obtain a multi-scale cross-modal fused feature map, the following steps are included: The enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map are concatenated along the channel dimension, and then channel compression is performed through a 1×1 convolutional layer to obtain a multi-scale cross-modal fusion feature map.

[0044] Understandably, concatenation along channels is the simplest multimodal fusion method, preserving all information from both 2D and 3D features without losing any modal details. The role of the 1×1 convolutional layer is to perform channel compression and linear combination on the concatenated high-dimensional features, learning how to assign weights to 2D and 3D features at each spatial location. Essentially, it's a learnable weighted summation, more flexible than fixed-weight fusion. The 1×1 convolutional kernel doesn't change the resolution in the spatial dimension, ensuring consistent feature map sizes before and after fusion, facilitating integration with subsequent decoders. Multi-scale fusion means performing independent fusion operations at each scale. Shallow fusion preserves edge details, deep fusion preserves semantic categories, and finally, during decoding, upsampling integrates the fused features from different scales, resulting in an output segmentation map with both accurate boundaries and correct category attribution.

[0045] S500. Decode the multi-scale cross-modal fusion feature map to generate a preliminary defect segmentation map.

[0046] Specifically, the decoding employs a context-driven fusion model. This model includes a lightweight dilated spatial pyramid pooling module and a context-aware coordinate attention module. The lightweight dilated spatial pyramid pooling module uses depthwise separable convolutions to extract multi-scale contextual features. The context-aware coordinate attention module is used to generate spatial attention weights to focus on the defect region. The decoding process recovers a high-resolution defect segmentation map by progressively upsampling and fusing features at different scales.

[0047] S600. Perform heuristic optimization processing on the preliminary defect segmentation map to obtain the final defect detection map.

[0048] Specifically, the heuristic optimization process includes edge-guided suppression, adaptive binarization, and morphological operations; edge-guided suppression is used to reduce background interference by utilizing image edge information, adaptive binarization is used to convert the probability map into a binary mask, and morphological operations are used to optimize the continuity and smoothness of defect boundaries.

[0049] In other embodiments of this application, the method for detecting surface defects in 3C structural components based on semantic memory networks is characterized by further comprising the following steps: S700. Based on the final defect detection image, determine whether there is a defect on the surface of the 3C structural component. When it is determined that there is a defect on the surface of the 3C structural component, output a rejection instruction to the downstream sorting mechanism.

[0050] Specifically, when determining whether there are defects on the surface of the 3C structural component based on the final defect detection image, the following steps are included: The total area containing defective pixels is obtained, and based on the total area, it is determined whether there are defects on the surface of the 3C structural component; wherein... When the total area is greater than a preset area threshold, it is determined that there is a defect on the surface of the 3C structural component; When the total area is less than or equal to a preset area threshold, it is determined that there are no defects on the surface of the 3C structural component.

[0051] In this embodiment, the preset area threshold is 100, which corresponds to an actual area of ​​approximately 0.05 mm. 2 The production line can be controlled by industrial computers, PLCs, or other commonly used technologies in this field.

[0052] Understandably, directly linking the detection algorithm with the execution mechanism forms an automated closed loop of "detection-decision-execution," eliminating the need for manual intervention and significantly improving the automation level of the production line. Using the total area of ​​defective pixels as a judgment indicator is simple and effective, comprehensively reflecting the severity of defects and avoiding missed judgments that may occur based solely on the number of defects or their maximum size (for example, multiple tiny, scattered pinholes, although small in individual area, should be judged as unqualified if their total area exceeds the threshold). The preset area threshold can be flexibly adjusted according to different customer quality standards; for example, a stricter threshold (such as 0.01mm) can be set for appearance parts. 2 The internal structural components can be appropriately relaxed, making it highly adaptable.

[0053] Understandably, a light-shielding inspection chamber is installed on the 3C structural component conveyor belt. This chamber effectively shields against interference from external natural light and stray light from the workshop, ensuring a consistent imaging environment and preventing the impact of lighting changes on defect identification. The chamber contains a controllable light source array, which adaptively adjusts the lighting mode (brightness, color temperature, angle) based on different materials and surface roughness to maximize the highlighting of defect features and reduce interference from reflected light on the 3C structural component surface. When the 3C structural component enters the light-shielding inspection chamber with the conveyor belt, under the illumination of the controllable light source array in a preset lighting mode, two-dimensional images and three-dimensional topographic point cloud data of the 3C structural component surface are simultaneously acquired. This provides spatiotemporally aligned multidimensional information for subsequent cross-modal fusion, offering a reliable data foundation. The 3D topographic point cloud data is preprocessed to obtain a 3D topographic depth image, converting the point cloud data into a depth map for subsequent feature extraction. Feature extraction is performed on the 2D image and the 3D topographic depth image to obtain multi-scale 2D texture feature maps and multi-scale 3D topographic feature maps, respectively. The multi-scale feature maps take into account both global semantics and local details, enabling the model to capture both large-scale scratches and pinholes. The multi-scale 2D texture feature maps and multi-scale 3D topographic feature maps are fed into a pre-constructed cross-modal semantic memory enhancement module to obtain enhanced multi-scale 2D texture feature maps and enhanced multi-scale 3D topographic feature maps. This approach combines the accuracy of 2D texture feature maps with the immunity to reflections in 3D topographic feature maps, accurately distinguishing between real defects and highly reflective artifacts, thus enabling accurate feature map enhancement. The enhanced multi-scale 2D texture feature maps and enhanced multi-scale 3D topographic feature maps are adaptively fused to obtain a multi-scale cross-modal fused feature map, which integrates complementary information and improves the clarity of defect boundaries. The multi-scale cross-modal fusion feature map is decoded to generate a preliminary defect segmentation map, ensuring that the generated preliminary defect segmentation map retains details of minute defects. Heuristic optimization processing is then performed on the preliminary defect segmentation map to obtain the final defect detection map. Further optimization of the segmentation results reduces isolated noise and improves the quality of the final defect detection map.

[0054] In summary, this invention overcomes the interference of high reflectivity artifacts by using a light-shielding chamber for acquisition, dual-path feature extraction, cross-modal memory enhancement, adaptive fusion, decoding, and post-processing to achieve pixel-level defect detection, significantly reducing false detections and missed detections, and meeting the online high-precision quality inspection requirements of 3C structural components.

[0055] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting surface defects in 3C structural components based on semantic memory networks, characterized in that, Includes the following steps: A light-shielding detection chamber is mounted on the 3C structural component conveyor belt, and a controllable light source array is set inside the light-shielding detection chamber. When the 3C structural component enters the light-shielding detection chamber with the conveyor belt, under the illumination of the controllable light source array in a preset lighting mode, two-dimensional images and three-dimensional topographic point cloud data of the surface of the 3C structural component are collected simultaneously. The three-dimensional topography point cloud data is preprocessed to obtain a three-dimensional topography depth image; features are extracted from the two-dimensional image and the three-dimensional topography depth image to obtain a multi-scale two-dimensional texture feature map and a multi-scale three-dimensional topography feature map, respectively. The multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are fed into a pre-constructed cross-modal semantic memory enhancement module to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map. The enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional topography feature map are adaptively fused to obtain a multi-scale cross-modal fusion feature map. The multi-scale cross-modal fusion feature map is decoded to generate a preliminary defect segmentation map; The preliminary defect segmentation map is subjected to heuristic optimization processing to obtain the final defect detection map.

2. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 1, characterized in that, When performing feature extraction on the two-dimensional image and the three-dimensional shape depth image to obtain multi-scale two-dimensional texture feature maps and multi-scale three-dimensional shape feature maps, the process includes: Based on the first two-dimensional convolutional neural network, feature extraction is performed on the two-dimensional image data to obtain a multi-scale two-dimensional texture feature map; Feature extraction is performed on the three-dimensional topographic depth image based on a second two-dimensional convolutional neural network to obtain a multi-scale three-dimensional topographic feature map; wherein... The second two-dimensional convolutional neural network has the same structure as the first two-dimensional convolutional neural network; The three-dimensional topographic feature map at each scale has the same spatial size as the two-dimensional texture feature map at the same scale.

3. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 2, characterized in that, The pre-built cross-modal semantic memory enhancement module includes a cross-modal semantic memory pool corresponding to each scale. Each memory pool stores several memory units. Each memory unit contains a two-dimensional defect texture prototype feature vector and a three-dimensional shape prototype feature vector that is semantically aligned with the two-dimensional defect texture prototype feature vector.

4. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 3, characterized in that, The construction of the cross-modal semantic memory enhancement module also includes a training phase: The real defect regions in the training images are pre-labeled, wherein the training images include two-dimensional training images and three-dimensional topographic depth training images corresponding to the two-dimensional training images; Feature extraction is performed on the training images to obtain multi-scale feature maps of two-dimensional training images and multi-scale feature maps of three-dimensional topographic depth training images; The labeled real defect pixel regions in the two-dimensional training image are downsampled according to the spatial resolution of the feature maps at each scale of the two-dimensional training image to obtain binary mask mapping maps at each scale of the two-dimensional training image. The labeled real defect pixel regions in the three-dimensional topography depth training image corresponding to the two-dimensional training image are downsampled according to the spatial resolution of the feature maps at each scale of the three-dimensional topography depth training image to obtain the binary mask mapping map at each scale of the three-dimensional topography depth training image. In the binary mask mapping, a value of 1 for each coordinate point indicates that the location belongs to the real defect area, and a value of 0 for each coordinate point indicates the background. For each scale of the binary mask mapping map of the two-dimensional training image, the corresponding feature vectors of the feature map of the corresponding scale of the two-dimensional training image are extracted from the coordinate points with a value of 1 to obtain the first feature vector set. The first feature vector set is then subjected to average pooling to obtain the candidate two-dimensional defect texture prototype feature vector. For each scale of the binary mask mapping map of the 3D topography depth training image, the corresponding feature vectors in the feature map of the corresponding scale of the 3D topography depth training image are extracted from the coordinate points with a value of 1 to obtain the second feature vector set. The second feature vector set is then subjected to average pooling to obtain the candidate 3D topography prototype feature vector. The candidate two-dimensional defect texture prototype feature vector and the corresponding candidate three-dimensional shape prototype feature vector are combined to obtain the candidate memory unit; Obtain the similarity between the candidate memory unit and the existing memory units in the memory pool, and determine the memory unit with the highest similarity. When the similarity is greater than a preset similarity threshold, the memory unit with the highest similarity is updated using an exponential moving average mechanism. When the similarity is less than a preset similarity threshold, the candidate memory unit is added to the memory pool as a new memory unit.

5. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 4, characterized in that, When the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are fed into a pre-constructed cross-modal semantic memory enhancement module to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map, the process includes: Two-dimensional texture feature vectors are extracted from two-dimensional texture feature maps at each scale for each spatial location, and three-dimensional shape feature vectors are extracted from three-dimensional shape feature maps at each scale for the same spatial location; wherein, the spatial location is each coordinate point on the two-dimensional texture feature map and the three-dimensional shape feature map at each scale. The two-dimensional texture feature vector is matched with all two-dimensional defect texture prototype feature vectors in the memory pool to obtain the best matching two-dimensional defect texture prototype feature vector and the first similarity. The three-dimensional shape feature vector corresponding to the two-dimensional texture feature vector is matched with all three-dimensional shape prototype feature vectors in the memory pool to obtain the best matching three-dimensional shape prototype feature vector and the second similarity. Based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity, the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map are processed respectively to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map.

6. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 5, characterized in that, The process of processing the multi-scale two-dimensional texture feature map and the multi-scale three-dimensional shape feature map based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity to obtain the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map includes: Based on the best-matched two-dimensional defect texture prototype feature vector, the first similarity, the best-matched three-dimensional shape prototype feature vector, and the second similarity, it is determined whether a real defect exists at the corresponding spatial location; wherein, When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to different memory units, it is determined that there is no real defect at the corresponding spatial location. When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to the same memory unit, and the first similarity is greater than or equal to a preset first similarity threshold and the second similarity is greater than or equal to a preset second similarity threshold, it is determined that there is a real defect at the corresponding spatial location. When the best-matching two-dimensional defect texture prototype feature vector and the best-matching three-dimensional shape prototype feature vector belong to the same memory unit, and the first similarity is less than a preset first similarity threshold and / or the second similarity is less than a preset second similarity threshold, it is determined that there is no real defect at the corresponding spatial location. When it is determined that there is a real defect at the corresponding spatial location, the corresponding two-dimensional texture feature vector is enhanced based on the best-matched two-dimensional defect texture prototype feature vector, and the corresponding three-dimensional shape feature vector is enhanced based on the best-matched three-dimensional shape prototype feature vector. When it is determined that there is no real defect at the corresponding spatial location, the corresponding two-dimensional texture feature vector and three-dimensional shape feature vector are set to zero; After enhancing or setting all two-dimensional texture feature vectors in the multi-scale two-dimensional texture feature map to zero, the enhanced multi-scale two-dimensional texture feature map is obtained. After enhancing or setting all three-dimensional topographic feature vectors in the multi-scale three-dimensional topographic feature map to zero, the enhanced multi-scale three-dimensional topographic feature map is obtained.

7. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 6, characterized in that, When enhancing the corresponding two-dimensional texture feature vector based on the best-matched two-dimensional defect texture prototype feature vector, and when enhancing the corresponding three-dimensional shape feature vector based on the best-matched three-dimensional shape prototype feature vector, the process includes: Based on the best-matched two-dimensional defect texture prototype feature vector, a first scaling factor and a first translation factor are generated by a first multilayer perceptron, and a channel-level affine transformation is performed on the corresponding two-dimensional texture feature vector based on the first scaling factor and the first translation factor. Based on the best-matched 3D shape prototype feature vector, a second scaling factor and a second translation factor are generated through a second multilayer perceptron. A channel-level affine transformation is then performed on the corresponding 3D shape feature vector based on the second scaling factor and the second translation factor. The first multilayer perceptron and the second multilayer perceptron have the same structure.

8. The surface defect detection method for 3C structural components based on semantic memory networks according to claim 7, characterized in that, When adaptively fusing the enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional topography feature map to obtain a multi-scale cross-modal fused feature map, the following steps are included: The enhanced multi-scale two-dimensional texture feature map and the enhanced multi-scale three-dimensional shape feature map are concatenated along the channel dimension, and then channel compression is performed through a 1×1 convolutional layer to obtain a multi-scale cross-modal fusion feature map.

9. The method for detecting surface defects of 3C structural components based on semantic memory networks according to any one of claims 1-8, characterized in that, It also includes the following steps: Based on the final defect detection image, it is determined whether there are defects on the surface of the 3C structural component. When it is determined that there are defects on the surface of the 3C structural component, a rejection instruction is output to the downstream sorting mechanism.

10. The method for detecting surface defects of 3C structural components based on semantic memory networks according to claim 9, characterized in that, When determining whether there are defects on the surface of the 3C structural component based on the final defect detection image, the following steps are included: The total area containing defective pixels is obtained, and based on the total area, it is determined whether there are defects on the surface of the 3C structural component; wherein... When the total area is greater than a preset area threshold, it is determined that there is a defect on the surface of the 3C structural component; When the total area is less than or equal to a preset area threshold, it is determined that there are no defects on the surface of the 3C structural component.

Images