Appearance defect detection method and device based on multiple view angles, equipment and storage medium

Abstract

The invention relates to an appearance defect detection method, device and equipment based on multiple visual angles and a storage medium, the appearance defect detection method, device and equipment are applied to a control module of a multi-visual-angle detection system, the multi-visual-angle detection system further comprises a detection platform, multi-light-source illumination equipment and an image acquisition unit, and a to-be-detected object is placed on the detection platform; the method comprises the following steps: sequentially illuminating a to-be-detected object by adopting different light sources through multi-light-source illuminating equipment, and acquiring images of the to-be-detected object at different visual angles through an image acquisition unit in each illuminating process to obtain a first images; preprocessing the a first images to obtain a second images; and performing defect detection on the a second images through a preset multi-view apparent defect detection model to obtain a target apparent defect detection result of the to-be-detected object. By adopting the embodiment of the invention, the accuracy of appearance defect detection can be improved.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a method, apparatus, device and storage medium for detecting appearance defects based on multiple perspectives. Background Technology

[0002] With the rapid development of the electronics manufacturing industry and automated testing technology, product appearance defect detection has become a key link in production quality control.

[0003] Currently, most deep learning-based methods for detecting appearance defects employ single-light source imaging schemes, using convolutional neural networks to achieve automatic defect identification. However, these methods still suffer from problems such as insufficient illumination coverage (failing to reveal different defect features on complex surfaces) and unstable detection results (accuracy decreases due to changes in angle and illumination), resulting in low accuracy in appearance defect detection.

[0004] Therefore, improving the accuracy of appearance defect detection has become an urgent problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, device, and storage medium for detecting appearance defects based on multiple perspectives to address the aforementioned technical problems, thereby improving the accuracy of appearance defect detection.

[0006] Firstly, this application provides a multi-view appearance defect detection method, applied to the control module of a multi-view detection system. The multi-view detection system further includes: a detection platform, a multi-source illumination device, and an image acquisition unit. The object to be detected is placed on the detection platform. The method includes: The object to be tested is illuminated sequentially by different light sources using a multi-light source illumination device. During each illumination process, the image acquisition unit acquires images of the object to be tested from different perspectives, resulting in a first image; where a is an integer greater than 1. Preprocess a first images to obtain a second images; By using a pre-defined multi-view appearance defect detection model, defect detection is performed on a second image to obtain the target appearance defect detection result of the object to be detected.

[0007] Secondly, this application provides a multi-view appearance defect detection device, applied to the control module of a multi-view detection system. The multi-view detection system further includes: a detection platform, a multi-source illumination device, and an image acquisition unit. The object to be detected is placed on the detection platform. The device includes: The acquisition module is used to illuminate the object to be detected sequentially using different light sources through a multi-light source lighting device. During each illumination process, the image acquisition unit acquires images of the object to be detected from different perspectives, resulting in a first image; where a is an integer greater than 1. The preprocessing module is used to preprocess a first images to obtain a second images; The defect detection module is used to perform defect detection on a second image using a preset multi-view appearance defect detection model, and obtain the target appearance defect detection result of the object to be detected.

[0008] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method described above.

[0009] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0010] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described above.

[0011] The aforementioned multi-view appearance defect detection method first highlights different spectral or morphological features of defects using different light sources (e.g., coaxial light reveals scratches, and strip light reveals stains). Then, under each light source condition, images of the object under inspection are acquired from different perspectives to obtain a first image. This solves the problem that defect features are easily obscured or weakened under a single perspective or light source, providing a more comprehensive image data foundation for detection. Then, by using a multi-view appearance defect detection model specifically adapted to multi-dimensional image (i.e., a first image) input, it can effectively mine and integrate defect features from multiple light sources and multiple perspective images. Compared with traditional methods or single-input models, it can more accurately identify various defects, thereby improving the accuracy of appearance defect detection. Attached Figure Description

[0012] Figure 1 An application environment diagram of a multi-view appearance defect detection method provided in this application embodiment; Figure 2 An internal structure diagram of a multi-view detection system provided in an embodiment of this application; Figure 3 An internal structure diagram of a control module provided in an embodiment of this application; Figure 4 A flowchart illustrating a multi-view appearance defect detection method provided in this application embodiment; Figure 5 A flowchart illustrating a multi-view appearance defect detection method provided in this application embodiment; Figure 6A structural block diagram of a multi-view appearance defect detection device provided in this application embodiment; Figure 7 An internal structural diagram of a computer device provided in an embodiment of this application; Figure 8 An internal structural diagram of another computer device provided in an embodiment of this application; Figure 9 This is an internal structural diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0014] First, let me explain some technical terms or phrases used in this application: Softmax layer: Normalization layer. Its core function is to transform the raw scores (logits) output by the model into probability values ​​between 0 and 1, and the sum of the probabilities of all classes is 1.

[0015] Defect probability map: A probability distribution matrix in units of image pixels. Each pixel position corresponds to a set of values, representing the probability that the position belongs to different defect categories (e.g., "scratch probability 0.95, stain probability 0.03, no defect probability 0.02").

[0016] Model quantization is the process of converting high-precision floating-point parameters in a neural network into low-precision representations. Its main purpose is to reduce model size, improve computational efficiency, and reduce energy consumption. Common methods include weight quantization, activation value quantization, and dynamic quantization.

[0017] TensorRT framework: Tensor Real-Time is a deep learning inference acceleration framework optimized for GPUs. It can improve model inference speed through layer fusion, quantization and other methods, and is suitable for industrial real-time detection scenarios.

[0018] ".onnx": This stands for Open Neural Network Exchange, an open neural network exchange format. It is a universal model format between different deep learning frameworks, facilitating cross-framework model deployment.

[0019] NetworkDefinitionAPI: One of the core interfaces of the TensorRT framework, used to parse ".onnx" format model files and transform the models into an internal network structure that can be optimized by the TensorRT framework.

[0020] IBuilderConfig.set_flag(BuilderFlag.FP16): Code instruction in the TensorRT framework to enable FP16 half-precision mode: By setting the flag in BuilderConfig, the TensorRT framework enables FP16 quantization when building optimized models.

[0021] ReLU: Modified Linear Unit, a commonly used activation function that introduces non-linearity into the model and enhances its feature representation capabilities.

[0022] Please see Figure 1 , Figure 1 This diagram illustrates an application environment for a multi-view appearance defect detection method provided in this application embodiment. The terminal 102 communicates with the server 104 via a communication network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104 or located in the cloud or on other network servers. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0023] It should be noted that terminal 102 or server 104 can execute any of the implementation methods described in the multi-view appearance defect detection method provided in the embodiments of this application, and will not be repeated here. The computer device described in the embodiments of this application may include at least one of terminal 102 or server 104.

[0024] Please see Figure 2 , Figure 2 This application provides an embodiment of an internal structure diagram of a multi-view detection system. The multi-view detection system includes: a control module, a detection platform, a multi-source illumination device, and an image acquisition unit, wherein: The inspection platform is used to place or fix the object to be inspected (e.g., the casing of a TV stick), providing a stable support for the inspection process. The inspection platform can be configured as a movable platform or a conveyor belt to adapt to online inspection scenarios on production lines, enabling continuous flow and automated inspection of the object. Multiple light source illumination devices and image acquisition units can be sequentially installed on top of the inspection platform, forming a vertical inspection architecture of "platform-light source-acquisition unit".

[0025] Multi-source lighting equipment is used to highlight various defect features on the surface of an object under inspection by combining different types of light sources, solving the problem that defect features are easily obscured or weakened under a single light source. Multi-source lighting equipment can be installed above or around the inspection platform, arranged around the object under inspection to ensure that the inspection area is fully illuminated.

[0026] The image acquisition unit is used to acquire clear images of the object to be inspected under different light sources and different viewing angles, providing a high-quality data foundation for subsequent defect detection.

[0027] The control module, as the central control unit of the system, coordinates the operation of the detection platform, multi-source lighting equipment, and image acquisition unit to achieve automation and intelligence in the detection process.

[0028] Please see Figure 3 , Figure 3 This is an internal structural diagram of a control module provided in an embodiment of this application; it can be seen that the control module may include: an image receiving and preprocessing unit, a deep learning unit, and a result display unit, wherein: The image receiving and preprocessing unit is used to receive multi-source and multi-view images acquired by the image acquisition unit in real time, ensuring the stability and integrity of data transmission. Then, it can also perform operations such as denoising, distortion correction, and brightness equalization on the received raw images to obtain preprocessed images. Specifically, denoising eliminates image noise interference, distortion correction corrects camera imaging deviations, and brightness equalization unifies the image brightness benchmark under different light sources, effectively improving the accuracy of subsequent model inference.

[0029] The deep learning unit is equipped with a pre-defined multi-view appearance defect detection model. This model is used to mine defect features in pre-processed images, enabling accurate defect identification. Furthermore, technical optimizations are used to improve inference speed and adapt to the real-time requirements of online detection.

[0030] In a specific embodiment, the core architecture and functions of the multi-view appearance defect detection model are as follows: Model structure: It adopts a deep convolutional neural network architecture and designs a pattern in which a single input channel corresponds to the feature response of a single light source. It is paired with a channel weighting layer, which can dynamically assign weights to different light source inputs. The weights are learned autonomously during the training phase and can adaptively highlight the defect features of effective light sources and weaken invalid interference. Defect probability generation: After the model output is normalized by the Softmax layer, a defect probability map of a single location is generated, which intuitively reflects the confidence level of each region belonging to a certain type of defect. Deployment and Acceleration: Based on the TensorRT framework, the model is first exported as a ".onnx" file, then parsed using TensorRT's NetworkDefinition API. Inference speed is improved through two core methods: Layer fusion: Combines common continuous operations such as Conv+BatchNorm+ReLU and Conv+Add+Activation into single / composite operators, reducing the number of memory switching and improving cache hit rate; Model quantization: Enable FP16 half-precision mode (configured via IBuilderConfig.set_flag(BuilderFlag.FP16)), which automatically calls the GPU's TensorCore unit to perform half-precision matrix multiplication, achieving an inference speed improvement of 1.5 to 2 times with almost no loss of precision.

[0031] The results display unit is used to convert the output of the multi-view appearance defect detection model into visual results that can be intuitively interpreted by humans, thus completing the output and presentation of the detection results. It should be noted that the results display unit may include a display screen to show the visual results. Additionally, the results display unit can also save the detection results locally or upload them to the detection server.

[0032] It should be explained that Conv represents convolution operation; BatchNorm represents batch normalization operation; Add represents addition operation; Activation represents activation operation; FP16 represents 16-bit floating-point data format; GPU represents graphics processing unit; TensorCore represents the computing unit in GPU specifically designed for low-precision matrix multiplication, which can significantly accelerate matrix operations in FP16 format (the core operation of deep learning models).

[0033] like Figure 4 As shown, this application provides a multi-view appearance defect detection method, applied to the control module of a multi-view detection system. The multi-view detection system further includes: a detection platform, a multi-source illumination device, and an image acquisition unit. The object to be detected is placed on the detection platform. It should be explained that the control module can be... Figure 1 Terminal 102 or server 104 in the middle. The method includes: S101. The object to be tested is illuminated sequentially by different light sources using a multi-light source lighting device. During each illumination process, the image acquisition unit acquires images of the object to be tested from different perspectives to obtain a first image; a is an integer greater than 1.

[0034] Among them, multi-source lighting equipment refers to a lighting system composed of a variety of different types of light sources that can be switched or worked in concert as needed. Its core function is to highlight various defect features on the surface of the object to be inspected (such as scratches, dents, stains, abnormal textures, etc.) through differentiated light illumination methods.

[0035] It should be explained that multi-source lighting equipment may include at least two of the following light sources: coaxial light source, strip light source, ring light source, area light source, point light source, etc.; the image acquisition unit may include at least one of the following: industrial camera, hyperspectral camera, infrared thermal imager, etc.; the control module may include industrial PC (IPC) or embedded processor.

[0036] Specifically, the object to be inspected (e.g., the casing of a TV stick) can be placed on the inspection platform to ensure its stable position. A multi-light source lighting device and an image acquisition unit are installed sequentially above the inspection platform. The multi-light source lighting device can be arranged around the object to be inspected, and the image acquisition unit is fixed in the center of the multi-light source lighting device, vertically aligned with the object to be inspected to ensure consistent shooting angle. If online inspection is required, the inspection platform can be switched to a movable platform or conveyor belt mode, and the transmission speed and dwell time of the object to be inspected can be preset.

[0037] Next, the control module can configure the brightness parameters of each light source in the multi-light source lighting device, set the lighting sequence of the light sources to ensure that only one light source is lit at a time to avoid light interference. At the same time, it can also set a uniform shooting resolution for the image acquisition unit and enable the trigger-type shooting mode. It establishes synchronous linkage between the multi-light source lighting device and the image acquisition unit to ensure that the image acquisition action and the light source lighting status are strictly synchronized. The number of shooting angles is determined according to the detection requirements (e.g., angle 1, angle 2, angle 3), and each angle corresponds to a fixed shooting position of the image acquisition unit to ensure that the spatial position of multiple shots under the same angle is without deviation. Then, the control module can control the light sources in the multi-light source lighting device to light up sequentially according to the above lighting sequence. Each time a light source is lit, the image acquisition unit is controlled to capture images from different angles. For example, one image can be captured from one angle. In this way, a first image can be obtained, where the value of a is equal to the number of shooting angles multiplied by the number of light sources.

[0038] To illustrate, suppose a multi-source lighting device includes a coaxial light source (2-1), a strip light source (2-2), and a ring light source (2-3). The image acquisition unit is an industrial camera with two viewing angles: viewing angle 1 and viewing angle 2. Ultimately, six first images will be acquired. The specific acquisition process is as follows: Set the lighting sequence for the three light sources (sequential lighting, single light source operation, no light overlap), and uniformly adjust the brightness to the parameters suitable for detection (such as avoiding reflections or excessively low light); at the same time, a uniform shooting resolution can be set for the industrial camera (e.g., 1024×1024 or 2048×1536); for example, the lighting sequence can be: coaxial light source (2-1), bar light source (2-2), and ring light source (2-3). Predetermine the shooting positions of viewpoint 1 and viewpoint 2, and adjust the camera angles: viewpoint 1 can be set directly above the object to be detected (vertical shooting), and viewpoint 2 can be set at a 45° angle to the top (side shooting). After calibration, fix the camera parameters of the two viewpoints to ensure that the spatial reference of the images taken under the same viewpoint is consistent.

[0039] Next, the three light sources can be controlled to illuminate sequentially according to the lighting sequence, and two perspectives can be taken under each light source: Step 1: Light up the coaxial light source (2-1) and capture images from angle 1 and angle 2. The control module sends a command to the multi-light source lighting device to illuminate only the coaxial light source (2-1), while keeping the other two light sources off. The coaxial light source illuminates the object under inspection perpendicularly, highlighting its surface scratches and dents. After the light stabilizes (approximately 0.5 seconds to avoid light fluctuations affecting imaging), the control module receives the light source stabilization signal and triggers the industrial camera to switch to viewing angle 1 to capture one raw image, which is temporarily stored as the raw data for "viewing angle 1 - coaxial light source". After the viewing angle 1 image is captured, the industrial camera does not need to move the light source; it simply switches to the preset viewing angle 2 position and triggers the capture again to obtain the second raw image, which is temporarily stored as the raw data for "viewing angle 2 - coaxial light source". A total of two images are acquired in this stage, corresponding to the two viewing angles of the coaxial light source (2-1).

[0040] Step 2: Switch to bar light source (2-2) and take pictures from angle 1 and angle 2. The control module shuts down the coaxial light source (2-1) and illuminates the bar light source (2-2) according to a preset timing sequence, while keeping the other light sources off. The bar light source illuminates the object under inspection at a 45° angle to highlight the differences in its surface texture. Once the light stabilizes, the control module triggers the industrial camera, first taking a picture at viewpoint 1 to obtain the third raw image, which is temporarily stored as the raw data for "viewpoint 1 - bar light source". Then, the camera switches to viewpoint 2 and takes another picture to obtain the fourth raw image, which is temporarily stored as the raw data for "viewpoint 2 - bar light source". A total of two images are acquired in this stage, corresponding to the two viewpoints of the bar light source (2-2).

[0041] Step 3: Switch to ring light source (2-3) and take pictures from angle 1 and angle 2. The control module shuts down the bar light source (2-2) and illuminates the ring light source (2-3) according to a preset timing sequence, while keeping the other light sources off. The ring light source evenly illuminates the object under test, suppressing shadows during imaging, until the light stabilizes. The control module then triggers the industrial camera, first taking a picture at viewpoint 1 to obtain the 5th raw image, which is temporarily stored as the raw data for "viewpoint 1 - ring light source". Subsequently, the camera switches to viewpoint 2 and takes another picture to obtain the 6th raw image, which is temporarily stored as the raw data for "viewpoint 2 - ring light source". A total of 2 images are acquired in this stage, corresponding to the two viewpoints of light source 2-3.

[0042] III. Data Collection and Summarization After confirming that "multi-view shooting" has been completed for all three light sources, all original images are compiled, resulting in a total of 6 original images. These 6 original images are also the first image 'a' mentioned above. Then, they can be organized according to a unified naming convention as follows: V1_L1.png (view 1 - coaxial), V2_L1.png (view 2 - coaxial), V1_L2.png (view 1 - bar), V2_L2.png (view 2 - bar), V1_L3.png (view 1 - ring), V2_L3.png (view 2 - ring).

[0043] In this way, different light sources have different illumination methods and effects. Illuminating them in sequence can highlight different types of defects such as scratches, texture abnormalities, and dents, avoiding the problem of some defect features being covered or weakened under a single light source. By acquiring multi-view images during the illumination process of each light source, different positions of the object to be inspected can be covered, solving the problem of defects being blocked under a single view (e.g., workpiece corners and grooves), and ensuring that hidden defects can also be captured.

[0044] S102. Preprocess a first images to obtain a second images.

[0045] Preprocessing may include at least one of the following: noise reduction, distortion correction, brightness equalization, etc.

[0046] Specifically, for each first image, preprocessing can be performed. For example, preprocessing can include denoising. The type of noise contained in the first image can be determined first, and the corresponding denoising method can be selected to denoise the image according to the type of noise. For example, median filtering can be used for salt-and-pepper noise, and bilateral filtering can be used for high-frequency noise.

[0047] S103. Defect detection is performed on a second image using a preset multi-view appearance defect detection model to obtain the target appearance defect detection result of the object to be detected.

[0048] In some embodiments, please refer to Figure 5 , Figure 5The flowchart of a multi-view appearance defect detection method provided for embodiments of this application is as follows: Step 1: Begin The trigger node for process initiation marks the start of the appearance defect detection task.

[0049] Step 2: Image acquisition from multiple perspectives and multiple light sources Viewing angle and light source configuration: Three independent viewing angles are set (viewing angle 1, viewing angle 2, and viewing angle 3). At the same time, it is also equipped with two different types of light sources: "coaxial light" and "ring light" (different light sources can highlight different types of defects; for example, coaxial light is suitable for detecting scratches on a plane, while ring light is suitable for detecting dents).

[0050] Image generation logic: Each viewpoint (3 in total) will acquire 1 image under 2 light sources, resulting in 3×2=6 images (i.e. "6 first images"), which cover the visual information of the object to be detected under different angles and lighting conditions, avoiding the omission of defects caused by a single viewpoint / light source.

[0051] Step 3: Input 6 first images The six first images collected will be used as input data for the subsequent detection model.

[0052] Step 4: Processing the multi-view appearance defect detection model Inputting six images into the "multi-view appearance defect detection model" will allow the model to integrate image features from different perspectives and under different light sources, enabling more comprehensive defect feature extraction and discrimination.

[0053] Step 5: Output the target appearance defect detection results After the model is processed, the final detection results are output (including information such as the type, location, and number of defects).

[0054] Step 6: End This marks the completion of the appearance defect inspection process.

[0055] In some embodiments, the multi-view appearance defect detection model includes: a channel merging weighted layer, a convolutional layer, a cross-resolution fusion layer, a feature aggregation layer, a global average pooling layer, a fully connected layer, and a softmax layer; the model performs defect detection on *a* second images using a preset multi-view appearance defect detection model to obtain the target appearance defect detection result of the object to be detected, including: A1. Perform tensor processing on a second image through a channel merging weighting layer to obtain the first tensor; A2. The first tensor is subjected to progressive feature extraction through convolutional layers to obtain b feature maps; the resolutions of the b feature maps are all different; b is a positive integer; A3. The b feature maps are fused through a cross-resolution fusion layer to obtain a fused feature map; A4. The first tensor and the fused feature map are aggregated through the feature aggregation layer to obtain the target aggregated feature; A5. The target aggregated features are subjected to global average pooling through a global average pooling layer to obtain the target vector; A6. Process the target vector through a fully connected layer to obtain the classification probability distribution corresponding to the preset number of defect categories; A7. Normalize the classification probability distribution using the Softmax layer to obtain the target probability distribution; A8. Determine the target defect category label information corresponding to the target probability distribution; A9. Based on the target defect category label information and a second image, determine the target appearance defect detection result of the object to be detected.

[0056] Among them, the channel merging weighting layer is a multi-source feature integration layer used to perform channel dimension merging and adaptive weighting operations on feature maps corresponding to different input images (e.g., images acquired from multiple perspectives and multiple light sources); the cross-resolution fusion layer is a feature scale adaptation layer used to perform resolution alignment and feature fusion operations on feature maps with different spatial resolutions; the preset number of defect categories can be preset in advance or defaulted, which represents the number of predefined defect types. For example, the preset number of defect categories can be 3: no defects, scratches, stains, or it can also be 5: no defects, scratches, dents, texture abnormalities, stains, etc.

[0057] Specifically, a second image channels can be input into a weighted layer to obtain a first tensor. Then, a convolutional layer can be used to extract features from the first tensor stepwise to obtain b feature maps. Specifically, the convolutional layer can include multiple convolutions. The first tensor is input into the convolutional layer. The first convolution in the multiple convolutions first captures the low-order features of the first tensor (e.g., the edges of defects, pixel grayscale changes). Subsequent convolutional layers extract more complex high-order features based on the low-order features (e.g., the complete outline of scratches, the shape of stains). At the same time, downsampling operations can be inserted between different convolutions. Each downsampling operation halves the resolution of the feature map (e.g., 1024×1024 → 512×512 → 256×256). Downsampling reduces the computational cost of the model and expands the receptive field (covering a larger area of ​​the image), adapting to the real-time requirements of industrial inspection. The final output is b feature maps, each corresponding to a different level of convolution or downsampling, resulting in different resolutions. For example, assuming b equals 3, the output of the convolutional layer can be feature maps with resolutions of 512×512, 256×256, and 128×128.

[0058] In some embodiments, a single convolutional layer in a convolutional layer has the following structure: Conv3×3, stride=2, padding=1, out_channels=64→BN→ReLU →Conv3×3,stride=1,out_channels=64→BN→ReLU The first stage is as follows: Conv3×3, stride=2, padding=1, out_channels=64 → BN → ReLU Conv3×3: This means scanning the input tensor (i.e., the first tensor) with a 3×3 convolution kernel to capture basic features; stride=2 (stride size 2): Implements downsampling, halves the feature map resolution, and reduces computational cost; padding=1 (padding 1): to prevent loss of edge information after convolution; `out_channels=64`: Outputs 64 feature maps (each feature map corresponds to a basic defect feature). Here, "64" is the number of channels in a single convolutional layer. It should be explained that these 64 feature maps are the "number of channels at a single resolution level," and the 3 feature maps output by the convolutional layer are "simplified names / fusion results of 3 different resolution levels." First, 3 convolutional layers output 3 resolution levels (64 images per layer), and then through "simplified description" or "channel fusion," the final output of the convolutional layer is "3 feature maps of different resolutions." Batch Normalization (BN): Stabilizes feature distribution and accelerates training; ReLU (activation function): introduces nonlinearity to capture complex defect features.

[0059] Second stage: Conv3×3, stride=1, out_channels=64 → BN → ReLU Change the step size to 1, do not downsample, and keep the feature map resolution unchanged; Based on the coarse feature extraction in the first stage, higher-order defect features (such as complete scratch outlines) are extracted in a more refined manner. The output channels remain at 64, maintaining consistent feature dimensions and ensuring the continuity of feature propagation.

[0060] A convolutional layer can contain three convolutional layers, that is, the above convolutional structure is stacked three times, and each group performs progressive downsampling on the input resolution, thereby outputting three feature maps with different resolutions.

[0061] Then, the b feature maps can be input into the cross-resolution fusion layer for fusion to obtain the fused feature map. Next, the first tensor and the fused feature map can be input into the feature aggregation layer for aggregation to obtain the target aggregated feature. Then, the target aggregated feature is input into the global average pooling layer for global average pooling to obtain the target vector. Specifically, the target aggregated feature F∈R^{C×H×W} is subjected to GlobalAveragePool (i.e., global average pooling) on ​​the target aggregated feature F to obtain a C-dimensional vector, which is the target vector. Here, F∈R^{C×H×W} is read as "F belongs to a C×H×W-dimensional tensor in the real number field", F represents the target aggregated feature, R represents the real number field, H represents the height (representing the number of pixels in the vertical direction of a single feature map), and W represents the width (representing the number of pixels in the horizontal direction of a single feature map).

[0062] Furthermore, the target vector can be input into the fully connected layer. Each dimension of the target vector corresponds to a global response of a defect feature (e.g., the first dimension corresponds to "scratch edge feature", the second dimension corresponds to "stain texture feature", etc.). The fully connected layer has a built-in weight matrix (size "C × preset number of defect categories"). The C-dimensional target vector is multiplied by the weight matrix to map the C-dimensional feature vector to the original score of the "preset number of defect categories" dimension. Thus, the classification probability distribution corresponding to the preset number of defect categories is obtained. For example, assuming the target vector is 64-dimensional and the preset number of defect categories is 3 (no defect, scratch, stain), the weight matrix is ​​64×3. After the operation, a set of original scores (also called "logits") is output, for example, [2.5, 8.1, 0.3], which is the classification probability distribution.

[0063] Since the values ​​of logits can be positive or negative and have no fixed range, the magnitude of the logits values ​​can only reflect the "degree of tendency to belong to a certain class", but cannot be directly interpreted as probability (for example, a score of 8 does not represent an 80% probability). Furthermore, the sum of scores for each class is not equal to 1, which does not conform to the basic definition of probability. Therefore, it is necessary to input the classification probability distribution into the Softmax layer and normalize it to obtain the target probability distribution.

[0064] It should be explained that the output of the fully connected layer is the image-level classification result (i.e., the classification probability distribution): the fully connected layer outputs a set of raw scores → Softmax layer → the class probability distribution of the entire image (i.e., the target probability distribution); in some embodiments, pixel-level defect detection can also be performed. The multi-view appearance defect detection model can bypass the global average pooling layer and the fully connected layer, and instead directly perform channel mapping (e.g., 1×1 convolution) on the feature map (dimension C×H×W) output by the convolutional layer to obtain pixel-level raw scores of dimension num_classes×H×W (each pixel corresponds to num_classes scores), where num_classes represents the preset number of defect categories; the Softmax layer performs normalization on each pixel in the channel dimension to obtain the pixel-level target probability distribution P (i.e., the defect probability map, which presents the confidence of each pixel belonging to each type of defect in the form of a "graph", intuitively reflecting the probability distribution of defects in the image space); where P∈R^{num_classes×H×W}.

[0065] Next, the target defect category label information corresponding to the target probability distribution can be determined; finally, the target appearance defect detection result of the object to be detected can be determined based on the target defect category label information and a second images.

[0066] Thus, on the one hand, the channel merging weighted layer integrates multi-view and multi-source image features, and the cross-resolution fusion layer and feature aggregation layer further fuse features of different resolutions, fully preserving the details and global information of defects and reducing missed detections; on the other hand, the global average pooling condenses features, and the fully connected layer and Softmax layer are used to achieve image-level accurate classification, balancing computational efficiency and classification accuracy; finally, the detection results are output by combining category labels and original images, which facilitates defect localization and visualization verification of detection results in industrial scenarios.

[0067] In some embodiments, tensor processing is performed on a second image through a channel merging weighting layer to obtain a first tensor, including: B1. Map each of the a second images to a tensor to obtain a tensors; B2. Concatenate the a tensors to obtain the concatenated tensor; B3. Determine the target number of channels corresponding to the splicing tensor; B4. Determine the same number of global scalar weights as the target number of channels, resulting in c global scalar weights; where c is the target number of channels. B5. Adjust the spliced ​​tensor according to c global scalar weights to obtain the first tensor.

[0068] Among them, c global scalar weights refer to a set of independent, numerical weight parameters introduced in the feature fusion process to differentiate and enhance or suppress the feature information of different channels in the spliced ​​tensor.

[0069] Specifically, for each of the a second images, a uniform format can be used to ensure that the pixel dimensions of each image are consistent. For example, the resolution can be normalized to a preset resolution (e.g., 1024×1024), and the color space can be converted to RGB. Then, the values ​​of all pixels can be extracted in the order of "channel → height → width". For example, each pixel of an RGB image contains three values ​​of 0 to 255: R, G, and B. The extracted pixel value array can be converted into a multidimensional tensor, and the value type can be uniformly set to floating point (e.g., 0 to 255 can be normalized to 0 to 1). In this way, a tensors can be obtained.

[0070] Next, the a tensors can be concatenated to obtain a concatenated tensor. Specifically, they can be concatenated along the "channel dimension" or, as needed, along the "batch dimension" to obtain a concatenated tensor. For example, assuming concatenation along the "channel dimension", if the dimension of a single tensor is C×H×W, the dimension of the concatenated tensor is (a×C)×H×W, that is, the channel number of the a tensors is superimposed, while keeping the H×W spatial dimension unchanged (for example, 6 tensors of 3×512×512 are concatenated to obtain a tensor of 18×512×512).

[0071] Furthermore, the target number of channels corresponding to the spliced ​​tensor can be determined. Specifically, the number of channels in a single image of the second images can be obtained to get the first number of channels. The target number of channels is obtained by multiplying the first number of channels by a. For example, assuming that the second images are all RGB images and the first number of channels is 3 (R, G, and B channels), a=6, 3×6=18, that is, the target number of channels is 18. Then, the same number of global scalar weights as the target number of channels can be determined to obtain c global scalar weights. Specifically, a trainable weight vector w∈R^{C} can be defined: this is a vector containing c elements (i.e., c global scalar weights). The training weight vector is a one-dimensional vector containing the initial values ​​of the global scalar weights, where the element w[x] corresponds to the weight of the x-th channel among the c global scalar weights; the starting index of w is 1; x is a natural number greater than 0 and less than or equal to c; the initial values ​​of the training weight vector are usually randomly initialized (e.g., normally distributed), and are updated through backpropagation during the training process of the training weight vector. The multi-view appearance defect detection model will autonomously adjust the weights of each channel (e.g., assign high weights to channels containing defect features and low weights to channels without defects or redundancy), thus obtaining the trained weight vector w, which is also the c global scalar weights.

[0072] Finally, the concatenated tensor can be adjusted based on c global scalar weights, denoted as X_in, as follows: X_weighted[x] = w[x] * X_in[x]; Where X_weighted[x] represents the feature data of the x-th channel in the first tensor; w[x] represents the x-th global scalar weight among c global scalar weights; X_in[x] represents the feature data of the x-th channel corresponding to w[x] in the concatenated tensor; thus, the first tensor can be obtained.

[0073] In this way, by stitching together a image tensors, information from multiple perspectives and multiple light sources can be integrated to enrich the dimensions of defect features. In addition, by configuring trainable weights for each channel, the model can autonomously strengthen effective defect channels and suppress redundant channels, thereby improving feature representation capabilities.

[0074] In some embodiments, b feature maps are fused through a cross-resolution fusion layer to obtain a fused feature map, including: C1. Align the resolutions of the b feature maps to obtain b first feature maps; C2. Align and merge the channels of the b first feature maps to obtain a fused feature map.

[0075] Specifically, we can first obtain the resolution of each of the b feature maps, resulting in b resolutions. Based on these b resolutions, we can determine the baseline resolution. For example, we can use the highest resolution among the b resolutions as the baseline resolution, or we can calculate the average resolution of these b resolutions and use the average resolution as the baseline resolution. Then, we can adjust the resolution of all b feature maps to the baseline resolution. Specifically, we can upsample (e.g., interpolation, transpose convolution) the feature maps with resolutions lower than the baseline resolution to the baseline size (i.e., the baseline resolution), and downsample (e.g., pooling) the feature maps with resolutions higher than the baseline resolution to the baseline size. Then, we can correct the feature deviation after sampling using 1×1 convolution (e.g., to compensate for the blurring caused by upsampling and the loss of information by downsampling) to ensure that the spatial dimensions are completely matched. Finally, we obtain a first feature map with b resolutions that are completely consistent, which preserves the details of high resolution while being compatible with the semantic information of low resolution.

[0076] Then, the channels of the b first feature maps can be aligned and fused to obtain a fused feature map. Specifically, a 1×1 convolution can be used to unify the number of channels in all the first feature maps to the same value (denoted as C_target), eliminating the channel dimension differences, thus obtaining b aligned first feature maps. C_target can be preset, or it can be the average value of the channels corresponding to the b first feature maps. Then, the b aligned first feature maps can be fused together according to a preset fusion method to obtain the fused feature map. The preset fusion method can include one of the following: weighted summation fusion method, channel concatenation plus convolution fusion method, attention mechanism fusion method, etc. For example, assuming the preset fusion method is the attention mechanism fusion method, the specific steps are as follows: Calculate channel attention weights for the aligned b first feature maps (e.g., generate an importance score for each channel through a global average pooling layer + a fully connected layer). The channels of each feature map are weighted using channel attention weights, and then summed and fused to obtain a fused feature map.

[0077] In this way, by integrating the advantages of features at different resolutions through the fusion process, high-resolution details improve the accuracy of small defect localization, while low-resolution semantics enhance the ability to distinguish defect categories. Combined with the dimensional consistency after alignment, the fused feature map has both details and semantics, which greatly reduces the rate of missed and false detections of small defects.

[0078] In some embodiments, the first tensor and the fused feature map are aggregated through a feature aggregation layer to obtain target aggregated features, including: D1. Determine the second feature map corresponding to the first tensor; D2. Adjust the resolution of the second feature map and the fused feature map to obtain the third feature map and the fourth feature map; the third feature map and the fourth feature map have the same resolution. D3. Concatenate the third and fourth feature maps along the channel dimension to obtain the concatenated feature map; D4. Perform dimensionality reduction and smoothing on the spliced ​​feature map to obtain the target aggregated feature.

[0079] Specifically, the first tensor is a channel-weighted tensor, which already possesses both spatial and channel dimensions. Therefore, "determining the second feature map" essentially involves performing basic formatting or dimension adjustments on the first tensor to adapt it to the subsequent aggregation process, as follows: Verify the validity of the dimensions of the first tensor: confirm that it is a three-dimensional tensor (channels × height × width), with no missing dimensions or format errors; The first tensor can be directly used as the second feature map without additional feature transformation. The first tensor has already undergone channel weighting and retains the effective features of the multi-source images, so it can be directly used as one of the input features for aggregation. Then, the resolution of the second feature map and the fused feature map can be adjusted to obtain the third and fourth feature maps. Specifically, the first resolution corresponding to the second feature map and the second resolution corresponding to the fused feature map can be determined, and the larger of the first and second resolutions can be used as the target resolution. Then, the resolutions of the second feature map and the fused feature map can be adjusted to the target resolution to obtain the third and fourth feature maps. Alternatively, a preset resolution can be obtained, and the resolutions of the second feature map and the fused feature map can be adjusted to the preset resolution to obtain the third and fourth feature maps.

[0080] Next, the third and fourth feature maps can be concatenated along the channel dimension to obtain a concatenated feature map. For example, assuming the third feature map is C1×H1×W1 and the fourth feature map is C2×H1×W1, concatenating these two feature maps yields a concatenated feature map (C1+C2)×H1×W1. Finally, the concatenated feature map can be dimensionality reduced and smoothed to obtain the target aggregated feature. Specifically, a 1×1 convolution kernel can be used to reduce the dimensionality of the concatenated feature map, compressing the number of channels from (C1+C2) to a preset number of channels while maintaining the resolution at (H1×W1). Then, a 3×3 convolution kernel (with padding of 1 to ensure the resolution remains unchanged) can be used to eliminate feature noise caused by concatenation and dimensionality reduction, and strengthen spatial contextual relevance, thereby obtaining a smoothed feature map, which is the target aggregated feature. The preset number of channels is a value less than (C1+C2), and the preset number of channels can be preset in advance or left as a default value.

[0081] It should be explained that the feature aggregation layer can include: Concat (sew-together layer), 1×1Conv (1×1 convolutional kernel), and 3×3Conv (3×3 convolutional kernel); Concat is used to perform the sewing operation; 1×1Conv is used to perform the dimensionality reduction operation; and 3×3Conv is used to perform the smoothing operation.

[0082] In this way, by adjusting the resolution to unify the dimension of the feature space, we can ensure that the pixel positions correspond accurately when the channels are stitched together, thus avoiding fusion failure.

[0083] In addition, by performing dimensionality reduction and smoothing on the spliced ​​feature map, redundant channels are eliminated and channel interaction is realized. Smoothing eliminates fusion noise and enhances feature continuity, so that the target aggregated features have both information richness and discrimination effectiveness, thus improving the detection accuracy of small defects.

[0084] In some embodiments, the target appearance defect detection result of the object to be detected is determined based on the target defect category label information and a second image, including: E1. Determine the target mask corresponding to the target defect category label information; E2. Label each of the a second images according to the target mask to obtain a labeled images; E3. Based on image a, determine the detection results of the target appearance defects of the object to be inspected.

[0085] The target defect category label information may include: defect category (e.g., fixed categories such as scratches, oil stains, cracks, dirt, etc.) and pixel-level location / region information corresponding to each defect category.

[0086] Specifically, you can first create a blank mask matrix (mask) with the exact same size as the second image (a), and set its initial value to the background identifier (for example, the initial value can be 0, representing no defects); then, assign values ​​to the mask as follows: Based on the pixel-level location / region information in the target defect category label information, replace the pixel value of the corresponding defect location in the mask with the exclusive identifier of the defect category. For example, 1 represents scratch, 2 represents dent, 3 represents texture abnormality, and 4 represents stain. If multiple types of defects exist at the same location (which is relatively rare in engineering, and if they do exist, assign values ​​according to priority), or if there are multiple defect areas of the same category (for example, 20 scratch areas), then mark the corresponding pixel positions one by one; Thus, the target mask can be obtained. For example, assuming the size of the second image is 2×3 (H=2 rows, W=3 columns), the target mask can be: [ [0,1,2], [1,0,1] ] Among them, type 1 defects appear at three pixel positions (0,1), (1,0), and (1,2) (multiple defects of the same type), while type 2 defects appear only at one position (0,2); the two pixel positions (0,0) and (1,1) are defect-free areas; it can be seen that the target mask completely maps the pixel-level positions of all defect types in the image.

[0087] Next, the a second images can be labeled according to the target mask to obtain a labeled images. Specifically, for each second image, the defect pixels on the image are determined according to the target mask to obtain multiple defect pixels. The corresponding colors are superimposed on the multiple defect pixels according to the preset color mapping rules (preserving the original image texture and only highlighting the defects). Contour labeling / text labeling can also be selected (as needed). In this way, a labeled images can be obtained.

[0088] The preset color mapping rules can be preset in advance or defaulted. For example, the preset color mapping rules can be: 0 (no defects) corresponds to no marking, 1 (scratches) corresponds to red, 2 (dents) corresponds to yellow, 3 (texture abnormalities) corresponds to blue, 4 (stains) corresponds to green, and so on.

[0089] Finally, one labeled image can be used as the result of the detection of the target appearance defects of the object to be detected.

[0090] In this way, by accurately mapping the location and type of defects at the pixel level through target masks, the spatial distribution of defects can be intuitively presented based on the mask-annotated image, thus solving the problem that pure numerical results are not intuitive. In addition, a single mask is used to batch annotate a images of the same size, avoiding repeated mask generation and improving the efficiency of visualization of detection results.

[0091] In some embodiments, determining the target defect category label information corresponding to the target probability distribution includes: F1. Determine the reference output tensor corresponding to the target probability distribution; F2. Determine the reference defect label map corresponding to the reference output tensor; F3. Determine the target defect category label information based on the reference defect label image.

[0092] The target probability distribution can be a pixel-level target probability distribution P, P∈R^{num_classes×H×W}, where num_classes represents the number of preset defect categories; H×W is the image size; each position (c,h,w) represents the confidence probability of the c-th defect at pixel (h,w); the confidence probability ranges from 0 to 1.

[0093] Specifically, for each pixel position (h, w) in the target probability distribution, the class index corresponding to the maximum probability in the num_classes dimension probability vector is taken (i.e., argmax operation), resulting in an index matrix of dimension H×W; at the same time, the probability value corresponding to the maximum value is retained to generate a confidence matrix of the same size as the index matrix; the reference output tensor is a combined tensor of "index matrix + confidence matrix" (the dimension can be represented as 2×H×W), where the first channel is the class index and the second channel is the corresponding confidence probability.

[0094] For example, assuming num_classes is 5, the probability vector of num_classes dimension corresponding to pixel position (1,2) is (0.05, 0.80, 0.10, 0.03, 0.02). Taking argmax of this probability vector, we find the index corresponding to the maximum value of 0.80: 1 (the corresponding defect category is scratch). Therefore, the value at position (1,2) in the index matrix is ​​1, and the value at position (1,2) in the confidence matrix is ​​0.80 (i.e., confidence).

[0095] Then, the reference defect label map corresponding to the reference output tensor can be determined. Specifically, the index matrix in the reference output tensor can be directly converted into a visualized reference defect label map (essentially an H×W pixel-level label matrix). Then, the target defect category label information can be determined based on the reference defect label map. Specifically, the reference defect label map can be filtered by confidence to remove low-confidence categories. Specifically, based on the confidence matrix in the reference output tensor, each pixel of the reference defect label map can be traversed. If the category index of a pixel is not equal to 0 (defective), but its corresponding confidence is less than a preset confidence threshold (e.g., 0.5), its category index is reset to 0 (determined as no defect). Alternatively, if its corresponding confidence is greater than or equal to the preset confidence threshold, its category index will be retained. After filtering, a "high-confidence defect label map" is obtained, eliminating pixels that are misjudged due to low confidence. Then, connected component filtering is performed on the high-confidence defect label image to remove excessively small noise. Specifically, for the high-confidence defect label image, a connected component analysis algorithm is used to identify all connected components of the same category of defects, and the number of pixels in each connected component is calculated. If its area (i.e., the number of pixels) is less than a preset area threshold (e.g., 5 pixels), the category index of all pixels in the connected component is reset to 0; if its area is greater than or equal to the preset area threshold, the category index of all pixels in the connected component is retained. In this way, the filtered defect label image, i.e., the target defect category label information, can be obtained.

[0096] In some embodiments, the target probability distribution can be the category probability distribution of the entire image. The maximum probability in the target probability distribution can be directly obtained, and the maximum probability and its corresponding defect category can be used as the target defect category label information. For example, assuming the preset number of defect categories is 3 (no defect, scratch, stain), the category probability distribution of the entire image can be 0.25 (corresponding to no defect), 0.70 (corresponding to scratch), and 0.05 (corresponding to stain). Then the target defect category label information is: scratch category, category probability 0.7.

[0097] The aforementioned multi-view appearance defect detection method first highlights different spectral or morphological features of defects using different light sources (e.g., coaxial light reveals scratches, and strip light reveals stains). Then, under each light source condition, images of the object under inspection are acquired from different perspectives to obtain a first image. This solves the problem that defect features are easily obscured or weakened under a single perspective or light source, providing a more comprehensive image data foundation for detection. Then, by using a multi-view appearance defect detection model specifically adapted to multi-dimensional image (i.e., a first image) input, it can effectively mine and integrate defect features from multiple light sources and multiple perspective images. Compared with traditional methods or single-input models, it can more accurately identify various defects, thereby improving the accuracy of appearance defect detection.

[0098] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0099] Based on the same inventive concept, this application also provides a multi-view appearance defect detection device. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more multi-view appearance defect detection device embodiments provided below can be found in the limitations of the multi-view appearance defect detection method above, and will not be repeated here.

[0100] Please see Figure 6 , Figure 6 A structural block diagram of a multi-view appearance defect detection device provided in this application embodiment is included. This device is applied to the control module of a multi-view detection system. The multi-view detection system further includes: a detection platform, a multi-source illumination device, and an image acquisition unit. An object to be detected is placed on the detection platform. The multi-view appearance defect detection device 600 includes: The acquisition module 601 is used to illuminate the object to be detected sequentially using different light sources through a multi-light source lighting device. During each illumination process, the image acquisition unit acquires images of the object to be detected from different perspectives to obtain a first image; where a is an integer greater than 1. Preprocessing module 602 is used to preprocess a first images to obtain a second images; The defect detection module 603 is used to perform defect detection on a second image using a preset multi-view appearance defect detection model to obtain the target appearance defect detection result of the object to be detected.

[0101] In some embodiments, the multi-view appearance defect detection model includes: a channel merging weighted layer, a convolutional layer, a cross-resolution fusion layer, a feature aggregation layer, a global average pooling layer, a fully connected layer, and a Softmax layer; in terms of performing defect detection on a second image using the preset multi-view appearance defect detection model to obtain the target appearance defect detection result of the object to be detected, the defect detection module 603 is specifically used for: The first tensor is obtained by performing tensor processing on a second image through a channel merging weighting layer; The first tensor is subjected to progressive feature extraction through convolutional layers to obtain b feature maps; the resolutions of the b feature maps are all different; b is a positive integer. The b feature maps are fused by a cross-resolution fusion layer to obtain a fused feature map; The first tensor and the fused feature map are aggregated by the feature aggregation layer to obtain the target aggregated feature; The target aggregated features are subjected to global average pooling through a global average pooling layer to obtain the target vector; The target vector is processed by a fully connected layer to obtain a classification probability distribution corresponding to the preset number of defect categories; The classification probability distribution is normalized by using a Softmax layer to obtain the target probability distribution; Determine the target defect category label information corresponding to the target probability distribution; Based on the target defect category label information and a second image, determine the target appearance defect detection result of the object to be detected.

[0102] In some embodiments, in obtaining a first tensor by performing tensor processing on a second image through a channel merging weighting layer, the defect detection module 603 is specifically used for: Map each of the a second images to a tensor to obtain a tensors; Concatenate a tensors to obtain a concatenated tensor; Determine the target number of channels corresponding to the splicing tensor; Determine the same number of global scalar weights as the target number of channels, resulting in c global scalar weights; where c is the target number of channels. The concatenated tensor is adjusted according to c global scalar weights to obtain the first tensor.

[0103] In some embodiments, in fusing b feature maps through a cross-resolution fusion layer to obtain a fused feature map, the defect detection module 603 is specifically used for: Align the resolutions of the b feature maps to obtain b first feature maps; Align and merge the channels of the b first feature maps to obtain a fused feature map.

[0104] In some embodiments, in aggregating the first tensor and the fused feature map through a feature aggregation layer to obtain the target aggregated feature, the defect detection module 603 is specifically used for: Determine the second feature map corresponding to the first tensor; The resolution of the second feature map and the fused feature map is adjusted to obtain the third feature map and the fourth feature map; the third feature map and the fourth feature map have the same resolution. The third and fourth feature maps are concatenated along the channel dimension to obtain the concatenated feature map; The spliced ​​feature map is subjected to dimensionality reduction and smoothing to obtain the target aggregated feature.

[0105] In some embodiments, in determining the target appearance defect detection result of the object to be detected based on the target defect category label information and a second image, the defect detection module 603 is specifically used for: Determine the target mask corresponding to the target defect category label information; Based on the target mask, label each of the a second images to obtain a labeled images; Based on image a, determine the detection results of the target appearance defects of the object to be inspected.

[0106] In some embodiments, the defect detection module 603 is specifically used for determining the target defect category label information corresponding to the target probability distribution in the following ways: Determine the reference output tensor corresponding to the target probability distribution; Determine the reference defect label map corresponding to the reference output tensor; Based on the reference defect label map, determine the target defect category label information.

[0107] Each module in the aforementioned multi-view appearance defect detection device 600 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0108] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data related to the multi-view appearance defect detection method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps of the multi-view appearance defect detection method described above.

[0109] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the steps in the multi-view-based appearance defect detection method described above. The display unit of the computer device is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen; the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs or touchpads set on the casing of the computer device, or external keyboards, touchpads or mice, etc.

[0110] Those skilled in the art will understand that Figure 7 or Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0111] In some embodiments, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0112] In some embodiments, such as Figure 9 The diagram shows the internal structure of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described method embodiments.

[0113] In some embodiments, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting appearance defects based on multiple perspectives, characterized in that, A control module for a multi-view detection system, the multi-view detection system further comprising: a detection platform, a multi-source illumination device, and an image acquisition unit, wherein an object to be detected is placed on the detection platform, and the method includes: The multi-light source lighting device illuminates the object to be detected sequentially using different light sources. During each illumination process, the image acquisition unit acquires images of the object to be detected from different perspectives, resulting in a first image; where a is an integer greater than 1. Preprocess the a first images to obtain a second images; The a second images are subjected to defect detection using a preset multi-view appearance defect detection model to obtain the target appearance defect detection result of the object to be detected.

2. The method according to claim 1, characterized in that, The multi-view appearance defect detection model includes: channel merging weighted layer, convolutional layer, cross-resolution fusion layer, feature aggregation layer, global average pooling layer, fully connected layer, and Softmax layer; The step of performing defect detection on the a second images using a preset multi-view appearance defect detection model to obtain the target appearance defect detection result of the object to be detected includes: The first tensor is obtained by performing tensor processing on the a second images through the channel merging weighting layer. The first tensor is subjected to stepwise feature extraction through the convolutional layer to obtain b feature maps; the resolutions of the b feature maps are all different; and b is a positive integer. The b feature maps are fused through the cross-resolution fusion layer to obtain a fused feature map; The first tensor and the fused feature map are aggregated through the feature aggregation layer to obtain the target aggregated feature; The target aggregated features are subjected to global average pooling through the global average pooling layer to obtain the target vector; The target vector is processed by a fully connected layer to obtain a classification probability distribution corresponding to a preset number of defect categories; The classification probability distribution is normalized by the Softmax layer to obtain the target probability distribution; Determine the target defect category label information corresponding to the target probability distribution; Based on the target defect category label information and the a second image, the target appearance defect detection result of the object to be detected is determined.

3. The method according to claim 2, characterized in that, The step of performing tensor processing on the a second images through the channel merging weighting layer to obtain the first tensor includes: Each of the a second images is mapped to a tensor, resulting in a tensors; The a tensors are concatenated to obtain a concatenated tensor; Determine the target number of channels corresponding to the splicing tensor; Determine the same number of global scalar weights as the target number of channels, resulting in c global scalar weights; where c is the target number of channels. The spliced ​​tensor is adjusted according to the c global scalar weights to obtain the first tensor.

4. The method according to claim 2 or 3, characterized in that, The process of fusing the b feature maps through the cross-resolution fusion layer to obtain a fused feature map includes: Align the resolutions of the b feature maps to obtain b first feature maps; The channels of the b first feature maps are aligned and merged to obtain a merged feature map.

5. The method according to claim 2 or 3, characterized in that, The step of aggregating the first tensor and the fused feature map through the feature aggregation layer to obtain the target aggregated feature includes: Determine the second feature map corresponding to the first tensor; The resolution of the second feature map and the fused feature map is adjusted to obtain a third feature map and a fourth feature map; the third feature map and the fourth feature map have the same resolution. The third feature map and the fourth feature map are concatenated along the channel dimension to obtain a concatenated feature map; The spliced ​​feature map is subjected to dimensionality reduction and smoothing processing to obtain the target aggregated feature.

6. The method according to claim 2 or 3, characterized in that, The step of determining the target appearance defect detection result of the object to be detected based on the target defect category label information and the a second images includes: Determine the target mask corresponding to the target defect category label information; The a second images are labeled according to the target mask to obtain a labeled images; Based on the a labeled images, the detection result of the target appearance defect of the object to be detected is determined.

7. The method according to claim 2 or 3, characterized in that, The determination of the target defect category label information corresponding to the target probability distribution includes: Determine the reference output tensor corresponding to the target probability distribution; Determine the reference defect label map corresponding to the reference output tensor; Based on the reference defect label map, determine the target defect category label information.

8. A multi-view appearance defect detection device, characterized in that, A control module for a multi-view inspection system, the multi-view inspection system further comprising: an inspection platform, a multi-source illumination device, and an image acquisition unit, wherein the object to be inspected is placed on the inspection platform, and the device includes: The acquisition module is used to illuminate the object to be detected sequentially using different light sources through the multi-light source lighting device. During each illumination process, the image acquisition unit acquires images of the object to be detected from different perspectives to obtain a first images; where a is an integer greater than 1. The preprocessing module is used to preprocess the a first images to obtain a second images; The defect detection module is used to perform defect detection on the a second images using a preset multi-view appearance defect detection model, and obtain the target appearance defect detection result of the object to be detected.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images