Method and device for detecting surface cleanliness of substrate for PVD coating

Abstract

The present application discloses a method and device for detecting the surface cleanliness of a substrate for PVD coating, which collects the surface state image of the PVD coating substrate through a camera deployed at a key position of the PVD production line, and uses computer vision-based image analysis and coding technology to perform panoramic segmentation on the surface state image of the PVD coating substrate, and then extracts features from the segmented PVD coating substrate surface state mask representation image, and at the same time, performs grid division and grid feature extraction on the PVD coating substrate surface state image, so as to automatically generate a cleanliness label of the PVD coating substrate surface based on the fine-grained ablation multi-dimensional semantic coding representation between the PVD coating substrate surface state grid feature and the PVD coating substrate surface state mask feature, and judge whether to generate a warning signal accordingly. In this way, the cleanliness label can be generated instantly, and real-time monitoring in the continuous production process can be realized, thereby improving the accuracy of cleanliness detection.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection, and more specifically, to a method and device for detecting the surface cleanliness of a substrate used for PVD coating. Background Art

[0002] As the manufacturing industry continues to improve its requirements for product quality, PVD (physical vapor deposition) coating technology, as an advanced surface treatment technology, has been widely used in many industrial fields. However, the quality of PVD coating is directly related to the performance and life of the product, especially the cleanliness of the substrate surface has a crucial impact on the adhesion, uniformity and durability of the coating layer.

[0003] Traditional methods for detecting the surface cleanliness of substrates, such as chemical residue analysis, contact angle measurement, and surface energy spectrum analysis, are not only time-consuming and difficult to achieve real-time online monitoring, but are also limited to the acquisition of local information and cannot comprehensively evaluate the overall state of the substrate. For example, contact angle measurement relies on limited measurement points and cannot capture changes in the entire surface. In addition, these methods mainly focus on the chemical composition and macroscopic features of the surface, ignoring the subtle changes in the microstructure and deep semantic information, making it difficult to detect tiny contaminants and identify complex surface morphology patterns.

[0004] Therefore, an optimized substrate surface cleanliness detection scheme for PVD coating is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and device for detecting the surface cleanliness of a PVD-coated substrate, which collects the surface state image of the PVD-coated substrate through a camera deployed at a key position of the PVD production line and transmits it to a cloud processor. In the transmission to the cloud processor, the image analysis and coding technology based on computer vision is used to perform panoramic segmentation on the surface state image of the PVD-coated substrate, and then the segmented PVD-coated substrate surface state mask representation image is subjected to feature extraction, and the PVD-coated substrate surface state image is gridded and grid feature extracted at the same time, so as to automatically generate a cleanliness label of the surface of the PVD-coated substrate based on the fine-grained ablation multi-dimensional semantic coding representation between the surface state grid feature of the PVD-coated substrate and the surface state mask feature of the PVD-coated substrate, and judge whether a warning signal is generated accordingly. In this way, the cleanliness label can be generated instantly, and real-time monitoring in the continuous production process can be realized, thereby improving the accuracy of cleanliness detection.

[0006] According to one aspect of the present application, a method for detecting the surface cleanliness of a substrate for PVD coating is provided, comprising:

[0007] The surface status images of PVD coated substrates are collected by cameras deployed at key positions of the PVD production line;

[0008] Transmitting the surface state image of the PVD coated substrate to a cloud processor;

[0009] In the cloud processor, performing panoramic segmentation on the surface state image of the PVD-coated substrate to obtain a mask representation of the surface state of the PVD-coated substrate;

[0010] Performing grid division and surface state grid feature extraction on the surface state image of the PVD coated substrate to obtain the surface state grid features of the PVD coated substrate;

[0011] Extracting surface state mask features from the PVD-coated substrate surface state mask representation to obtain PVD-coated substrate surface state mask features;

[0012] Performing fine-grained ablation attention semantic coding on the grid features of the surface state of the PVD-coated substrate and the mask features of the surface state of the PVD-coated substrate to obtain multi-dimensional semantic ablation coding features of the surface state of the PVD-coated substrate;

[0013] Based on the multi-dimensional semantic ablation coding features of the surface state of the PVD coated substrate, a substrate surface cleanliness detection result is obtained, and based on the substrate surface cleanliness detection result, it is determined whether to generate a warning signal.

[0014] According to another aspect of the present application, a substrate surface cleanliness detection device for PVD coating is provided, comprising:

[0015] An image acquisition module is used to acquire the surface state image of the PVD coating substrate through a camera deployed at a key position of the PVD production line;

[0016] An image transmission module, used to transmit the surface state image of the PVD coated substrate to a cloud processor;

[0017] An image panoramic segmentation module is used to perform panoramic segmentation on the surface state image of the PVD-coated substrate in the cloud processor to obtain a mask representation of the surface state of the PVD-coated substrate;

[0018] A PVD coated substrate surface state grid feature extraction module is used to perform grid division and surface state grid feature extraction on the PVD coated substrate surface state image to obtain the PVD coated substrate surface state grid feature;

[0019] A surface state mask feature extraction module, used for performing surface state mask feature extraction on the surface state mask representation of the PVD-coated substrate to obtain surface state mask features of the PVD-coated substrate;

[0020] A fine-grained ablation attention semantic coding module, used for performing fine-grained ablation attention semantic coding on the grid features of the surface state of the PVD-coated substrate and the mask features of the surface state of the PVD-coated substrate to obtain a multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate;

[0021] The detection result generating module is used to obtain the substrate surface cleanliness detection result based on the multi-dimensional semantic ablation coding feature of the surface state of the PVD coated substrate, and determine whether to generate a warning signal based on the substrate surface cleanliness detection result.

[0022] Compared with the prior art, the present application provides a method and device for detecting the surface cleanliness of a PVD-coated substrate. The camera deployed at a key position of the PVD production line collects the surface state image of the PVD-coated substrate and transmits it to a cloud processor. In the transmission to the cloud processor, the image analysis and coding technology based on computer vision is used to perform panoramic segmentation on the surface state image of the PVD-coated substrate. Then, the segmented PVD-coated substrate surface state mask representation image is subjected to feature extraction. At the same time, the PVD-coated substrate surface state image is gridded and grid feature extracted. In this way, the cleanliness label of the PVD-coated substrate surface is automatically generated based on the fine-grained ablation multi-dimensional semantic coding representation between the PVD-coated substrate surface state grid feature and the PVD-coated substrate surface state mask feature, and it is judged whether a warning signal is generated. In this way, the cleanliness label can be generated instantly, and real-time monitoring in the continuous production process can be realized, thereby improving the accuracy of cleanliness detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 It is a flow chart of a method for detecting the surface cleanliness of a substrate for PVD coating according to an embodiment of the present application;

[0025] Figure 2 A schematic diagram of data flow of a method for detecting surface cleanliness of a substrate for PVD coating according to an embodiment of the present application;

[0026] Figure 3 It is a flowchart of sub-step S6 of the method for detecting the surface cleanliness of a substrate for PVD coating according to an embodiment of the present application;

[0027] Figure 4 4 is a block diagram of a substrate surface cleanliness detection device for PVD coating according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0029] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0030] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0031] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0032] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0033] In the technical solution of the present application, a method for detecting the surface cleanliness of a substrate for PVD coating is proposed. Figure 1 Flow chart of a method for detecting the surface cleanliness of a substrate for PVD coating according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a method for detecting the surface cleanliness of a substrate for PVD coating according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to the embodiment of the present application, the surface cleanliness detection method for PVD-coated substrates includes the following steps: S1, collecting a surface state image of a PVD-coated substrate by a camera deployed at a key position of a PVD production line; S2, transmitting the surface state image of the PVD-coated substrate to a cloud processor; S3, in the cloud processor, performing panoramic segmentation on the surface state image of the PVD-coated substrate to obtain a mask representation of the surface state of the PVD-coated substrate; S4, performing grid division and surface state grid feature extraction on the surface state image of the PVD-coated substrate to obtain a surface state mask representation of the PVD-coated substrate. state grid feature; S5, extracting the surface state mask feature of the PVD-coated substrate surface state mask representation to obtain the surface state mask feature of the PVD-coated substrate; S6, performing fine-grained ablation attention semantic encoding on the surface state grid feature of the PVD-coated substrate and the surface state mask feature of the PVD-coated substrate to obtain the multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate; S7, obtaining the substrate surface cleanliness detection result based on the multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate, and determining whether to generate a warning signal based on the substrate surface cleanliness detection result.

[0034] In particular, the S1 collects the surface state image of the PVD coated substrate by a camera deployed at a key position of the PVD production line. Here, in order to ensure that the tiny features and potential contaminants on the substrate surface can be captured, it is necessary to first select a high-resolution industrial camera to ensure that every detail in the image is clearly recorded, thereby providing sufficient information for subsequent analysis. In addition, considering that the speed of the PVD production line may be very fast, the selected camera must have a sufficient frame rate so that high-quality images can be continuously obtained even at high speeds. The frame rate should be adjusted according to the speed of the substrate movement on the production line and the required detection accuracy to ensure that neither the production efficiency is affected by too slow speed nor the image is blurred or information is lost due to too fast speed. At the same time, the spectral response characteristics of the camera are also crucial. Different substrate materials and the intended detection targets (such as specific types of contamination or defects) may have different requirements for light sources. Therefore, when selecting a camera, it should be considered whether it can work effectively in different spectral ranges such as visible light, near infrared or ultraviolet bands. This helps to improve the accuracy of detection, especially for those subtle changes or hidden problems that are difficult to detect with the naked eye. In addition, given that the PVD production environment may have factors such as high temperature, high pressure, and corrosive gases, the selected camera must have a good level of protection to ensure that it can still operate stably and maintain high performance in harsh environments.

[0035] In particular, S2 transmits the surface state image of the PVD coated substrate to the cloud processor. In the PVD production line, high-resolution industrial cameras deployed at key positions are responsible for capturing the state image of the substrate surface. In order to ensure the image quality and the accuracy of subsequent processing, some preliminary preprocessing is usually performed before the image is uploaded. These preprocessing operations include denoising, brightness adjustment, contrast enhancement, and resizing, which are designed to improve image clarity, highlight important features, and reduce unnecessary data volume, making transmission more efficient. For example, denoising can reduce random noise in the image and make the image clearer; brightness adjustment ensures that all images have similar brightness levels for subsequent comparative analysis; contrast enhancement helps to highlight surface textures or potential contaminants; and resizing crops or scales the original image as needed to meet the needs of subsequent processing. These preprocessing steps can not only significantly improve image quality, but also lay a good foundation for subsequent in-depth analysis.

[0036] Considering that high-definition image files are large, uploading them directly to the cloud will occupy more bandwidth and increase latency, so images are usually compressed before transmission. The choice of compression method depends on the specific application scenario and technical requirements. Lossless compression (such as PNG format) can reduce the file size without losing any information, and is suitable for occasions where no information loss is allowed. Although lossy compression (such as JPEG format) will cause a certain degree of information loss, it can significantly reduce the file size while maintaining sufficient visual quality, which is suitable for most situations. In addition, more advanced compression algorithms such as JPEG 2000 can also be used, which provides better compression ratios and higher image quality. For particularly large image sets, using video coding technology (such as H.264 / AVC or HEVC) to pack multiple consecutive frames is also an effective method, which can not only further reduce the bandwidth required for transmission, but also better preserve time series information.

[0037] Establishing a stable and fast network connection is an important prerequisite for ensuring smooth transmission of image data to the cloud processor. If the camera and the cloud server are located in the same physical location, they can be directly connected via a high-speed local area network. This method provides extremely low latency and high bandwidth, which is very suitable for internal data transmission. However, when the camera and the cloud server are separated by a long distance, they need to be connected via the Internet or other forms of wide area network. At this time, it is particularly important to choose a suitable network service provider and optimize the network configuration. Wireless networks (such as Wi-Fi, 5G) may also be necessary in some special cases, such as on mobile production lines. At this time, ensure that the signal strength is sufficient and take measures to avoid interference to ensure the stability of data transmission.

[0038] The choice of communication protocol is also crucial. HTTP(S) is a commonly used protocol suitable for batch uploading of large files; MQTT is more suitable for real-time streaming of small message packets, especially in the Internet of Things (IoT) environment; and CoAP is a lightweight protocol designed for resource-constrained devices and suitable for low-power scenarios. Different protocols have their own advantages and disadvantages, and they need to be flexibly selected according to actual conditions. In order to cope with possible network failures or congestion, edge computing nodes can be set up close to the data source. Edge computing allows some data analysis tasks to be completed locally, and only the necessary results are sent to the cloud, thereby reducing the pressure on the main server. At the same time, this also means that even during a network outage, the system can continue to collect and store image data and upload it after the network returns to normal.

[0039] In addition, in order to prevent data loss due to network problems, a cache mechanism can be established locally. That is, when a network anomaly is detected, it automatically switches to offline mode and temporarily saves the data that has not been successfully uploaded in a temporary storage space. Once the network condition improves, the cached data is immediately resent to ensure that no important information is missed. This approach can not only improve the fault tolerance of the system, but also ensure the integrity of the entire data transmission link.

[0040] In particular, in the S3, the cloud processor performs panoramic segmentation on the surface state image of the PVD-coated substrate to obtain a mask representation of the surface state of the PVD-coated substrate. Considering that the surface state image of the PVD-coated substrate may contain the same type of potential problems or features, such as scratches and particles, and each surface morphological feature or contaminant has boundary information. Based on this, in order to more accurately divide and segment each category, in the technical solution of the present application, the surface state image of the PVD-coated substrate is panoramically segmented to obtain a mask representation of the surface state of the PVD-coated substrate. It should be understood that panoramic segmentation is a computer vision technology that combines the advantages of instance segmentation and semantic segmentation, and can simultaneously identify which category each pixel in the image belongs to and distinguish different instances. That is, panoramic segmentation can provide finer object boundaries, which is particularly important for detecting tiny contaminants on the surface of the substrate. It can accurately depict the outline of each specific area on the substrate, thereby providing high-precision input for subsequent analysis. At the same time, panoramic segmentation can not only identify the categories to which these features belong, but also distinguish them as independent instances, which helps to more carefully evaluate the impact of each individual surface state on the overall cleanliness. Therefore, the results obtained by panoramic segmentation can be directly converted into a mask form, that is, a binary image or a label image, in which each pixel has a corresponding category label. This mask representation is intuitive and easy to process, and is very suitable for further feature extraction and quantitative analysis.

[0041] In particular, the S4 performs meshing and surface state mesh feature extraction on the surface state image of the PVD coated substrate to obtain the surface state mesh feature of the PVD coated substrate. That is, in a specific example of the present application, the surface state image of the PVD coated substrate is meshed and then the mesh feature extractor of the substrate surface state image based on the CNN-LSTM model is used to obtain the surface state mesh feature vector of the PVD coated substrate as the surface state mesh feature of the PVD coated substrate. Considering that there are different features between different local areas in the surface state image of the PVD coated substrate, in order to more carefully capture and extract the subtle surface state features in each area, in the technical solution of the present application, the surface state image of the PVD coated substrate is meshed to divide the image into multiple small grids, each of which can be regarded as an independent small area, which enables the model to focus more on local details, such as tiny surface state textures or contaminants. Further, the gridded PVD coated substrate surface state image is passed through the substrate surface state image grid feature extractor based on the CNN-LSTM model to combine the spatial feature extraction capability of the convolutional neural network (CNN) and the time series modeling capability of the long short-term memory network (LSTM) to obtain the PVD coated substrate surface state grid feature vector. Specifically, the CNN in the CNN-LSTM model is good at extracting spatial features such as edges, textures, and shapes from two-dimensional images. That is, by processing the gridded image, each grid can be regarded as an independent small area, and the CNN can focus on the detailed features in these local areas, such as tiny contaminants. The accurate capture of such local features is crucial for evaluating the surface state of the substrate. In the PVD production line, the substrate moves continuously, so there may be temporal correlation between the features of adjacent grids. LSTM is particularly suitable for processing data with time series characteristics. It can remember long-term dependencies, which is very important for understanding the development trend of surface information or contaminants over time. For example, some types of potential problems may gradually form or spread, and LSTM can help identify these dynamic changes. In this way, not only can the local and global spatial features be captured efficiently, but also the correlation between various regions can be effectively processed, providing a more accurate and comprehensive surface condition assessment.

[0042] In particular, the S5 performs surface state mask feature extraction on the surface state mask representation of the PVD-coated substrate to obtain the surface state mask feature of the PVD-coated substrate. That is, in a specific example of the present application, the surface state mask representation of the PVD-coated substrate is passed through a substrate surface state image mask feature extractor based on the FCN model to obtain a PVD-coated substrate surface state mask feature map as the surface state mask feature of the PVD-coated substrate. Here, in order to further refine the category information of each area in the mask representation map, in the technical solution of the present application, the surface state mask representation of the PVD-coated substrate is passed through a substrate surface state image mask feature extractor based on the FCN model to obtain a PVD-coated substrate surface state mask feature map. In particular, FCN is specially designed for the task of image semantic segmentation, which can assign a label to each pixel. For the status analysis of the surface of PVD-coated substrates, this means that every point on the surface can be classified, such as distinguishing normal areas from areas with potential problems, thereby achieving extremely fine positioning to obtain accurate pixel-level semantic classification results and obtain key semantic information.

[0043] In particular, the S6 performs fine-grained ablation attention semantic coding on the surface state grid features of the PVD-coated substrate and the surface state mask features of the PVD-coated substrate to obtain multi-dimensional semantic ablation coding features of the surface state of the PVD-coated substrate. Considering that the surface state grid features of the PVD-coated substrate capture the changes in local details of the substrate surface such as texture, roughness, brightness, etc. The surface state mask feature map of the PVD-coated substrate is mainly used for positioning and semantic segmentation purposes to identify areas that require special attention, such as where contaminants exist. Therefore, in order to effectively integrate information from these two different sources to form a more complete surface state description, the present application introduces a fine-grained ablation attention semantic coding mechanism to integrate the surface state grid features of the PVD-coated substrate and the surface state mask features of the PVD-coated substrate to obtain multi-dimensional semantic ablation coding features of the surface state of the PVD-coated substrate. In particular, fine-grained ablation technology identifies small problems that may affect the final coating quality but are not easy to detect, such as tiny particles, slight scratches and other surface information, thereby capturing more details, which helps to generate richer semantic representations and improve the ability to understand complex feature patterns. By introducing the attention mechanism, the model can automatically focus on the key areas that are most likely to contain potential problems, reduce false alarm rates, and improve detection accuracy. In a specific example of this application, Figure 3As shown, the S6 includes: S61, performing feature fine-grained decoupling on the PVD-coated substrate surface state mask feature map to obtain a set of PVD-coated substrate surface state local mask feature matrices; S62, based on the PVD-coated substrate surface state grid feature vector, calculating the fine-grained ablation weight factor of each PVD-coated substrate surface state local mask feature matrix in the set of PVD-coated substrate surface state local mask feature matrices to obtain a set of PVD-coated substrate surface state multi-dimensional fine-grained ablation weight factors; S63, based on the set of PVD-coated substrate surface state multi-dimensional fine-grained ablation weight factors, performing feature weighted aggregation on the set of PVD-coated substrate surface state local mask feature matrices to obtain a PVD-coated substrate surface state multi-dimensional semantic ablation coding feature map as the PVD-coated substrate surface state multi-dimensional semantic ablation coding feature.

[0044] Specifically, the S61 performs feature fine-grained decoupling on the PVD-coated substrate surface state mask feature map to obtain a set of local mask feature matrices of the PVD-coated substrate surface state. In an embodiment of the present application, the PVD-coated substrate surface state mask feature map is feature decoupled along the channel dimension to obtain a set of local mask feature matrices of the PVD-coated substrate surface state. More specifically, the PVD-coated substrate surface state mask feature map is feature fine-grained decoupling using the following feature decoupling formula to obtain a set of local mask feature matrices of the PVD-coated substrate surface state; wherein the feature decoupling formula is:

[0045]

[0046] in, is the surface state mask feature map of the PVD coated substrate, For feature decoupling operations, , , and It is the first, second, and third in the set of local mask feature matrices of the surface state of the PVD coating substrate. and A local mask feature matrix of the surface state of a PVD coated substrate.

[0047] Specifically, the S62, based on the grid feature vector of the surface state of the PVD-coated substrate, calculates the fine-grained ablation weight factor of each local mask feature matrix of the surface state of the PVD-coated substrate in the set of local mask feature matrices of the surface state of the PVD-coated substrate to obtain a set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate. In an embodiment of the present application, first, using each local mask feature matrix of the surface state of the PVD-coated substrate in the set of local mask feature matrices of the surface state of the PVD-coated substrate as a key matrix, the grid feature vector of the surface state of the PVD-coated substrate and the local mask feature matrix of the surface state of the PVD-coated substrate are input into an ablation factor encoder based on a simulated converter structure to obtain a set of multi-dimensional fine-grained ablation factors of the surface state of the PVD-coated substrate. That is, the use of a simulated converter structure to calculate fine-grained ablation factors actually draws on the idea of ​​the self-attention mechanism, and makes adaptive improvements to be suitable for multimodal data processing. In this process, the grid feature vector of the surface state of the PVD coated substrate is linearly transformed to obtain a query vector and a value vector, and the local mask feature matrix of the surface state of the PVD coated substrate is used as a key matrix. In this way, the model can establish a direct connection between the two modes, and then use the ablation metric function to measure the importance of removing this connection. Then, the set of multidimensional fine-grained ablation factors of the surface state of the PVD coated substrate is input into the ablation effect encoding module containing the normalization function and the mask function to obtain the set of multidimensional fine-grained ablation weight factors of the surface state of the PVD coated substrate. Normalization is a commonly used technique in machine learning to ensure that data of different scales are compared in the same framework. In this step, normalization helps adjust the distribution of fine-grained ablation factors so that the weight distribution is more reasonable. On the other hand, the role of the mask function is to selectively highlight certain important local features while suppressing irrelevant or redundant information. More specifically, based on the grid feature vector of the surface state of the PVD-coated substrate, the fine-grained ablation weight factor of each PVD-coated substrate surface state local mask feature matrix in the set of the PVD-coated substrate surface state local mask feature matrix is ​​calculated using the following fine-grained ablation weight calculation formula to obtain a set of multi-dimensional fine-grained ablation weight factors of the PVD-coated substrate surface state; wherein the fine-grained ablation weight calculation formula is:

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] in, is the grid feature vector of the surface state of the PVD coated substrate, is matrix multiplication, and They are the substrate surface state grid feature query embedding matrix and the substrate surface state grid feature query bias vector, is the matrix surface state grid feature query vector, and are the substrate surface state grid eigenvalue embedding matrix and the substrate surface state grid eigenvalue bias vector, respectively. is the grid eigenvalue vector of the substrate surface state, and are the substrate surface state local mask feature bond embedding matrix and the substrate surface state local mask feature bond bias vector, yes The corresponding substrate surface state local mask feature bond matrix, yes The transposed matrix of yes The scale is the width of the matrix multiplied by the height of the matrix, yes function, It is the first in the set of multi-dimensional fine-grained interaction vectors of the surface state of the PVD coating substrate. A multi-dimensional fine-grained interaction vector of the surface state of the PVD-coated substrate, To extract the maximum value operation, and They are The mean and variance of is a hyperparameter, For calculation The fine-grained ablation factor, It is the first in the set of multi-dimensional fine-grained ablation factors of the surface state of PVD coating substrate. Multi-dimensional fine-grained ablation factor of the surface state of the PVD-coated substrate, For calculation The fine-grained ablation weight factor, represents the exponential function value with the natural constant e as the base, It is the first in the set of multi-dimensional fine-grained ablation factors of the normalized PVD coating substrate surface state. Normalized multi-dimensional fine-grained ablation factor of the surface state of the PVD-coated substrate, is a mask operation, is the preset threshold, , , and It is the first, second, and third in the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD coated substrate. and A multi-dimensional fine-grained ablation weight factor for the surface state of a PVD-coated substrate.

[0055] Specifically, the S63, based on the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, performs feature weighted aggregation on the set of local mask feature matrices of the surface state of the PVD-coated substrate to obtain a multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate as the multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate. In the technical solution of the present application, firstly, based on the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, the set of local mask feature matrices of the surface state of the PVD-coated substrate is subjected to fine-grained ablation modulation to obtain a set of local mask feature matrices of the surface state of the cross-domain guided ablation PVD-coated substrate. Furthermore, the set of local mask feature matrices of the surface state of the cross-domain guided ablation PVD-coated substrate is subjected to feature aggregation to obtain the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate. More specifically, based on the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, the set of local mask feature matrices of the surface state of the PVD-coated substrate is subjected to feature weighted aggregation using the following feature weighted aggregation formula to obtain a multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate; wherein the feature weighted aggregation formula is:

[0056]

[0057] in, It is a multi-dimensional semantic ablation coding feature map of the surface state of the PVD coated substrate.

[0058] In particular, the S7 obtains the surface cleanliness detection result of the substrate based on the multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate, and determines whether to generate a warning signal based on the surface cleanliness detection result of the substrate. In the technical solution of the present application, the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is passed through a surface cleanliness detector based on a classifier to obtain the surface cleanliness detection result of the substrate, and the surface cleanliness detection result of the substrate is used to represent the cleanliness label of the surface of the PVD-coated substrate. That is, the multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate obtained by fine-grained ablation coding using the grid feature vector of the surface state of the PVD-coated substrate and the mask feature map of the surface state of the PVD-coated substrate is classified and processed to automatically generate the cleanliness label of the surface of the PVD-coated substrate, and judge whether to generate a warning signal based on this. In this way, by real-time image acquisition by the camera deployed on the production line and rapid processing with the help of the cloud processor, the system can instantly generate cleanliness labels and support real-time monitoring in the continuous production process. Moreover, it not only considers the overall distribution of the surface, but also pays special attention to the subtle changes in the local area, providing a more detailed and rich data description, thereby improving the accuracy of the detection. Specifically, the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is passed through a surface cleanliness detector based on a classifier to obtain the surface cleanliness detection result of the substrate, including: expanding the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate into a multi-dimensional semantic ablation coding description coding vector of the surface state of the PVD-coated substrate based on a row vector or a column vector; using multiple fully connected layers of the classifier to fully connect the multi-dimensional semantic ablation coding description coding vector of the surface state of the PVD-coated substrate to obtain a coded multi-dimensional semantic ablation coding description coding vector of the surface state of the PVD-coated substrate; and passing the coded multi-dimensional semantic ablation coding description coding vector of the surface state of the PVD-coated substrate through the Softmax classification function of the classifier to obtain the surface cleanliness detection result of the substrate.

[0059] In a preferred example, considering that the PVD-coated substrate surface state grid feature vector and the PVD-coated substrate surface state mask feature map respectively represent the image semantic features of the PVD-coated substrate surface state image at different local image semantic space domain association scales, when performing cross-domain attention joint encoding based on fine-grained ablation, due to the differences in the cross-domain ablation mechanism caused by the differences in the semantic association scale feature distribution of the source data, the PVD-coated substrate surface state multi-dimensional semantic ablation coding feature map will also have sparse ablation associations based on different attention joint encoding enhancements, thereby affecting the accuracy of the substrate surface cleanliness detection results obtained by the classifier-based surface cleanliness detector due to the lack of classification reasoning degree.

[0060] Therefore, when the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is passed through a surface cleanliness detector based on a classifier, the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is first optimized, including the following steps:

[0061] The multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is expanded into a multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate, and the first Eigenvalues ​​and The eigenvalues Distance and Distance, thereby obtaining the first PVD coating substrate surface state multi-dimensional semantic ablation coding description distance matrix and the second PVD coating substrate surface state multi-dimensional semantic ablation coding description distance matrix:

[0062]

[0063]

[0064] in, and The first and second dimensional semantic ablation coding feature vectors of the surface state of the PVD coating substrate are respectively represented by Eigenvalues ​​and Eigenvalues, and Respectively represent calculation Distance and distance. The multi-dimensional semantic ablation coding description distance matrix represents the surface state of the first PVD coating substrate. A multi-dimensional semantic ablation coding description distance matrix representing the surface state of the second PVD-coated substrate;

[0065] Calculate the weighted sum of the multidimensional semantic ablation coding description distance matrix of the surface state of the first PVD coated substrate and the multidimensional semantic ablation coding description distance matrix of the surface state of the second PVD coated substrate, and determine the eigenvalues ​​of the distance weighted sum matrix arrive , so as to arrange the eigenvalues ​​to obtain the multi-dimensional semantic ablation coding description distance eigenvector of the surface state of the PVD coating substrate :

[0066]

[0067] in, represents each eigenvalue of the distance weighted sum matrix;

[0068] The multi-dimensional semantic ablation coding description of the surface state of the PVD coating substrate is described by the distance eigenvector Performing interpolation to obtain an interpolated eigenvector describing the multi-dimensional semantic ablation coding of the surface state of the PVD-coated substrate having the same length as the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate;

[0069] The multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate is matrix-multiplied with the first multi-dimensional semantic ablation coding description distance matrix of the surface state of the PVD-coated substrate to obtain the multi-dimensional semantic ablation coding description intermediate feature vector of the surface state of the PVD-coated substrate, and the self-correlation matrix of the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate is multiplied with the second multi-dimensional semantic ablation coding description distance matrix of the surface state of the PVD-coated substrate and the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate, that is, Perform matrix multiplication to obtain the intermediate feature matrix describing the multi-dimensional semantic ablation coding of the surface state of the PVD-coated substrate. represents the transpose of a vector, represents matrix multiplication, A multi-dimensional semantic ablation coding feature vector representing the surface state of the PVD-coated substrate;

[0070] After matrix multiplication of the intermediate feature vector of the multidimensional semantic ablation coding description of the surface state of the PVD-coated substrate and the intermediate feature matrix of the multidimensional semantic ablation coding description of the surface state of the PVD-coated substrate, it is further point multiplied with the interpolation eigenvector of the multidimensional semantic ablation coding description of the surface state of the PVD-coated substrate to obtain an optimized multidimensional semantic ablation coding description coding vector of the surface state of the PVD-coated substrate. Finally, the optimized multidimensional semantic ablation coding description coding vector of the surface state of the PVD-coated substrate is passed through a surface cleanliness detector based on a classifier to obtain the surface cleanliness detection result of the substrate.

[0071] Based on this, the first distance matrix and the second distance matrix of the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate after the multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is expanded are used as the fine-grained metric association cluster representation of the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate, and the dynamic programming of the relationship between the association clusters of different association clusters is performed on the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate and the self-association representation of the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate, respectively, to simulate the sparse activation based on neuron clusters of the association system, and the fine-grained predictable sparsity of the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate is coordinated with the intrinsic representation of the metric association cluster of the first distance matrix and the second distance matrix of the multi-dimensional semantic ablation coding feature vector of the surface state of the PVD-coated substrate, so as to avoid the lack of decoding inference degree affected by insufficient association caused by sparsity, and improve the accuracy of the substrate surface cleanliness detection result obtained by the classifier-based surface cleanliness detector.

[0072] In summary, according to the embodiment of the present application, the surface cleanliness detection method for PVD-coated substrates is explained, and the surface state image of the PVD-coated substrate is collected by a camera deployed at a key position of the PVD production line, and transmitted to a cloud processor. In the transmission to the cloud processor, the image analysis and coding technology based on computer vision is used to perform panoramic segmentation on the surface state image of the PVD-coated substrate, and then the segmented PVD-coated substrate surface state mask representation image is subjected to feature extraction, and the surface state image of the PVD-coated substrate is gridded and grid feature extracted at the same time, so as to automatically generate a cleanliness label of the surface of the PVD-coated substrate based on the fine-grained ablation multi-dimensional semantic coding representation between the surface state grid feature of the PVD-coated substrate and the surface state mask feature of the PVD-coated substrate, and judge whether a warning signal is generated accordingly. In this way, the cleanliness label can be generated instantly, and real-time monitoring in the continuous production process can be realized, thereby improving the accuracy of cleanliness detection.

[0073] Furthermore, a substrate surface cleanliness detection device for PVD coating is also provided.

[0074] Figure 4 FIG. 1 is a block diagram of a substrate surface cleanliness detection device for PVD coating according to an embodiment of the present application. Figure 4As shown, according to an embodiment of the present application, a substrate surface cleanliness detection device 300 for PVD coating includes: an image acquisition module 310, which is used to acquire a surface state image of a PVD coating substrate through a camera deployed at a key position of a PVD production line; an image transmission module 320, which is used to transmit the surface state image of the PVD coating substrate to a cloud processor; an image panoramic segmentation module 330, which is used to perform panoramic segmentation on the surface state image of the PVD coating substrate in the cloud processor to obtain a mask representation of the surface state of the PVD coating substrate; a PVD coating substrate surface state grid feature extraction module 340, which is used to perform grid division and surface state grid feature extraction on the surface state image of the PVD coating substrate to obtain a PVD coating substrate surface state mask representation. Substrate surface state grid features; a surface state mask feature extraction module 350, used to extract surface state mask features from the PVD-coated substrate surface state mask representation to obtain PVD-coated substrate surface state mask features; a fine-grained ablation attention semantic coding module 360, used to perform fine-grained ablation attention semantic coding on the PVD-coated substrate surface state grid features and the PVD-coated substrate surface state mask features to obtain PVD-coated substrate surface state multi-dimensional semantic ablation coding features; a detection result generation module 370, used to obtain a substrate surface cleanliness detection result based on the PVD-coated substrate surface state multi-dimensional semantic ablation coding features, and based on the substrate surface cleanliness detection result, determine whether to generate a warning signal.

[0075] As described above, the substrate surface cleanliness detection device 300 for PVD coating according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a substrate surface cleanliness detection algorithm for PVD coating. In a possible implementation, the substrate surface cleanliness detection device 300 for PVD coating according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the substrate surface cleanliness detection device 300 for PVD coating can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the substrate surface cleanliness detection device 300 for PVD coating can also be one of the many hardware modules of the wireless terminal.

[0076] Alternatively, in another example, the substrate surface cleanliness detection device 300 for PVD coating and the wireless terminal may also be separate devices, and the substrate surface cleanliness detection device 300 for PVD coating may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0077] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting the surface cleanliness of a substrate for PVD coating, characterized in that: include: The surface status images of PVD coated substrates are collected by cameras deployed at key positions of the PVD production line; Transmitting the surface state image of the PVD coated substrate to a cloud processor; In the cloud processor, performing panoramic segmentation on the surface state image of the PVD-coated substrate to obtain a mask representation of the surface state of the PVD-coated substrate; Performing grid division and surface state grid feature extraction on the surface state image of the PVD coated substrate to obtain the surface state grid features of the PVD coated substrate; Extracting surface state mask features from the PVD-coated substrate surface state mask representation to obtain PVD-coated substrate surface state mask features; Performing fine-grained ablation attention semantic coding on the grid features of the surface state of the PVD-coated substrate and the mask features of the surface state of the PVD-coated substrate to obtain multi-dimensional semantic ablation coding features of the surface state of the PVD-coated substrate; Based on the multi-dimensional semantic ablation coding features of the surface state of the PVD coated substrate, a substrate surface cleanliness detection result is obtained, and based on the substrate surface cleanliness detection result, it is determined whether to generate a warning signal.

2. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 1, characterized in that: The surface state image of the PVD-coated substrate is gridded and surface state grid features are extracted to obtain the surface state grid features of the PVD-coated substrate, including: after the surface state image of the PVD-coated substrate is gridded, a substrate surface state image grid feature extractor based on a CNN-LSTM model is used to obtain a PVD-coated substrate surface state grid feature vector as the surface state grid feature of the PVD-coated substrate.

3. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 2, characterized in that: Surface state mask feature extraction is performed on the PVD-coated substrate surface state mask representation image to obtain the PVD-coated substrate surface state mask feature, including: passing the PVD-coated substrate surface state mask representation image through a substrate surface state image mask feature extractor based on an FCN model to obtain a PVD-coated substrate surface state mask feature image as the PVD-coated substrate surface state mask feature.

4. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 3, characterized in that: Fine-grained ablation attention semantic coding is performed on the surface state grid feature of the PVD coated substrate and the surface state mask feature of the PVD coated substrate to obtain multi-dimensional semantic ablation coding features of the surface state of the PVD coated substrate, including: Performing feature fine-grained decoupling on the PVD-coated substrate surface state mask feature map to obtain a set of PVD-coated substrate surface state local mask feature matrices; Based on the grid feature vector of the surface state of the PVD-coated substrate, the fine-grained ablation weight factor of each local mask feature matrix of the surface state of the PVD-coated substrate in the set of local mask feature matrices of the surface state of the PVD-coated substrate is calculated to obtain a set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate; Based on the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, the set of local mask feature matrices of the surface state of the PVD-coated substrate is subjected to feature weighted aggregation to obtain a multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate as the multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate.

5. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 4, characterized in that: The PVD-coated substrate surface state mask feature map is subjected to feature fine-grained decoupling to obtain a set of PVD-coated substrate surface state local mask feature matrices, including: the PVD-coated substrate surface state mask feature map is subjected to feature decoupling along the channel dimension to obtain a set of PVD-coated substrate surface state local mask feature matrices.

6. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 5, characterized in that: Based on the grid feature vector of the surface state of the PVD-coated substrate, the fine-grained ablation weight factor of each local mask feature matrix of the surface state of the PVD-coated substrate in the set of local mask feature matrices of the surface state of the PVD-coated substrate is calculated to obtain a set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, including: Using each PVD-coated substrate surface state local mask feature matrix in the set of the PVD-coated substrate surface state local mask feature matrix as a key matrix, inputting the PVD-coated substrate surface state grid feature vector and the PVD-coated substrate surface state local mask feature matrix into an ablation factor encoder based on an analog converter structure to obtain a set of PVD-coated substrate surface state multi-dimensional fine-grained ablation factors; The set of multi-dimensional fine-grained ablation factors of the surface state of the PVD-coated substrate is input into an ablation effect encoding module including a normalization function and a mask function to obtain a set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate.

7. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 6, characterized in that: Based on the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, a set of local mask feature matrices of the surface state of the PVD-coated substrate is subjected to feature weighted aggregation to obtain a multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate, including: Based on the set of multi-dimensional fine-grained ablation weight factors of the surface state of the PVD-coated substrate, fine-grained ablation modulation is performed on the set of local mask feature matrices of the surface state of the PVD-coated substrate to obtain a set of local mask feature matrices of the surface state of the cross-domain guided ablation PVD-coated substrate; The set of local mask feature matrices of the cross-domain guided ablation PVD coated substrate surface state is feature aggregated to obtain a multi-dimensional semantic ablation coding feature map of the PVD coated substrate surface state.

8. The method for detecting the surface cleanliness of a substrate for PVD coating according to claim 7, characterized in that: Based on the multi-dimensional semantic ablation coding features of the surface state of the PVD coated substrate, a substrate surface cleanliness detection result is obtained, and based on the substrate surface cleanliness detection result, whether to generate a warning signal is determined, including: The multi-dimensional semantic ablation coding feature map of the surface state of the PVD-coated substrate is passed through a surface cleanliness detector based on a classifier to obtain a surface cleanliness detection result of the substrate, and the surface cleanliness detection result of the substrate is used to represent a cleanliness label of the surface of the PVD-coated substrate; Based on the substrate surface cleanliness detection result, it is determined whether to generate the warning signal.

9. A substrate surface cleanliness detection device for PVD coating, characterized in that: include: An image acquisition module is used to acquire the surface state image of the PVD coating substrate through a camera deployed at a key position of the PVD production line; An image transmission module, used to transmit the surface state image of the PVD coated substrate to a cloud processor; An image panoramic segmentation module is used to perform panoramic segmentation on the surface state image of the PVD-coated substrate in the cloud processor to obtain a mask representation of the surface state of the PVD-coated substrate; A PVD coated substrate surface state grid feature extraction module is used to perform grid division and surface state grid feature extraction on the PVD coated substrate surface state image to obtain the PVD coated substrate surface state grid feature; A surface state mask feature extraction module, used for performing surface state mask feature extraction on the surface state mask representation of the PVD-coated substrate to obtain surface state mask features of the PVD-coated substrate; A fine-grained ablation attention semantic coding module, used for performing fine-grained ablation attention semantic coding on the grid features of the surface state of the PVD-coated substrate and the mask features of the surface state of the PVD-coated substrate to obtain a multi-dimensional semantic ablation coding feature of the surface state of the PVD-coated substrate; The detection result generating module is used to obtain the substrate surface cleanliness detection result based on the multi-dimensional semantic ablation coding feature of the surface state of the PVD coated substrate, and determine whether to generate a warning signal based on the substrate surface cleanliness detection result.

Images