Laser transfer printing method and system based on deep learning image recognition

Abstract

The present application discloses a laser transfer printing method and system based on deep learning image recognition, which relates to the field of image recognition. The method comprises: in the current graphic element process flow of laser transfer printing, real-time acquisition of a live image is carried out through a camera; after the current graphic element process flow, laser transfer printing is carried out according to the laminated image obtained by the nickel stick graphic element laminated process, and the live image is respectively subjected to image recognition through a first deep learning model to obtain a first image feature; according to the first image feature, a quality assessment step is performed to generate a quality assessment report for each graphic element process flow; based on the image defect information, the wavelength and the angle of the narrow-band LED light source are adjusted; the potential defect information and the current process parameters of each graphic element process flow are input into a second deep learning model to obtain abnormal process parameters of each graphic element process flow; the current process parameters are adjusted according to the abnormal process parameters to obtain updated process parameters.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image recognition, and in particular, to a laser transfer printing method and system based on deep learning image recognition. Background Art

[0002] In order to improve printing quality, laser transfer printing technology has become increasingly important in modern manufacturing. Laser transfer printing technology refers to the precise transfer of materials pre-coated on a substrate to a target substrate through the irradiation of a laser beam to achieve high-precision replication of patterns. It is currently widely used in the fields of electronics, optics, and materials science.

[0003] In the existing laser transfer printing method, the defects of the finished product are usually identified through a pre-built image recognition model. After the defects of the finished product are identified, the staff will promptly check whether each process flow is normal, so as to timely find the source of the defects of the finished product and solve it, thereby improving the yield rate.

[0004] However, in the above-mentioned defect detection method, since there are many process flows, such as photolithography development, element molding, etc., and many process parameters, manual troubleshooting is required one by one. Therefore, after identifying defects in the finished product, back-checking whether there are problems in the process will lead to a large and cumbersome troubleshooting workload, resulting in extremely low efficiency of laser transfer printing. Therefore, there is an urgent need for a method to effectively improve the efficiency of laser transfer printing. Summary of the invention

[0005] The embodiments of the present application provide a laser transfer printing method and system based on deep learning image recognition, which are used to effectively improve the efficiency of laser transfer printing.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a laser transfer printing method based on deep learning image recognition is provided, which is applied to a laser transfer printing system, wherein the laser transfer printing system includes a central control host, a camera, and a narrow-band LED light source, wherein the central control host is deployed with a first deep learning model and a second deep learning model, and the method includes:

[0008] In the current graphic element process flow of laser transfer printing, the camera is used to obtain a live image in real time, wherein each graphic element process flow is provided with at least one camera, the graphic element process flow includes a photolithography development process, an electroforming process and a nickel stick graphic element lamination process, the live image of the photolithography development process is a first live image, the live image of the electroforming process is a second live image, and the live image of the nickel stick graphic element lamination process is a third live image;

[0009] After the current graphic element process, laser transfer printing is performed according to the laminated image obtained by the nickel stick graphic element laminated process, and the first scene image, the second scene image and the third scene image are respectively subjected to image recognition through the first deep learning model to obtain at least one first image feature;

[0010] According to the first image feature, a quality assessment step is performed to generate a quality assessment report for each of the graphic element process flows, wherein the quality assessment report includes potential defect information and image defect information for each of the graphic element process flows, wherein the image defect information is used to characterize the shooting quality of the camera;

[0011] Based on the image defect information, adjusting the wavelength and the angle of the narrow-band LED light source to improve the image recognition accuracy in the next image element process flow;

[0012] Inputting the potential defect information of each of the graphic element process flows and the acquired current process parameters of each of the graphic element process flows into the second deep learning model to obtain the abnormal process parameters of each of the graphic element process flows;

[0013] The current process parameters are adjusted according to the abnormal process parameters to obtain updated process parameters of each of the graphic element process flows, so that each of the graphic element process flows in the next graphic element process flow adopts the updated process parameters for laser transfer printing.

[0014] In a possible implementation of the first aspect, the laser transfer printing system further includes a display, the central control host further deploys a third deep learning model, and after the lamination image obtained according to the nickel-stick graphic element lamination process is laser transferred printed, the method further includes:

[0015] Inputting the image element product obtained after laser transfer printing into the third deep learning model to obtain a plurality of second image features;

[0016] Determining, based on the plurality of second image features, image element products having quality defects;

[0017] Using the decision tree algorithm, the image element products with quality defects are classified to obtain multiple defect categories;

[0018] Output product information of at least one graphic element product with quality defects corresponding to each defect class, and display the product information through the display.

[0019] In another possible implementation manner of the first aspect, performing image recognition on the first scene image, the second scene image, and the third scene image through the first deep learning model to obtain at least one first image feature includes:

[0020] Performing image recognition on the first on-site image through the first deep learning model to generate features of the photolithography image obtained according to the photolithography development process;

[0021] Performing image recognition on the second on-site image through the first deep learning model to generate thickness characteristics and uniformity characteristics of the nickel layer in the electroforming process;

[0022] Performing image recognition on the third scene image through the first deep learning model to generate integrity features of the lamination pattern obtained according to the nickel stick element lamination process;

[0023] The features of the photolithography image, the thickness features and uniformity features of the nickel layer, and the integrity features of the lamination pattern are all used to characterize the first image features.

[0024] In another possible implementation manner of the first aspect, the quality assessment step includes:

[0025] For each of the graphic element process flows, a preset defect recognition algorithm is used to determine defect features in features of the photolithography image, thickness features and uniformity features of the nickel layer, and integrity features of the lamination pattern;

[0026] Using a classifier, identifying the defect category corresponding to the defect feature, and using a sliding window technique to locate the defect position of the scene image, generating a mask of the defect position to mark the defect position;

[0027] Determining the image blur, noise level, and exposure uniformity based on the live image, the features of the photolithography image, the thickness features and uniformity features of the nickel layer, and the integrity features of the lamination pattern;

[0028] The defect category and the defect position are used to characterize the potential defect information of each of the graphic element process flows, and the image blur level, the noise level and the exposure uniformity are used to characterize the image defect information of each of the graphic element process flows.

[0029] In another possible implementation manner of the first aspect, the feature of the photolithography image includes a first image edge, the thickness feature and the uniformity feature of the nickel layer both include a second image edge, the integrity feature of the lamination pattern includes an image shape, and determining the image blur, noise level, and exposure uniformity according to the feature of the photolithography image, the thickness feature and the uniformity feature of the nickel layer, and the integrity feature of the lamination pattern includes:

[0030] Determine the blur level of the image by applying Laplace transform to the first image edge, the second image edge and the image shape;

[0031] Calculating the standard deviation and average pixel value of each of the scene images respectively, wherein the standard deviation is used to characterize the noise of the corresponding scene image;

[0032] Calculating the noise level of the corresponding scene image according to the average pixel value and the standard deviation;

[0033] The grayscale histogram of each of the scene images is calculated respectively, and the exposure uniformity of the corresponding scene image is determined through the grayscale histogram.

[0034] In another possible implementation manner of the first aspect, adjusting the wavelength and the angle of the narrow-band LED light source based on the image defect information includes:

[0035] In response to collecting live images at different wavelengths and angles, collecting statistics on image defect information of each of the live images at different wavelengths and angles of the narrow-band LED light sources;

[0036] Data fitting is performed on the relationship between the image defect information and the wavelength and angle to obtain a function f(λ, θ), and an optimization algorithm is used to determine λ and θ that minimize the function f(λ, θ), wherein λ represents the wavelength of the narrow-band LED light source, and θ represents the angle of the narrow-band LED light source.

[0037] In another possible implementation manner of the first aspect, performing data fitting on the relationship between the image defect information and the wavelength and angle to obtain the function f(λ, θ) includes:

[0038] Using a pre-trained data fitting model to perform data fitting on the relationship between the image defect information and the wavelength and angle, to obtain a function f(λ, θ);

[0039] Wherein, the data fitting model corresponding to each image defect information is different.

[0040] In another possible implementation of the first aspect, the data fitting model corresponding to the image blur degree is a polynomial regression model, the data fitting model corresponding to the noise level is a ridge regression model, and the data fitting model corresponding to the exposure uniformity is a gradient boosting regression model.

[0041] In a second aspect, the present application provides a laser transfer printing system based on deep learning image recognition, which is applied to the above-mentioned laser transfer printing method based on deep learning image recognition, including: a central control host, a camera and a narrow-band LED light source, wherein the camera and the narrow-band LED light source are both connected to the central control host;

[0042] In the current graphic element process flow of laser transfer printing, the camera is used to acquire on-site images in real time, wherein each graphic element process flow is provided with at least one camera, the graphic element process flow includes a photolithography development process, an electroforming process and a nickel stick graphic element lamination process, and the narrow-band LED light source is used to improve the image recognition accuracy in the next graphic element process flow.

[0043] In a possible implementation manner of the second aspect, an adjustable fiber optic coupler is further included, and the narrow-band LED light source is angle-adjusted through the adjustable fiber optic coupler.

[0044] Through the above technical solution, by acquiring real-time on-site images in each graphic element process and using the deep learning model for image recognition, the potential defects and image defect information of each process can be quickly detected. By generating a quality assessment report, the problems in each process can be effectively known, thereby effectively avoiding the tedious process of manual one-by-one investigation. In addition, by adjusting the wavelength and angle of the narrow-band LED light source, the accuracy of image recognition is improved, and the detection efficiency is further improved. Inputting the potential defect information and current process parameters of each process into the second deep learning model can quickly identify abnormal process parameters and make timely adjustments to ensure that the next process adopts the updated process parameters, thereby improving the overall printing quality and efficiency. Through systematic and automated detection and adjustment, this technical solution significantly reduces the need for manual intervention, greatly improves the efficiency and yield of laser transfer printing, and effectively solves the problem of inefficiency in existing laser transfer printing methods.

[0045] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a laser transfer printing process provided in an embodiment of the present application;

[0047] Figure 2A schematic diagram of another laser transfer printing process provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a process of a laser transfer printing method based on deep learning image recognition provided in an embodiment of the present application;

[0049] Figure 4 A structural block diagram of a laser transfer printing system based on deep learning image recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0051] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0052] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0053] In order to facilitate the understanding of this solution, the laser transfer printing process is first explained as follows:

[0054] Figure 1 A schematic diagram of a laser transfer printing process is shown, Figure 1 As shown, the laser transfer printing process includes the graphic element process and the printing process. For details, refer to Figure 2 , for the graphic element process flow, it includes plane image design, suspended graphic element lithography design, lithography, development, electroforming nickel roller and nickel roller graphic element lamination.

[0055] Among them, suspended element lithography design is the process of converting a plane image into a photolithography mask. Through the photolithography design software, the plane image is converted into a pattern suitable for the photolithography process, and a photolithography mask for forming a pattern on the substrate is generated; photolithography is the process of transferring the pattern from the photolithography mask to the substrate. The principle is to coat a layer of photoresist on the substrate, and then expose it through the photolithography mask to cause a chemical change in the photoresist. After exposure, the substrate is developed, and the photoresist in the unexposed part is removed, leaving the pattern; development is the process of removing the unexposed photoresist to reveal the pattern after photolithography. The principle is that the substrate is treated with a developer, the unexposed photoresist is dissolved and removed, and the exposed part forms the desired pattern. The electroformed nickel roller is a process of forming a nickel layer on a substrate through an electroforming process to enhance the durability and precision of the pattern. Specifically, electroforming is performed on the developed substrate, and nickel ions are deposited on the surface of the substrate under the action of an electric field to form a nickel roller. Nickel roller element lamination is the process of transferring the pattern on the nickel roller to the target substrate. Through the lamination process, the pattern on the nickel roller is accurately transferred to the target substrate to form the final pattern.

[0056] Reference Figure 2 , for the printing process, including PET base film, PET coating, constant temperature heat dissipation A, image element molding, rotation heat dissipation, constant temperature heat dissipation B, roller cold transmission, aluminizing, gluing, paper covering, constant temperature heat dissipation C, naked eye 3D image element products, and finished product winding.

[0057] Among them, the PET (polyethylene terephthalate) base film can provide a stable substrate as the basic material for printing. PET coating refers to uniformly coating the coating liquid on the PET base film and drying it so that the coating is evenly attached to the surface of the PET base film. Constant temperature heat dissipation A is used to perform constant temperature heat dissipation on the coated PET base film to ensure uniform drying of the coating and avoid temperature fluctuations affecting the coating quality. It controls the temperature through a constant temperature device to uniformly dissipate heat and ensure the consistency of coating drying. Element molding is used to transfer the element pattern from the nickel roller to the coated PET base film. The high-precision pattern on the nickel roller can be used to accurately transfer the pattern to the PET base film through a molding process. Rotational heat dissipation is used to quickly dissipate heat during the molding process by rotating the heat dissipation device to prevent heat accumulation from affecting the pattern quality. Constant temperature heat dissipation B is used to perform constant temperature heat dissipation again after molding to ensure pattern stability and coating quality. Roller cold transfer is used to transfer the molded PET base film to the next process step through a cooling roller and perform cooling treatment at the same time. Aluminum plating is used to perform aluminum plating on the PET base film to increase the reflectivity and visual effect of the film. Through the vacuum aluminum plating process, the aluminum layer is evenly plated on the PET base film to improve its optical performance. Gluing is used to apply a layer of glue on the PET base film after aluminum plating to facilitate the subsequent paper covering process. Paper covering is used to cover the paper on the PET base film after glue coating to form a composite material. The paper covering equipment can be used to accurately fit the paper and the PET base film to form a stable composite material. Constant temperature heat dissipation C is used to perform constant temperature heat dissipation after paper covering to ensure the stability of the composite material. The temperature can be controlled by the constant temperature equipment, the heat is evenly dissipated, and the composite material structure is stabilized. Naked eye 3D graphics are used to use special 3D processing technology to make the graphics have naked eye 3D effects and enhance the visual effect. Finished product rolling is used to roll the finished product after final processing into a roll for easy storage and transportation. The finished product material can be rolled into a roll through the rolling equipment to ensure its neatness and convenience for subsequent use.

[0058] Figure 3 The flowchart of a laser transfer printing method based on deep learning image recognition according to an embodiment of the present application is schematically shown. Figure 3 As shown, based on the above-mentioned laser transfer printing process, an embodiment of the present application provides a laser transfer printing method based on deep learning image recognition, which is applied to a laser transfer printing system. The laser transfer printing system includes a central control host, a camera and a narrow-band LED light source. The central control host is deployed with a first deep learning model and a second deep learning model. The method may include the following steps.

[0059] S110, in the current graphic element process flow of laser transfer printing, a live image is acquired in real time through a camera, wherein each graphic element process flow is provided with at least one camera, the graphic element process flow includes a photolithography development process, an electroforming process, and a nickel stick graphic element lamination process, the live image of the photolithography development process is a first live image, the live image of the electroforming process is a second live image, and the live image of the nickel stick graphic element lamination process is a third live image;

[0060] S120, after the current graphic element process, laser transfer printing is performed according to the laminated image obtained by the nickel stick graphic element laminated process, and the first scene image, the second scene image, and the third scene image are respectively subjected to image recognition through a first deep learning model to obtain at least one first image feature;

[0061] S130, performing a quality assessment step according to the first image feature to generate a quality assessment report for each graphic element process flow, wherein the quality assessment report includes potential defect information and image defect information of each graphic element process flow, wherein the image defect information is used to characterize the shooting quality of the camera;

[0062] S140, adjusting the wavelength and angle of the narrow-band LED light source based on the image defect information to improve the image recognition accuracy in the next graphic element process flow;

[0063] S150, inputting the potential defect information of each graphic element process and the acquired current process parameters of each graphic element process into a second deep learning model to obtain abnormal process parameters of each graphic element process;

[0064] S160, adjusting the current process parameters according to the abnormal process parameters to obtain updated process parameters for each graphic element process flow, so that each graphic element process flow in the next graphic element process flow adopts the updated process parameters for laser transfer printing.

[0065] In this embodiment, at least one camera is provided in each graphic element process flow, which can ensure that high-quality images can be captured in different process flows. The graphic element process flow includes a photolithography development process, an electroforming process, and a nickel stick graphic element lamination process. The on-site image of the photolithography development process is the first on-site image, the on-site image of the electroforming process is the second on-site image, and the on-site image of the nickel stick graphic element lamination process is the third on-site image. By acquiring the on-site image in real time through the camera, each process of the graphic element process flow can be monitored and data collected in real time, providing an accurate data basis for subsequent image recognition and quality assessment.

[0066] After the current graphic element process, laser transfer printing is performed according to the laminated image obtained by the nickel stick graphic element lamination process, and the first scene image, the second scene image and the third scene image are respectively subjected to image recognition by the first deep learning model to obtain at least one first image feature. The first deep learning model is pre-trained and can accurately recognize the image features (first image features) in the current graphic element process, such as the integrity of the pattern, the clarity of the edge, the contrast of the image, etc.

[0067] Afterwards, a quality assessment step is performed based on the first image feature to generate a quality assessment report for each primitive process. The quality assessment report includes potential defect information and image defect information for each primitive process. Potential defect information may include defect categories such as pattern loss and pattern deformation, while image defect information is used to characterize the camera's shooting quality, such as image resolution, contrast, noise, etc. Through quality assessment, potential problems in the process can be discovered and corrected in a timely manner to ensure the quality and consistency of each process.

[0068] In actual production, due to the differences in printed patterns, light sources with fixed wavelengths and angles are not suitable. That is to say, if the wavelength and angle of the light source are fixed, for different printed patterns, the camera's shooting effect of the live image will not be good due to exposure and other reasons. Therefore, the embodiment of the present application uses a narrow-band LED light source with adjustable wavelength and angle to improve the image quality captured by the camera.

[0069] Specifically, the wavelength and angle of the narrowband LED light source can be adjusted based on the image defect information to improve the image recognition accuracy in the next primitive process. Light sources of different wavelengths have different effects on image capture of different materials. By adjusting the wavelength of the light source, the contrast and clarity of the image can be improved. At the same time, adjusting the angle of the light source can reduce the interference of reflected light and improve the overall quality of the image. The above adjustments can be precisely controlled according to the image defect information of the current process flow to ensure that the camera can capture higher quality images in the next process flow, thereby improving the accuracy of image recognition.

[0070] Next, the potential defect information of each primitive process and the current process parameters of each primitive process are input into the second deep learning model to obtain the abnormal process parameters of each primitive process. The second deep learning model has been pre-trained to identify and analyze abnormal process parameters in the primitive process, such as primitive spacing, primitive size, photolithography exposure time, alignment accuracy in the photolithography process, development time, development temperature, electroforming temperature, lamination pressure, etc.

[0071] Adjust the current process parameters according to the abnormal process parameters to obtain updated process parameters for each element process flow, so that each element process flow in the next element process flow adopts the updated process parameters for laser transfer printing. By adjusting the process parameters, abnormal conditions in the process flow can be corrected to ensure the stability and consistency of each process flow. For example, if it is found that the temperature in a process flow is too high and causes the pattern to deform, this problem can be corrected by adjusting the temperature parameters. The updated process parameters can ensure that in the next process flow, laser transfer printing can achieve higher quality and consistency.

[0072] This embodiment can quickly detect the potential defects and image defect information of each process flow by acquiring the on-site image in real time in each graphic element process flow and using the deep learning model for image recognition. By generating a quality assessment report, the problems in each process flow can be effectively known, thereby effectively avoiding the tedious process of manual one-by-one investigation. In addition, by adjusting the wavelength and angle of the narrow-band LED light source, the accuracy of image recognition is improved, and the detection efficiency is further improved. The potential defect information and current process parameters of each process flow are input into the second deep learning model, which can quickly identify abnormal process parameters and make timely adjustments to ensure that the next process flow adopts the updated process parameters, thereby improving the overall printing quality and efficiency. Through systematic and automated detection and adjustment, this technical solution significantly reduces the need for manual intervention, greatly improves the efficiency and yield of laser transfer printing, and effectively solves the problem of inefficiency in existing laser transfer printing methods.

[0073] In one implementation of this embodiment, the laser transfer printing system further includes a display, and the central control host further deploys a third deep learning model. After laser transfer printing is performed on the lamination image obtained according to the nickel stick element lamination process, the following steps are further included:

[0074] S210, inputting the image element product obtained after laser transfer printing into a third deep learning model to obtain a plurality of second image features;

[0075] S220, determining a graphic element product having quality defects according to the plurality of second image features;

[0076] S230, using a decision tree algorithm to classify the graphic element products with quality defects to obtain multiple defect classes;

[0077] S240: Output product information of at least one graphic element product with quality defects corresponding to each defect class, and display the product information on a display.

[0078] After laser transfer printing, the resulting primitive product is input into the third deep learning model to obtain multiple second image features. The third deep learning model is pre-trained to recognize and extract various image features of the primitive product, such as surface smoothness, edge integrity, color consistency, etc. For example, if there are slight scratches or color deviations on the surface of the primitive product, the third deep learning model can accurately recognize and record the image features. In this way, it can ensure that the quality characteristics of each primitive product are fully captured and analyzed.

[0079] After obtaining a plurality of second image features, quality defects in the image element product can be identified by analyzing the second image features. For example, if the edge of the image element product is incomplete or there are flaws on the surface, the above features are marked as quality defects.

[0080] Specifically, the third deep learning model can be used to determine the image element products with quality defects based on the second image features. That is, the third deep learning model can identify the image to obtain the second image features, and in addition, it can distinguish between normal products and image element products with quality defects based on the second image features.

[0081] The third deep learning model can be a convolutional neural network, which can extract features in the image, such as shape features, texture features, and color features. Shape features can be edge length, edge direction, shape contour, etc.; texture features can be grayscale co-occurrence matrix, local binary pattern, etc.; color features can be color histogram, color mean, etc.

[0082] First, the third deep learning model is trained, and after the third deep learning model is trained, multiple second image features can be input into the third deep learning model to perform defect detection and determine the image element products with quality defects.

[0083] Afterwards, the decision tree algorithm is used to classify the image element products with quality defects and obtain multiple defect classes. Among them, the decision tree algorithm is a commonly used classification method. By dividing the various features of the image element products layer by layer, different types of defects can be effectively classified. For example, different types of defects such as surface scratches, edge damage, and color deviation can be classified separately. The decision tree algorithm can clearly display the division basis and characteristics of each defect class through a tree structure.

[0084] Product information can include detailed information such as the number of the graphic element product, defect type, defect location, and defect degree. The display can display product information in real time, which is convenient for operators to view and process. For example, operators can quickly locate and process defective graphic elements based on the information on the display to ensure efficient and smooth production process. At the same time, through the detailed recording and display of defect information, data support can be provided for subsequent quality improvement and process optimization.

[0085] This embodiment can accurately identify and extract the image features of the image element product through the application of the third deep learning model, providing detailed data support for quality inspection. Through the application of the decision tree algorithm, different types of defects can be effectively classified to provide a basis for defect processing. Through the real-time display of the display, it is convenient for operators to view and process, improving the efficiency and quality of the production process.

[0086] In one implementation of this embodiment, performing image recognition on the first scene image, the second scene image, and the third scene image through a first deep learning model to obtain at least one first image feature includes the following steps:

[0087] S310, performing image recognition on the first scene image through a first deep learning model to generate features of a lithography image obtained according to a lithography development process;

[0088] S320, performing image recognition on the second on-site image through the first deep learning model to generate thickness characteristics and uniformity characteristics of the nickel layer in the electroforming process;

[0089] S330, performing image recognition on the third scene image through the first deep learning model to generate integrity features of the lamination pattern obtained according to the nickel stick element lamination process;

[0090] The features of the photolithography image, the thickness features and uniformity features of the nickel layer, and the integrity features of the lamination pattern are all used to characterize the first image features.

[0091] The first scene image is a lithography image obtained through the lithography development process. Lithography development is a micro-fabrication technology used to form fine patterns on the surface of a wafer. In order to ensure the accuracy and consistency of the pattern, the first scene image is input into the first deep learning model, which can generate features of the lithography image through steps such as preprocessing, feature extraction, and classification.

[0092] Among them, the preprocessing steps include grayscale, normalization and denoising of the image to improve the quality of the image and the recognition accuracy of the model. Grayscale is to convert the color image into a grayscale image to reduce the computational complexity; normalization is to standardize the image pixel values ​​to eliminate the influence of illumination changes; denoising is to use filters to remove noise in the image and enhance the image clarity.

[0093] In the feature extraction stage, the first deep learning model can automatically extract key features in the lithography image through a convolutional neural network (CNN). Key features may include edge information of the pattern, line width, symmetry of the pattern, etc. The convolutional neural network extracts high-level features of the image layer by layer through multi-layer convolution and pooling operations and generates a feature vector.

[0094] Finally, the first deep learning model classifies and identifies the image based on the extracted features and generates features of the lithography image. The features of the lithography image can be used to evaluate the quality and consistency of the lithography pattern and ensure the accuracy of the lithography development process. For example, if the edge information of an image is unclear or the line width does not meet the standard, the first deep learning model can identify and mark the abnormal features.

[0095] The second field image is an image of the nickel layer obtained through the electroforming process. Electroforming is an electrochemical deposition technology used to form a uniform nickel layer on the surface of the substrate. In order to ensure the thickness and uniformity of the nickel layer, it is necessary to analyze the image after electroforming. The second field image is input into the first deep learning model. Similar to the first deep learning model generating the features of the photolithography image obtained according to the photolithography development process, in this embodiment, the first deep learning model can generate the thickness features and uniformity features of the nickel layer through preprocessing, feature extraction and classification.

[0096] The first deep learning model can classify and identify images based on the extracted features, generate thickness features and uniformity features of the nickel layer to evaluate the quality and consistency of the electroforming process and ensure that the thickness and uniformity of the nickel layer meet the standards. For example, if the thickness distribution of an image is uneven or there are obvious particles on the surface, the first deep learning model can identify and mark the abnormal features.

[0097] The third scene image is a die-stamped pattern obtained through the nickel stick element die-stamping process. Nickel stick element die-stamping is a technology used to form fine patterns on the surface of a substrate. Ensuring the integrity and clarity of the pattern is key. In order to evaluate the quality of the die-stamped pattern, a detailed analysis of the image after die-stamping is required. The third scene image is input into the first deep learning model, which generates the integrity features of the die-stamped pattern through preprocessing, feature extraction, and classification.

[0098] This implementation can comprehensively and accurately identify and extract key image features in the photolithography development, electroforming, and nickel stick element lamination processes. The first deep learning model can automatically extract and analyze key features in the image to ensure the quality and consistency of each process. It not only improves the efficiency and accuracy of quality inspection, but also reduces the workload and misjudgment rate of manual inspection. Through a comprehensive analysis of the photolithography image features, nickel layer thickness and uniformity features, and lamination pattern integrity features, it can provide strong data support for subsequent quality control and process optimization, significantly improving the intelligence and automation level of the production process.

[0099] In one implementation of this embodiment, the quality assessment step includes the following steps:

[0100] S410, for each graphic element process flow, using a preset defect recognition algorithm to determine defect features in features of the photolithography image, thickness features and uniformity features of the nickel layer, and integrity features of the lamination pattern;

[0101] S420, using a classifier to identify a defect category corresponding to the defect feature, and using a sliding window technique to locate the defect position of the on-site image, and generating a mask of the defect position to mark the defect position;

[0102] S430, determining the image blur level, noise level, and exposure uniformity according to the features of the field image, the photolithography image, the thickness features and uniformity features of the nickel layer, and the integrity features of the lamination pattern;

[0103] Among them, the defect category and defect location are used to characterize the potential defect information of each primitive process, and the image blur degree, noise level and exposure uniformity are used to characterize the image defect information of each primitive process.

[0104] First, it is necessary to identify defects in each primitive process. The primitive process includes steps such as photolithography development, electroforming, and nickel stick primitive lamination. In order to identify the defect characteristics in the primitive process, a preset defect recognition algorithm can be used. The defect recognition algorithm is usually based on machine learning or deep learning technology, which can automatically detect and classify defects in the image. In specific implementation, the photolithography image, nickel layer image, and lamination pattern image are first preprocessed, including grayscale, normalization, and noise reduction, to improve the image quality and the recognition accuracy of the algorithm.

[0105] In the photolithography process, the defect recognition algorithm identifies possible defect features by analyzing the features of the photolithography image, such as edge information, line width, and pattern symmetry. For example, if the edges of a certain area are blurred or the line width is uneven, the defect recognition algorithm can mark it as a defect. In the electroforming process, the defect recognition algorithm identifies areas with uneven thickness or rough surfaces by analyzing the thickness characteristics and uniformity characteristics of the nickel layer. In the nickel stick element lamination process, the defect recognition algorithm can identify areas where the pattern is broken or missing by analyzing the integrity characteristics of the lamination pattern.

[0106] Defect recognition algorithms usually include two stages: feature extraction and classification. In the feature extraction stage, the defect recognition algorithm can extract key features in the image and generate feature vectors. In the classification stage, the defect recognition algorithm can classify the image according to the feature vector and identify the defect features.

[0107] The identified defect features are further analyzed using a classifier to determine the defect category. The classifier can be a support vector machine (SVM), which classifies the defect features according to preset defect categories, such as blurred edges, uneven thickness, pattern breakage, etc. In specific implementation, the defect features are first input into the classifier, which classifies the defects according to the feature vector and outputs the defect category.

[0108] In order to accurately locate the defect, the sliding window technology is used to scan the on-site image. The sliding window technology is a commonly used image processing method that moves a fixed-size window on the on-site image and analyzes each area one by one. In each window area, the classifier classifies the image and identifies the defect features. In this way, the position of the defect in the image can be accurately located.

[0109] Once the defect location is determined, a mask of the defect location is generated. A mask is a binary image where the defect location is marked as 1 and other locations are marked as 0. The mask can be used to visualize the defect location and help engineers quickly locate and repair the defect. For example, if the mask value of a certain area is 1, it means that there is a defect in that area, and engineers can focus on inspecting and repairing that area.

[0110] Through the classifier and sliding window technology, the defect features can be accurately identified and located, and a mask of the defect location can be generated. This not only improves the accuracy of defect detection, but also reduces the workload and misjudgment rate of manual inspection, providing strong data support for subsequent quality control.

[0111] The degree of image blur can be determined by analyzing the edge clarity and contrast of the image. Edge clarity can be evaluated by calculating the image gradient. Contrast can be evaluated by calculating the brightness histogram of the image. Blurred images usually have unclear edges and low contrast, and these indicators can be used to quantify the degree of blur in the image.

[0112] The noise level can be determined by the average pixel value and standard deviation of the scene image. The average pixel value represents the overall brightness level of the image and is the average of all pixel values. The standard deviation is used to describe the discreteness of the pixel value, that is, the degree to which the pixel value deviates from the average value. The larger the standard deviation, the greater the fluctuation of the pixel value and the higher the noise level.

[0113] Exposure uniformity can be determined by analyzing the brightness distribution of an image. An evenly exposed image has a smoother brightness distribution and higher consistency, whereas an unevenly exposed image has an uneven brightness distribution and may have overexposed or underexposed areas. Exposure uniformity can be evaluated by calculating the brightness histogram of the image.

[0114] The blur level, noise level and exposure uniformity parameters of the image can be used to evaluate the quality of the image and the stability of the process flow, ensuring that the image quality of each image element process meets the standard. For example, if an image has a high degree of blur or a high noise level, it may be necessary to adjust the process parameters or perform equipment maintenance to improve image quality and process stability.

[0115] This implementation can comprehensively and accurately identify and evaluate the defect characteristics and image quality in each primitive process. The combination of defect recognition algorithm and classifier makes defect detection more automated and efficient, reducing the workload and misjudgment rate of manual detection. The application of sliding window technology ensures the precise positioning of defect positions and provides strong data support for subsequent defect repair. Through detailed analysis of image blur, noise level and exposure uniformity, the stability of image quality and process flow can be evaluated to ensure that the image quality of each process flow meets the standards. It not only improves the efficiency and accuracy of quality inspection, but also facilitates process optimization and quality control.

[0116] In one implementation of this embodiment, the feature of the photolithography image includes a first image edge, the thickness feature and the uniformity feature of the nickel layer both include a second image edge, and the integrity feature of the lamination pattern includes an image shape. According to the feature of the photolithography image, the thickness feature and the uniformity feature of the nickel layer, and the integrity feature of the lamination pattern, determining the image blur, the noise level, and the exposure uniformity includes the following steps:

[0117] S510, using Laplace transform on the first image edge, the second image edge and the image shape to determine the image blur degree;

[0118] S520, respectively calculating the standard deviation and average pixel value of the pixel values ​​of each scene image, where the standard deviation is used to characterize the noise of the corresponding scene image;

[0119] S530, calculating the noise level of the corresponding scene image according to the average pixel value and the standard deviation;

[0120] S540, respectively calculating the grayscale histogram of each scene image, and determining the exposure uniformity of the corresponding scene image through the grayscale histogram.

[0121] The blur degree of the image is determined by applying Laplace transform to the first image edge, the second image edge and the image shape. Laplace transform is a second-order differential operator used to detect edges in an image. By calculating the Laplace transform of an image, high-frequency components in the image can be obtained, and these high-frequency components usually correspond to the edges and details of the image. In the implementation process, the edges in the lithography image and the nickel layer image are first extracted to obtain the first image edge and the second image edge. Then, Laplace transform is applied to these edge images and the shape image of the lamination pattern to calculate their Laplace operator values.

[0122] In practice, the Laplace transform can be calculated using the following formula:

[0123]

[0124] Among them, I(x,y) represents the grayscale value of the image at the position (x,y), and L(x,y) represents the value after Laplace transformation. By calculating the image after Laplace transformation, the edge and detail information in the image can be obtained. If the image is blurred, the edge and detail information will be weakened, and the value after Laplace transformation will be smaller. By counting the values ​​after Laplace transformation, the blurriness of the image can be quantified. For example, the mean value or variance of the values ​​after Laplace transformation can be calculated as an indicator of the blurriness of the image.

[0125] Afterwards, the standard deviation and average pixel value of the pixel values ​​of each scene image can be calculated to characterize the noise level of the scene image. The standard deviation is a statistic used to describe the degree of dispersion of data. For scene images, the standard deviation can reflect the degree of fluctuation of pixel values. The greater the fluctuation, the higher the noise level. The average pixel value is used to describe the overall brightness level of the image.

[0126] In the specific implementation, each scene image is first grayed out to convert the color image into a grayscale image. Then, the average value and standard deviation of the pixel values ​​of each grayscale image are calculated. The average value can be calculated by the following formula:

[0127]

[0128] Where μ represents the average pixel value, N represents the total number of pixels in the image, and I_ i represents the gray value of the i-th pixel. The standard deviation can be calculated by the following formula:

[0129]

[0130] Among them, σ represents the standard deviation, μ represents the average pixel value, and I_ i Represents the gray value of the i-th pixel.

[0131] By calculating the average pixel value and standard deviation of each live image, the noise level of the image can be quantified. The larger the standard deviation, the more noise there is in the image and the worse the image quality. In this way, the noise level of the image can be accurately assessed, which helps to identify and deal with images with large noise.

[0132] The noise level of the image is calculated based on the average pixel value and standard deviation of each live image. The noise level is an important indicator for evaluating image quality and reflects the degree of random noise in the image. By combining the average pixel value and standard deviation, the noise level of the image can be quantified more accurately.

[0133] In practice, the noise level can be calculated using the following formula:

[0134]

[0135] Among them, σ represents the standard deviation and μ represents the average pixel value. This formula represents the ratio of the standard deviation to the average pixel value, reflecting the proportion of random noise in the image relative to the overall brightness. The higher the noise level, the more random noise there is in the image, and the worse the image quality.

[0136] This embodiment can quantify the degree of random noise in the image by calculating the noise level of each live image. For example, if the noise level of an image is high, denoising may be required to improve the image quality. Denoising can use image processing techniques such as median filtering and Gaussian filtering to remove random noise in the image and enhance the clarity and details of the image.

[0137] Next, the grayscale histogram of each live image is calculated and the exposure uniformity of the image is determined by the grayscale histogram. The grayscale histogram is used to describe the distribution of pixels at different grayscale levels in the image. By analyzing the grayscale histogram, the brightness distribution and exposure uniformity of the image can be evaluated.

[0138] In the specific implementation, each live image is first grayed to convert the color image into a gray image. Then, the gray histogram of each gray image is calculated. The gray histogram can be generated by counting the number of pixels at each gray level in the image. For example, for an 8-bit gray image, the gray level ranges from 0 to 255, with a total of 256 gray levels. Each value in the gray histogram represents the number of pixels at the corresponding gray level.

[0139] By analyzing the grayscale histogram, you can evaluate the exposure uniformity of an image. For an evenly exposed image, the grayscale histogram distribution is relatively smooth, and the number of pixels at each grayscale level is relatively uniform. If the image is unevenly exposed, the grayscale histogram may have obvious peaks or valleys, indicating that the number of pixels at certain grayscale levels is abnormally high or low. For example, if the grayscale histogram has peaks at high grayscale levels, it means that there are overexposed areas in the image; if it has peaks at low grayscale levels, it means that there are underexposed areas in the image.

[0140] In this embodiment, the exposure uniformity of the image can be accurately evaluated, and images with uneven exposure can be identified and processed. For example, if the exposure uniformity of an image is poor, it may be necessary to adjust the exposure parameters or perform image enhancement processing to improve the image quality. By analyzing the grayscale histogram, the exposure uniformity of the image can be quantified to ensure that the image quality of each process meets the standard and reduce the defect rate in the production process.

[0141] This implementation can comprehensively and accurately evaluate the blur level, noise level and exposure uniformity of each on-site image. Laplace transform is used to detect edge and detail information in the image and quantify the blur level of the image; standard deviation and average pixel value are used to quantify the noise level of the image; grayscale histogram is used to evaluate the exposure uniformity of the image, which is conducive to accurately identifying and processing blurry, noisy and unevenly exposed images, effectively improving the efficiency and accuracy of quality inspection.

[0142] In one implementation of this embodiment, adjusting the wavelength and the angle of the narrow-band LED light source based on the image defect information includes the following steps:

[0143] S610, in response to collecting live images at different wavelengths and angles, counting image defect information of each live image at different wavelengths and angles of narrow-band LED light sources;

[0144] S620, performing data fitting on the relationship between the image defect information and the wavelength and angle to obtain the function f(λ, θ), and determining λ and θ that minimize the function f(λ, θ) through an optimization algorithm, where λ represents the wavelength of the narrow-band LED light source, and θ represents the angle of the narrow-band LED light source.

[0145] First, it is necessary to collect images of the scene at different wavelengths and angles. Specifically, a group of narrowband LED light sources can be used, which can emit light of different wavelengths. Each narrowband LED light source can precisely control its light-emitting angle to illuminate the target object at different angles. When collecting images, it should be ensured that each wavelength and angle combination can cover all key areas of the target object. In specific implementation, an automated rotating platform can be used to adjust the angle of the light source, and a wavelength-adjustable LED driver can be used to change the wavelength of the light source. After each adjustment of the wavelength and angle, a high-resolution camera is used to capture an image and record the current wavelength and angle parameters.

[0146] Next, each captured image needs to be analyzed to count the defect information in the image. Image defect information can include various indicators such as uneven exposure, color difference, blur, noise level, etc. These defect information can be extracted through image processing algorithms such as edge detection, histogram equalization, Fourier transform, etc. For example, the edge detection algorithm can be used to identify the blurred area in the image, and the histogram equalization can be used to analyze the uneven exposure. The image defect information is recorded with the corresponding wavelength and angle parameters to form a data set.

[0147] After that, it is necessary to perform data fitting on the relationship between the image defect information and the wavelength and angle. First, the data set obtained in step S610 is input into the data fitting model to obtain a function f(λ, θ), which describes the relationship between the wavelength and angle on the image defect. Specifically, a polynomial regression model can be selected, with the wavelength and angle as independent variables and the image defect information as the dependent variable, and the data can be fitted by an optimization algorithm such as the least squares method to obtain the specific form of the function.

[0148] Next, an optimization algorithm is used to determine the wavelength λ and angle θ that minimize the function f(λ, θ). The optimization algorithm can be a gradient descent method, etc. Taking the gradient descent method as an example, first, the gradient of the function f(λ, θ) needs to be calculated, that is, the partial derivatives of the wavelength and angle are calculated respectively, and then the wavelength and angle are gradually adjusted in the opposite direction of the gradient until the parameter combination that minimizes the function value is found. This process can be automatically completed using a computer program, and the optimal wavelength and angle parameters are finally obtained through iterative calculation and adjustment.

[0149] This implementation can effectively determine the optimal wavelength and angle of the narrow-band LED light source, minimize the defect information in the collected on-site images, significantly improve the image quality, reduce defects such as noise, blur and uneven brightness in the image, thereby improving the clarity and accuracy of the image, thereby improving detection accuracy and reliability, and reducing the risk of misjudgment and missed judgment.

[0150] In one implementation of this embodiment, data fitting is performed on the relationship between image defect information and wavelength and angle to obtain a function f(λ, θ), including the following steps:

[0151] S710, using a pre-trained data fitting model to perform data fitting on the relationship between image defect information and wavelength and angle to obtain a function f(λ, θ);

[0152] Among them, the data fitting model corresponding to each image defect information is different.

[0153] The pre-trained data fitting model is used to perform data fitting on the relationship between image defect information and wavelength and angle to obtain the function f(λ, θ).

[0154] In the specific implementation, a large amount of sample data is first collected. The sample data includes images at different wavelengths and angles, as well as the image defect information corresponding to the images. The sample data is divided into a training set and a test set. The training set is used to train the model, and the test set is used to verify the accuracy of the data fitting model. For each type of image defect information, a suitable fitting model is selected and trained using the training set data.

[0155] After the data fitting model is trained, the trained data fitting model is applied to the test set data to evaluate the fitting effect of the data fitting model. Evaluation indicators can include mean square error (MSE), determination coefficient (R 2 ), etc. The above indicators can quantify the fitting accuracy of the data fitting model. By adjusting the model parameters and selecting different model structures, the fitting effect of the model can be gradually improved to ensure that each image defect information can be accurately fitted.

[0156] In practical applications, when new image data is collected, it can be input into the pre-trained data fitting model to obtain the influence function f(λ, θ) of wavelength and angle on image defect information. The f(λ, θ) function describes the change law of image defect information at different wavelengths and angles.

[0157] This implementation method uses a pre-trained data fitting model to efficiently fit the relationship between image defect information and wavelength and angle, and obtains the function f(λ, θ). This not only improves the accuracy of the fitting, but also simplifies the data processing process, so that in practical applications, the function of the influence of wavelength and angle on image defect information can be quickly obtained, thereby further improving image quality and enhancing detection accuracy and reliability.

[0158] In one implementation of this embodiment, the data fitting model corresponding to the image blur degree is a polynomial regression model, the data fitting model corresponding to the noise level is a ridge regression model, and the data fitting model corresponding to the exposure uniformity is a gradient boosting regression model.

[0159] First, a polynomial regression model is used to fit the image blur level. The polynomial regression model is an extended linear regression model that captures the nonlinear relationship in the data by introducing higher-order terms. Assume that the relationship between the image blur level M and the wavelength λ and angle θ can be expressed as a quadratic polynomial function: M = aλ 2 +bθ 2 +cλθ+dλ+eθ+f, where a, b, c, d, e and f are model parameters. By collecting a large amount of sample data, the model is trained using the least squares method to minimize the sum of squared errors between the predicted value and the actual value. After the training is completed, the obtained polynomial function can accurately describe the effect of wavelength and angle on the degree of image blur.

[0160] Secondly, the ridge regression model is used to fit the noise level. Ridge regression is an improved linear regression model that prevents overfitting by adding a regularization term to the loss function. Assume that the relationship between the noise level N and the wavelength λ and angle θ can be expressed as a linear function: N = gλ + hθ + i, where g, h and i are model parameters. Ridge regression adds the sum of squares of the parameters as a regularization term on the basis of minimizing the sum of squared errors:

[0161]

[0162] Among them, α is the regularization parameter, which is used to control the weight of the regularization term. By adjusting the value of α, a balance can be achieved between fitting accuracy and model complexity. After training, the obtained ridge regression model can effectively prevent overfitting and accurately describe the impact of wavelength and angle on noise level.

[0163] Finally, for exposure uniformity, a gradient boosting regression model is used for fitting. Gradient boosting regression is an ensemble learning method that improves the prediction ability of the model by combining multiple weak learners (such as decision trees). It is assumed that the relationship between exposure uniformity E and wavelength λ and angle θ can be represented by the weighted sum of multiple decision trees:

[0164]

[0165] Among them, T k represents the kth decision tree, ω krepresents its corresponding weight, and K is the number of decision trees. By gradually building decision trees and minimizing the residual sum of squares at each step, the prediction ability of the model can be gradually improved. After training, the obtained gradient boosting regression model can accurately describe the impact of wavelength and angle on exposure uniformity.

[0166] This embodiment uses a polynomial regression model, a ridge regression model, and a gradient boosting regression model to fit the image blur, noise level, and exposure uniformity, respectively, and can accurately describe the effects of wavelength and angle on these image defect information. The polynomial regression model can capture the nonlinear relationship in the image blur, the ridge regression model prevents overfitting through regularization, and the gradient boosting regression model improves prediction capabilities through ensemble learning. Ultimately, these fitting models provide an accurate description of the image defect information in terms of wavelength and angle, and effectively optimize the light source parameters, thereby significantly improving image quality, reducing image defects, and improving detection accuracy and reliability.

[0167] Reference Figure 4 , the embodiment of the present application also provides a laser transfer printing system based on deep learning image recognition, which is applied to the above-mentioned laser transfer printing method based on deep learning image recognition, including: a central control host, a camera and a narrow-band LED light source, the camera and the narrow-band LED light source are both connected to the central control host;

[0168] In the current graphic element process flow of laser transfer printing, the camera is used to obtain on-site images in real time. There is at least one camera in each graphic element process flow. The graphic element process flow includes photolithography development process, electroforming process and nickel stick graphic element lamination process. The narrow-band LED light source is used to improve the image recognition accuracy in the next graphic element process flow.

[0169] In this embodiment, the laser transfer printing system based on deep learning image recognition also includes an adjustable fiber optic coupler, and the narrow-band LED light source achieves angle adjustment through the adjustable fiber optic coupler.

[0170] In different graphic element process flows, the required light source wavelength may be different, so the wavelength of the narrow-band LED light source of this embodiment can be flexibly adjusted. By adjusting the wavelength of the narrow-band LED light source, the lighting effect of the object can be optimized, allowing the camera to capture more details. For example, in the photolithography development process, using a light source with a specific wavelength can highlight the details of the pattern after development, while in the electroforming process, using a light source with a different wavelength can better capture the surface features of the electroforming layer.

[0171] Specifically, the angle of the narrowband LED light source can be adjusted through an adjustable fiber coupler, thereby optimizing the irradiation angle of the light source so that the light can be irradiated onto the subject at the best angle, thereby minimizing shadows and reflections and improving the contrast and clarity of the image.

[0172] Among them, the adjustable fiber coupler is an optical device, which is mainly used to effectively couple the light emitted by the light source into the optical fiber, and the angle of the optical fiber can be adjusted to change the emission direction of the light. The narrowband LED light source is connected to the adjustable fiber coupler through an optical fiber. The specific connection method is as follows: the narrowband LED light source emits light of a specific wavelength; one end of the optical fiber is connected to the output end of the narrowband LED light source, and the optical fiber transmits the light to the coupler; the other end of the optical fiber is connected to the input end of the adjustable fiber coupler. By adjusting the angle of the optical fiber, the irradiation direction of the light can be optimized. For example, in the photolithography development process, adjusting the angle of the optical fiber so that the light is evenly irradiated on the photoresist layer can reduce shadows and reflections and improve the contrast and clarity of the image.

[0173] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned laser transfer printing method based on deep learning image recognition.

[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0175] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0179] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0180] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0181] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0182] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A laser transfer printing method based on deep learning image recognition, characterized in that: Applied to a laser transfer printing system, the laser transfer printing system includes a central control host, a camera and a narrow-band LED light source, the central control host is deployed with a first deep learning model and a second deep learning model, and the method includes: In the current graphic element process flow of laser transfer printing, the camera is used to obtain a live image in real time, wherein each graphic element process flow is provided with at least one camera, the graphic element process flow includes a photolithography development process, an electroforming process and a nickel stick graphic element lamination process, the live image of the photolithography development process is a first live image, the live image of the electroforming process is a second live image, and the live image of the nickel stick graphic element lamination process is a third live image; After the current graphic element process, laser transfer printing is performed according to the laminated image obtained by the nickel stick graphic element laminated process, and the first scene image, the second scene image and the third scene image are respectively subjected to image recognition through the first deep learning model to obtain at least one first image feature; According to the first image feature, a quality assessment step is performed to generate a quality assessment report for each of the graphic element process flows, wherein the quality assessment report includes potential defect information and image defect information for each of the graphic element process flows, wherein the image defect information is used to characterize the shooting quality of the camera; Based on the image defect information, adjusting the wavelength and the angle of the narrow-band LED light source to improve the image recognition accuracy in the next image element process flow; Inputting the potential defect information of each of the graphic element process flows and the acquired current process parameters of each of the graphic element process flows into the second deep learning model to obtain the abnormal process parameters of each of the graphic element process flows; The current process parameters are adjusted according to the abnormal process parameters to obtain updated process parameters of each of the graphic element process flows, so that each of the graphic element process flows in the next graphic element process flow adopts the updated process parameters for laser transfer printing.

2. The method according to claim 1, characterized in that: The laser transfer printing system further includes a display, and the central control host further deploys a third deep learning model. After the lamination image obtained according to the nickel stick graphic element lamination process is laser transferred and printed, the method further includes: Inputting the image element product obtained after laser transfer printing into the third deep learning model to obtain a plurality of second image features; Determining, based on the plurality of second image features, image element products having quality defects; Using the decision tree algorithm, the image element products with quality defects are classified to obtain multiple defect categories; Output product information of at least one graphic element product with quality defects corresponding to each defect class, and display the product information through the display.

3. The method according to claim 1, characterized in that The performing image recognition on the first scene image, the second scene image, and the third scene image through the first deep learning model to obtain at least one first image feature includes: Performing image recognition on the first on-site image through the first deep learning model to generate features of the photolithography image obtained according to the photolithography development process; Performing image recognition on the second on-site image through the first deep learning model to generate thickness characteristics and uniformity characteristics of the nickel layer in the electroforming process; Performing image recognition on the third scene image through the first deep learning model to generate integrity features of the lamination pattern obtained according to the nickel stick element lamination process; The features of the photolithography image, the thickness features and uniformity features of the nickel layer, and the integrity features of the lamination pattern are all used to characterize the first image features.

4. The method according to claim 3, characterized in that The quality assessment steps include: For each of the graphic element process flows, a preset defect recognition algorithm is used to determine defect features in features of the photolithography image, thickness features and uniformity features of the nickel layer, and integrity features of the lamination pattern; Using a classifier, identifying the defect category corresponding to the defect feature, and using a sliding window technique to locate the defect position of the scene image, generating a mask of the defect position to mark the defect position; Determining the image blur, noise level, and exposure uniformity based on the live image, the features of the photolithography image, the thickness features and uniformity features of the nickel layer, and the integrity features of the lamination pattern; The defect category and the defect position are used to characterize the potential defect information of each of the graphic element process flows, and the image blur level, the noise level and the exposure uniformity are used to characterize the image defect information of each of the graphic element process flows.

5. The method according to claim 4, characterized in that The feature of the photolithography image includes a first image edge, the thickness feature and the uniformity feature of the nickel layer both include a second image edge, the integrity feature of the lamination pattern includes an image shape, and determining the image blur, noise level, and exposure uniformity according to the feature of the photolithography image, the thickness feature and the uniformity feature of the nickel layer, and the integrity feature of the lamination pattern includes: Determine the blur level of the image by applying Laplace transform to the first image edge, the second image edge and the image shape; Calculating the standard deviation and average pixel value of each of the scene images respectively, wherein the standard deviation is used to characterize the noise of the corresponding scene image; Calculating the noise level of the corresponding scene image according to the average pixel value and the standard deviation; The grayscale histogram of each of the scene images is calculated respectively, and the exposure uniformity of the corresponding scene image is determined through the grayscale histogram.

6. The method according to claim 4, characterized in that The adjusting the wavelength and the angle of the narrow-band LED light source based on the image defect information includes: In response to collecting live images at different wavelengths and angles, collecting statistics on image defect information of each of the live images at different wavelengths and angles of the narrow-band LED light sources; Data fitting is performed on the relationship between the image defect information and the wavelength and angle to obtain a function f(λ, θ), and an optimization algorithm is used to determine λ and θ that minimize the function f(λ, θ), wherein λ represents the wavelength of the narrow-band LED light source, and θ represents the angle of the narrow-band LED light source.

7. The method according to claim 6, characterized in that The data fitting of the relationship between the image defect information and the wavelength and angle to obtain the function f(λ, θ) comprises: Using a pre-trained data fitting model to perform data fitting on the relationship between the image defect information and the wavelength and angle, to obtain a function f(λ, θ); Wherein, the data fitting model corresponding to each image defect information is different.

8. The method according to claim 7, characterized in that The data fitting model corresponding to the image blur degree is a polynomial regression model, the data fitting model corresponding to the noise level is a ridge regression model, and the data fitting model corresponding to the exposure uniformity is a gradient boosting regression model.

9. A laser transfer printing system based on deep learning image recognition, applied to the laser transfer printing method based on deep learning image recognition according to any one of claims 1 to 8, characterized in that: include: A central control host, a camera and a narrow-band LED light source, wherein the camera and the narrow-band LED light source are both connected to the central control host; In the current graphic element process flow of laser transfer printing, the camera is used to acquire on-site images in real time, wherein each graphic element process flow is provided with at least one camera, the graphic element process flow includes a photolithography development process, an electroforming process and a nickel stick graphic element lamination process, and the narrow-band LED light source is used to improve the image recognition accuracy in the next graphic element process flow.

10. The system according to claim 9, characterized in that It also includes an adjustable optical fiber coupler, and the narrow-band LED light source can achieve angle adjustment through the adjustable optical fiber coupler.

Images