Defect image generation method and device of packaged semiconductor laser and computing equipment

Abstract

The invention discloses a defect image generation method and device for a packaged semiconductor laser, and computing equipment. The method comprises the following steps: acquiring a physical image of the packaged semiconductor laser; modeling and rendering the packaged semiconductor laser to generate a lead defect rendering image; mapping the real object image and the lead defect rendering image to a potential space by using a variational auto-encoder to obtain a corresponding real object potential variable and a corresponding lead defect rendering potential variable; respectively extracting real object style features and defect content features from the real object potential variables and the lead defect rendering potential variables; generating a real object style lead defect potential variable based on the real object style feature and the defect content feature; and reconstructing the potential variables of the physical style lead defect by using a decoder of the variational auto-encoder to obtain a physical style lead defect image. According to the invention, a high-quality defect image data set of the packaged semiconductor laser can be generated conveniently, quickly and efficiently.

Description

Technical Field

[0001] The present invention relates to the field of industrial defect data set generation, and in particular to a defect image generation method, a defect image generation device and a computing device for a packaged semiconductor laser. Background Art

[0002] In the actual production process, the quality of packaged semiconductor lasers is crucial, and defect detection of packaged semiconductor lasers is a key link in ensuring their quality. Accurately and efficiently detecting defects in lasers can improve product reliability and stability, reduce defective rates, and is of great significance to the development of the entire semiconductor laser industry. However, defect sample data for packaged semiconductor lasers is currently scarce, and the collection operation is cumbersome, resulting in a lack of sufficient and complete defect data sets to train machine learning models to achieve high-precision detection and diagnosis of packaged semiconductor laser defects.

[0003] Therefore, a defect image generation method for a packaged semiconductor laser is needed to solve the problems existing in the above technical solutions. Summary of the invention

[0004] To this end, the present invention provides a method and device for generating a defect image of a packaged semiconductor laser to solve or at least alleviate the above problems.

[0005] According to one aspect of the present invention, a defect image generation method for a packaged semiconductor laser is provided, which is executed in a computing device and includes: obtaining a physical image of a packaged semiconductor laser; modeling and rendering the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser; using a variational autoencoder, mapping the physical image and the lead defect rendering image to a latent space to obtain corresponding physical latent variables and lead defect rendering latent variables; extracting physical style features and defect content features from the physical latent variables and the lead defect rendering latent variables, respectively; generating physical style lead defect latent variables based on the physical style features and the defect content features; and reconstructing the physical style lead defect latent variables using a decoder of the variational autoencoder to obtain a physical style lead defect image.

[0006] Optionally, in the defect image generation method of the packaged semiconductor laser according to the present invention, the physical style features and the defect content features are respectively extracted from the physical latent variables and the lead defect rendering latent variables, including: using a denoising diffusion implicit model inversion method to add noise to the physical latent variables and the lead defect rendering latent variables respectively to generate corresponding physical latent Gaussian noise and lead defect rendering latent Gaussian noise, and collecting physical style features and defect content features.

[0007] Optionally, in the defect image generation method of the packaged semiconductor laser according to the present invention, the denoising diffusion implicit model inversion method is used to add noise to the physical potential variables and the lead defect rendering potential variables respectively to generate corresponding physical potential Gaussian noise and lead defect rendering potential Gaussian noise, and collect physical style features and defect content features, including: using the denoising diffusion implicit model inversion method, based on a predetermined time step, the physical potential variables and the lead defect rendering potential variables are added with noise respectively to generate corresponding physical potential Gaussian noise and lead defect rendering potential Gaussian noise; wherein, in the process of adding noise to the physical potential variables at each time step, the physical style key-value features of the corresponding time step are collected respectively to obtain the physical style features; in the process of adding noise to the lead defect rendering potential variables at each time step, the defect content query features of the corresponding time step are collected respectively to obtain the defect content features.

[0008] Optionally, in the defect image generation method of the packaged semiconductor laser according to the present invention, a physical style lead defect potential variable is generated based on the physical style features and the defect content features, including: using an adaptive instance normalization method to generate initial potential noise based on the physical style features and the defect content features; denoising the initial potential noise to inject the physical style features into the defect content features to generate a physical style lead defect potential variable.

[0009] Optionally, in the defect image generation method of a packaged semiconductor laser according to the present invention, the initial potential noise is denoised to inject physical style features into the defect content features, including: in the process of denoising the initial potential noise, injecting the physical style key-value features of each time step in the physical style features into the defect content query features of the corresponding time step.

[0010] Optionally, in the defect image generation method of the packaged semiconductor laser according to the present invention, the physical style key-value features of each time step in the physical style features are injected into the defect content query features of the corresponding time step, including: performing weighted calculation on the stylized potential query features generated at each time step in the initial potential noise denoising process and the defect content query features of the corresponding time step to obtain the weighted content query features of each time step; and performing attention calculation on the weighted content query features and the physical style key-value features of each time step based on the attention temperature scaling parameter.

[0011] Optionally, in the defect image generation method of a packaged semiconductor laser according to the present invention, denoising the initial potential noise comprises: denoising the initial potential noise by using a DDIM sampler.

[0012] Optionally, in the defect image generation method of the packaged semiconductor laser according to the present invention, a variational autoencoder is used to map the real image and the lead defect rendering image to a latent space to obtain corresponding real latent variables and lead defect rendering latent variables, including: using an encoder of the variational autoencoder to convert the real image into a corresponding first mean vector and a first variance vector, and converting the lead defect rendering image into a corresponding second mean vector and a second variance vector; reparameterizing the first mean vector and the first variance vector to obtain real latent variables, and reparameterizing the second mean vector and the second variance vector to obtain lead defect rendering latent variables.

[0013] Optionally, in the defect image generation method of a packaged semiconductor laser according to the present invention, obtaining a physical image of the packaged semiconductor laser includes: obtaining a physical image of the packaged semiconductor laser acquired based on a predetermined viewing angle; modeling and rendering the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser, including: modeling the packaged semiconductor laser to generate a three-dimensional model of the packaged semiconductor laser; rendering the three-dimensional model of the packaged semiconductor laser based on the predetermined acquisition viewing angle to generate a lead defect rendering image of the packaged semiconductor laser.

[0014] Optionally, in the defect image generation method of a packaged semiconductor laser according to the present invention, the lead defect types of the packaged semiconductor laser include lead missing, lead disconnection, lead tail wire, and lead re-soldering.

[0015] According to one aspect of the present invention, a defect image generating device is provided, which is deployed in a computing device, and the device includes: an acquisition unit, which is suitable for acquiring a physical image of a packaged semiconductor laser; a modeling and rendering unit, which is suitable for modeling and rendering the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser; a mapping unit, which is suitable for mapping the physical image and the lead defect rendering image to a latent space using a variational autoencoder to obtain corresponding physical latent variables and lead defect rendering latent variables; a feature extraction unit, which is suitable for extracting physical style features and defect content features from the physical latent variables and the lead defect rendering latent variables, respectively; a generation unit, which is suitable for generating physical style lead defect latent variables based on the physical style features and the defect content features; and a reconstruction unit, which is suitable for reconstructing the physical style lead defect latent variables using a decoder of the variational autoencoder to obtain a physical style lead defect image.

[0016] According to one aspect of the present invention, there is provided a computing device, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be suitable for being executed by the at least one processor, and the program instructions include instructions for executing the defect image generation method of the packaged semiconductor laser as described above.

[0017] According to one aspect of the present invention, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the method as described above when executed by a processor.

[0018] According to one aspect of the present invention, a readable storage medium storing program instructions is provided. When the program instructions are read and executed by a computing device, the computing device executes the defect image generation method for a packaged semiconductor laser as described above.

[0019] According to the technical solution of the present invention, a defect image generation method for a packaged semiconductor laser is provided, by collecting a physical image of a packaged semiconductor laser, and generating a lead defect rendering image of the packaged semiconductor laser through modeling and rendering, using a variational autoencoder to map the physical image and the lead defect rendering image to a latent space to obtain corresponding latent variables, and then extracting physical style features and defect content features from the latent variables, generating physical style lead defect latent variables based on the physical style features and defect content features, and finally, the decoder of the variational autoencoder can be used to reconstruct the physical style lead defect latent variables to obtain a physical style lead defect image. Based on this, the present invention only needs to adjust the defect part of the packaged semiconductor laser model to quickly generate a large number of different lead defect rendering images, and then, in combination with the physical image, a large number of different physical style lead defect images can be efficiently generated, so that the defect image data set of the packaged semiconductor laser can be conveniently, quickly and efficiently generated. In addition, the modeling and rendering method is convenient to simulate random lighting, camera angles and object positions, specify the defect style of the defect image, and improve the diversity of the defect image data set of the packaged semiconductor laser. It can be seen that the method according to the present invention can efficiently generate a high-quality defect image dataset, and then train a machine learning model to achieve high-precision detection and diagnosis of packaged semiconductor laser defects.

[0020] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to achieve the above and related purposes, some illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. By reading the following detailed description in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present invention will become more apparent. Throughout the present invention, the same reference numerals generally refer to the same parts or elements.

[0022] Figure 1 A schematic diagram of a computing device 100 provided according to an embodiment of the present invention is shown;

[0023] Figure 2 A schematic flow chart of a defect image generating method 200 for a packaged semiconductor laser according to an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram showing a physical image of a packaged semiconductor laser collected according to an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the structure distribution of a packaged semiconductor laser collected according to an embodiment of the present invention is shown;

[0026] Figure 5 A schematic diagram showing various lead defect rendering images generated according to an embodiment of the present invention;

[0027] Figure 6a A visualization schematic diagram of a process of adding noise to a physical latent variable using a denoising diffusion implicit model inversion method according to an embodiment of the present invention is shown;

[0028] Figure 6b A visualization schematic diagram of a process of adding noise to lead defect rendering potential variables using a denoising diffusion implicit model inversion method according to an embodiment of the present invention is shown;

[0029] Figure 7 A schematic diagram showing generation of a real-style lead defect image based on a real-object image of a packaged semiconductor laser and a lead defect rendering image according to an embodiment of the present invention is shown;

[0030] Figure 8 A schematic diagram showing various physical style lead defect images generated based on a physical image of a packaged semiconductor laser and various lead defect rendering images according to an embodiment of the present invention;

[0031] Fig. 9 A schematic diagram of a defect image generating device 900 provided according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0033] In view of the current problems of scarcity of defect sample data of packaged semiconductor lasers and cumbersome collection operations, the present invention proposes a defect image generation method for packaged semiconductor lasers, which can conveniently, quickly and efficiently generate a high-quality defect image dataset of packaged semiconductor lasers.

[0034] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0035] Figure 1 FIG. 1 is a schematic diagram of a computing device 100 provided according to an embodiment of the present invention. Figure 1 As shown, in a basic configuration, the computing device 100 includes at least one processing unit 102 and a system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 can be implemented as a processor. The system memory 104 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, the system memory 104 includes an operating system 105.

[0036] According to one aspect, operating system 105 is suitable for controlling the operation of computing device 100, for example. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. Figure 1 The basic configuration is shown in FIG. 1 by those components within the dashed lines. According to one aspect, computing device 100 has additional features or functionality. For example, according to one aspect, computing device 100 includes additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 1 1 is illustrated by a removable storage device 109 and a non-removable storage device 110.

[0037] As stated above, according to one aspect, a program module 103 is stored in the system memory 104. According to one aspect, the program module 103 may include one or more application programs, and the present invention does not limit the type of application program, for example, the application program may include: email and contact application programs, word processing application programs, spreadsheet application programs, database application programs, slide show application programs, drawing or computer-aided application programs, web browser application programs, etc.

[0038] According to one aspect, the program module 103 may include a plurality of program instructions suitable for executing the defect image generating method 200 of the packaged semiconductor laser of the present invention, so that the computing device 100 is configured to execute the defect image generating method 200 of the packaged semiconductor laser of the present invention.

[0039] According to one aspect, the program module 103 may include a defect image generating device 900 , which may be configured to execute the defect image generating method 200 of a packaged semiconductor laser of the present invention.

[0040] According to one aspect, the examples may be practiced on a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. Figure 1 Each or many components shown in can be integrated into a system on a chip (SOC) on a single integrated circuit to practice examples. According to one aspect, such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated via SOC, the functions described in this article can be operated via a dedicated logic integrated with other components of the computing device 100 on a single integrated circuit (chip). Embodiments of the present invention can also be practiced using other technologies capable of performing logical operations (such as AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present invention can be practiced in a general-purpose computer or in any other circuit or system.

[0041] According to one aspect, the computing device 100 may also have one or more input devices 112, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, etc. Output devices 114 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples and other devices may also be used. The computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuits; Universal Serial Bus (USB), parallel and / or serial ports.

[0042] The term computer-readable medium as used herein includes computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules 103). System memory 104, removable storage device 109, and non-removable storage device 110 are all examples of computer storage media (i.e., memory storage). Computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices, or any other products that can be used to store information and can be accessed by computing device 100. According to one aspect, any such computer storage medium can be a part of computing device 100. Computer storage media do not include carrier waves or other propagated data signals.

[0043] According to one aspect, communication media is implemented by computer readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0044] In an embodiment according to the present invention, the computing device 100 is configured to execute the defect image generation method 200 of the packaged semiconductor laser of the present invention. The computing device 100 includes one or more processors and one or more readable storage media storing program instructions. When the program instructions are configured to be executed by one or more processors, the computing device executes the defect image generation method 200 of the packaged semiconductor laser in the embodiment of the present invention.

[0045] Figure 2 A schematic flow chart of a defect image generation method 200 for a packaged semiconductor laser according to an embodiment of the present invention is shown. The defect image generation method 200 for a packaged semiconductor laser may be executed in a computing device (eg, the aforementioned computing device 100).

[0046] In an embodiment of the present invention, the computing device 100 for executing the defect image generating method 200 of the packaged semiconductor laser of the present invention may be a terminal or a server.

[0047] It should be noted that the defects of the packaged semiconductor laser in the embodiment of the present invention mainly include lead defects, which include lead missing, lead disconnection, lead tail wire, and lead re-welding.

[0048] like Figure 2 As shown, the defect image generation method 200 for packaging a semiconductor laser includes the following steps 210-260.

[0049] Step 210: The computing device 100 obtains a physical image of a packaged semiconductor laser.

[0050] In some embodiments, the computing device 100 may be communicatively connected to a collection device. Before the computing device 100 acquires a physical image of a packaged semiconductor laser, the collection device may be used to collect images of the packaged semiconductor laser to obtain a physical image of the packaged semiconductor laser. It should be noted that the collection viewing angle of the collection device may be pre-fixed to a predetermined viewing angle, and then the collection device may be used to collect a physical image of the packaged semiconductor laser based on the predetermined viewing angle. Furthermore, the computing device 100 may acquire from the collection device a physical image of the packaged semiconductor laser collected based on the predetermined viewing angle.

[0051] Figure 3 FIG. 1 is a schematic diagram showing a physical image of a packaged semiconductor laser collected according to an embodiment of the present invention, Figure 4 A schematic diagram of the structural distribution of a packaged semiconductor laser collected according to an embodiment of the present invention is shown.

[0052] like Figure 3 and Figure 4 As shown, the contents of the physical image of the packaged semiconductor laser mainly include a light emitter laser diode (LD), a heat sink, a light receiver, a photodetector (PD), four leads (respectively a first lead, a second lead, a third lead, and a fourth lead), four lead posts (respectively a first lead post, a second lead post, a third lead post, and a fourth lead post), and each lead post has a pin.

[0053] Step 220 : The computing device 100 may model and render the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser.

[0054] In some embodiments, in step 220, the packaged semiconductor laser may be first modeled to generate a three-dimensional model of the packaged semiconductor laser. Specifically, in the process of modeling the packaged semiconductor laser, the main structure of the packaged semiconductor laser may be first constructed, and then detailed structures (such as individual leads, lead posts, PD, LD, etc.) may be added to the main structure. Furthermore, the three-dimensional model of the packaged semiconductor laser may be rendered to generate a lead defect rendering image of the packaged semiconductor laser. Specifically, before rendering the three-dimensional model of the packaged semiconductor laser, a suitable texture and material may be selected, and the light source may be set to adjust the model brightness, and then the three-dimensional model of the packaged semiconductor laser may be rendered based on the set texture, material, and light source.

[0055] It should be noted that when rendering the three-dimensional model of a packaged semiconductor laser, the three-dimensional model of the packaged semiconductor laser can be rendered based on the above-mentioned predetermined viewing angle (i.e., the same viewing angle as the acquisition viewing angle of the actual image), ensuring that the rendering angle is consistent with the acquisition viewing angle of the actual image. After rendering, a rendered image of the lead defects of the packaged semiconductor laser can be obtained.

[0056] In an embodiment of the present invention, in step 220, the computing device 100 may generate various lead defect rendering images of the packaged semiconductor laser by modeling and rendering the packaged semiconductor laser.

[0057] Figure 5 FIG. 2 is a schematic diagram showing various lead defect rendering images generated according to an embodiment of the present invention. Figure 5 As shown, different types of lead defect rendering images have different lead defect types and / or defect locations. Figure 5 The corresponding lead defect type and defect location are indicated below each lead defect rendering image.

[0058] In an embodiment of the present invention, the lead defect types of the packaged semiconductor laser include lead missing, lead broken, lead tail, and lead re-soldering. Among them, lead missing may be caused by forgetting to weld during the production process, or improper operation causing the lead to fall off due to loose welding. Lead broken may be caused by vibration caused by loose welding of the end point, or the lead is broken in the middle due to poor quality. Lead tail is mainly caused by equipment or process defects causing the lead that exceeds the welding point during welding to not be cut off. Lead re-soldering is mainly caused by the same two pins being welded multiple times. It should also be pointed out that the occurrence locations of various lead defects are also different. Lead missing and lead broken can occur on all leads, while lead tail will only occur at the welding point between the first lead and the second lead column, and at the welding point between the second lead and the fourth lead column. Lead re-soldering will only occur at the first lead and the second lead. Based on this, 12 types of lead defect rendering images may be generated in step 220, wherein different types of lead defect rendering images have different lead defect types and / or defect locations. Figure 5 Shown are some typical rendering images of lead defects.

[0059] Step 230: For any lead defect rendering image, the computing device 100 can use a variational autoencoder to map the physical image Xs and the lead defect rendering image Xc to a low-dimensional latent space to obtain a physical latent variable Z corresponding to the physical image Xs. s0 , the lead defect rendering latent variable Z corresponding to the lead defect rendering image Xc c0 Here, the real object image Xs and the lead defect rendering image Xc are the input images.

[0060] It should be noted that by mapping the real image and the lead defect rendering image to the latent space, the real latent variables corresponding to the real image and the lead defect rendering latent variables corresponding to the lead defect rendering image are obtained, so that the subsequent image processing level is changed from the pixel level to the feature level, which can improve the image effectiveness and speed up the image generation speed.

[0061] In the embodiment of the present invention, in step 230, the encoder of the variational autoencoder (VAE) can be used to map (compress) the physical image Xs and the lead defect rendering image Xc to a low-dimensional latent space to obtain the corresponding physical latent variable Z s0 , Lead defect rendering potential variable Z c0. Specifically, the encoder of the variational autoencoder can convert the input image (real image, lead defect rendering image) into a probability distribution (usually a Gaussian distribution) of the latent space, thereby compressing the input image (real image, lead defect rendering image) into the latent space and converting it into the corresponding latent variables (real latent variables, lead defect rendering latent variables). For example, the input image can be represented as X(3,512,512), and the encoder of the variational autoencoder can compress the input image X(3,512,512) into the latent space and convert it into the corresponding latent variable Z(4,64,64). It should be noted that the latent variables cover multiple dimensions such as texture, color, shape, spatial relationship, semantic content, noise and irregularity, style, and image detail level.

[0062] In some embodiments, in step 230, the computing device 100 may input the real image Xs and the lead defect rendering image Xc into an encoder of the variational self-encoder during the process of mapping the real image Xs and the lead defect rendering image Xc to the latent space using the variational self-encoder. The encoder of the variational self-encoder may first convert the real image Xs into a corresponding first mean vector and a first variance vector, and convert the lead defect rendering image Xc into a corresponding second mean vector and a second variance vector. Then, the first mean vector and the first variance vector corresponding to the real image Xs may be re-parameterized to obtain a real latent variable Z corresponding to the real image Xs. s0 , and reparameterize the second mean vector and the second variance vector corresponding to the lead defect rendering image Xc to obtain the lead defect rendering latent variable Z corresponding to the lead defect rendering image Xc c0 .

[0063] Specifically, the mean vector and variance vector can be reparameterized by the following formula:

[0064] Z=μ+σε (1)

[0065] Where Z is the latent variable, μ is the mean vector, σ is the variance vector, and ε is the noise term sampled from the standard normal distribution.

[0066] Step 240: The computing device 100 can obtain the physical latent variable Z s0 and lead defect rendering latent variable Z c0 In the above example, we extract the style features and defect content features of the object respectively. That is, from the object latent variable Z s0 Extract the physical style features from the lead defect rendering latent variable Z c0 Extract defect content features.

[0067] In some embodiments, in step 240, the computing device 100 may use a denoising diffusion implicit model inversion method to perform a denoising diffusion implicit model inversion on the physical latent variable Z. s0 and lead defect rendering latent variable Z c0 Add noise respectively to generate the corresponding real potential Gaussian noise Z sT and lead defect rendering potential Gaussian noise Z cT Furthermore, in the process of adding noise to the object latent variables and the lead defect rendering latent variables respectively (inversion process), the object style features and defect content features are collected.

[0068] Figure 6a FIG. 4 is a visualization diagram showing a process of adding noise to a physical latent variable using a denoising diffusion implicit model inversion method according to an embodiment of the present invention. Figure 6b A visualization diagram of the process of adding noise to the lead defect rendering latent variables using the denoising diffusion implicit model inversion method according to an embodiment of the present invention is shown. In the process of adding noise to the physical latent variables using the denoising diffusion implicit model inversion method, the physical style features can be collected. In the process of adding noise to the lead defect rendering latent variables using the denoising diffusion implicit model inversion method, the defect content features can be collected.

[0069] It should be noted that the Denoising Diffusion Implicit Models Inversion (DDIMInversion) method is a method proposed based on the diffusion model, which aims to generate samples according to the reverse diffusion process of the diffusion model. The reverse diffusion process of the diffusion model is a process of continuously removing the prediction noise from the standard Gaussian distribution noise map and finally generating an image. The key idea of ​​the inversion process is to continuously add prediction noise to the image according to the prediction noise in the reverse diffusion process of the diffusion model, and finally generate a non-standard Gaussian distribution noise map, which will retain the distribution characteristics of the original image.

[0070] Furthermore, in step 240, the denoising diffusion implicit model inversion method (DDIM Inversion method) can be used to gradually calculate the physical latent variable Z based on a predetermined time step t={0, ..., T}. s0 and lead defect rendering latent variable Z c0 Add noise respectively to generate the corresponding real potential Gaussian noise Z sT and lead defect rendering potential Gaussian noise Z cTIn addition, in the process of adding noise to the object latent variables and the lead defect rendering latent variables based on each time step, the object style features and defect content features are collected. Specifically, in the process of adding noise to the object latent variables at each time step, the object style key-value features (K st ,V st ) to obtain the physical style features, that is, the physical style features can be obtained based on the physical style key-value features collected at each time step. In the process of adding noise to the lead defect rendering latent variables at each time step, the defect content query features (Q ct ) to obtain defect content features, that is, the defect content features can be obtained based on the defect content query features collected at each time step. That is to say, in the above inversion process, the physical style features extracted from the physical latent variables can specifically include the physical style key value features (K st ,V st ), the defect content features extracted from the lead defect rendering latent variables may specifically include defect content query features (Q ct ).

[0071] Assume that the underlying Gaussian noise Z t It is generated from the initial latent variable Z0 through a series of steps, and can be reversely deduced by the following formula:

[0072]

[0073] α t =1-β t (4)

[0074] In the formula, Z t-1 is the noisy latent variable at time step t-1, β t is the noise scheduling parameter, ε θ (Z t ,t) and ε θ (Z t-1 ,t) approximation, ε θ (Z t-1 ,t) is the Gaussian noise predicted at time step t-1.

[0075] Based on the physical style features and defect content features extracted in the above step 240 , the following step 250 may be executed to generate physical style lead defect potential variables.

[0076] Step 250: Generate a physical style lead defect potential variable based on the physical style feature and the defect content feature.

[0077] In some embodiments, in step 250, an adaptive instance normalization method (AdaIN) may be used to generate an initial potential noise Z based on the object style features and defect content features. csT . Then, for the initial potential noise Z csT Denoising is performed to inject the physical style features into the defect content features to generate the physical style lead defect latent variables. Here, after the initial latent noise Z csT During the denoising process, the physical style features can be injected into the defect content features to generate physical style lead defect latent variables.

[0078] It should be noted that the style feature injection based only on attention is insufficient in capturing color tones. In some embodiments of the present invention, in the process of generating the initial potential noise based on the physical style features and the defect content features using the adaptive instance normalization method (AdaIN), the physical style features can be color corrected (adjusting their channel statistics), so that the generated initial potential noise image can better reflect the color tones of the physical style while maintaining the defect content structure.

[0079] Specifically, the adaptive instance normalization method (AdaIN) is used to generate the initial latent noise Z csT The principle can be expressed as the following formula:

[0080]

[0081] In the formula, Z csT is the initial potential noise. cT Rendering of the potential Gaussian noise for lead defects. μ(Z sT ) and σ(Z sT ) are the mean and standard deviation of the physical style features. cT ) and σ(Z cT ) are the mean and standard deviation of the defect content characteristics, respectively.

[0082] In some embodiments, the initial potential noise Z csT During the denoising process, the physical style key-value features of each time step in the physical style features can be injected into the defect content query features of the corresponding time step, so as to inject the physical style features into the defect content features. Finally, after the denoising process is completed, the physical style lead defect latent variables can be generated.

[0083] Furthermore, in the process of injecting the physical style key-value features of each time step in the physical style features into the defect content query features of the corresponding time step, the initial potential noise Z csT The query parameter Q generated at each time step during the denoising process t(i.e., the stylized latent query feature) and the defect content query feature Q at the corresponding time step ct Perform weighted calculations to obtain weighted content query features at each time step Furthermore, the weighted content query features of each time step can be and physical style key features (K st ,V st ) to perform attention calculation. The process can be expressed as the following formula:

[0084]

[0085] In the formula, Q t is the stylized latent query feature at time step t, Q ct is the defect content query feature at time step t. λ is the mixing degree, that is, the query preservation ratio, and the value range of λ is [0,1]. λ can be used to adjust the degree of style transfer. Specifically, a higher λ can maintain more defect content, while a lower λ can enhance the effect of physical style transfer.

[0086] In addition, it is worth noting that the injection of physical style features will replace the original style of the defect content features, resulting in the style being irrelevant to the content. Therefore, the attention map calculated during the physical style feature injection process may be blurred or smoothed. In order to make the attention map clearer, an attention temperature scaling parameter can be introduced to weight the content query features at each time step based on the attention temperature scaling parameter. and physical style key features (K st ,V st ) to perform attention calculation. The attention calculation process based on the attention temperature scaling parameter can be expressed as follows:

[0087]

[0088] Here, τ = 1.5 can be used as the default setting to effectively calibrate the attention map to a standard deviation similar to its original value.

[0089] In some embodiments, a DDIM sampler may be used to sample the initial potential noise Z csT Here, the reverse process of DDIM no longer depends on the random step size of the Markov chain, and the noise removal process at each time step can be determined by the following formula:

[0090]

[0091] α t =1-β t (11)

[0092] In the formula, Z tis the noisy latent variable at time step t, β t is the noise scheduling parameter, ε θ (Z t ,t) is the Gaussian noise predicted at time step t-1.

[0093] According to the physical-style lead defect potential variables generated in step 250 , the following step 260 may be performed to generate a physical-style lead defect image.

[0094] Step 260 , the computing device 100 may utilize the decoder of the variational autoencoder (VAE) to reconstruct the real-style lead defect latent variables to obtain a real-style lead defect image (i.e., a lead defect image injected with the real-style).

[0095] Specifically, the physical style lead defect latent variable Z can be cs0 Input the decoder of the variational autoencoder (VAE), the decoder of the variational autoencoder (VAE) can be used to generate the physical style lead defect latent variable Z cs0 Reconstruct and obtain the real-style lead defect image X cs .

[0096] It should be understood that the lead defect type corresponding to the physical style lead defect image generated according to the embodiment of the present invention is the same as that of the input lead defect rendering image.

[0097] Figure 7 A schematic diagram of generating a real-style lead defect image based on a real-object image of a packaged semiconductor laser and a lead defect rendering image according to an embodiment of the present invention is shown. Figure 8 A schematic diagram of various physical style lead defect images generated based on a physical image of a packaged semiconductor laser and various lead defect rendering images according to an embodiment of the present invention is shown, wherein: Figure 8 Each physical style lead defect image in Figure 5 Corresponding to a lead defect rendering image in Figure 8 The corresponding lead defect type and defect location are indicated below each physical style lead defect image.

[0098] like Figure 7 and Figure 8As shown, according to the defect image generation method 200 of the packaged semiconductor laser provided by the embodiment of the present invention, a plurality of real-style lead defect images can be generated based on the real image of the packaged semiconductor laser and the plurality of lead defect rendering images. Specifically, according to the defect image generation method 200 of the packaged semiconductor laser provided by the embodiment of the present invention, for each lead defect rendering image, a corresponding real-style lead defect image can be generated based on the real image and the lead defect rendering image. For example, Figure 5 Rendering images of each lead defect in Figure 8 A real-life style lead defect image corresponding to .

[0099] That is to say, the present invention can generate different physical style lead defect images based on physical images and different lead defect rendering images. In this way, a large number of different lead defect rendering images are generated by adjusting the defective part of the three-dimensional model of the packaged semiconductor laser, and then, a large number of different physical style lead defect images can be efficiently generated in combination with the physical image, so that the defect image data set of the packaged semiconductor laser can be generated conveniently, quickly and efficiently.

[0100] Therefore, according to the defect image generation method 200 of the packaged semiconductor laser of the present invention, by collecting the physical image of the packaged semiconductor laser, and generating the lead defect rendering image of the packaged semiconductor laser through modeling rendering, the physical image and the lead defect rendering image are mapped to the latent space by using the variational autoencoder to obtain the corresponding latent variables, and then, the physical style features and the defect content features are extracted from the latent variables, and the physical style lead defect latent variables are generated based on the physical style features and the defect content features. Finally, the decoder of the variational autoencoder can be used to reconstruct the physical style lead defect latent variables to obtain the physical style lead defect image. Based on this, the present invention only needs to adjust the defect part of the packaged semiconductor laser model to quickly generate a large number of different lead defect rendering images, and then, in combination with the physical image, a large number of different physical style lead defect images can be efficiently generated, so that the defect image data set of the packaged semiconductor laser can be generated conveniently, quickly and efficiently. In addition, the modeling and rendering method is convenient to simulate random lighting, camera angles and object positions, specify the defect style of the defect image, and improve the diversity of the defect image data set of the packaged semiconductor laser. It can be seen that the method according to the present invention can efficiently generate a high-quality defect image dataset, and then train a machine learning model to achieve high-precision detection and diagnosis of packaged semiconductor laser defects.

[0101] Fig. 9A schematic diagram of a defect image generating device 900 provided according to an embodiment of the present invention is shown. The defect image generating device 900 may be deployed in a computing device 100, and the defect image generating device 900 is configured to execute the defect image generating method 200 of a packaged semiconductor laser of the present invention.

[0102] like Fig. 9 As shown, in an embodiment of the present invention, a defect image generating device 900 includes an acquisition unit 910, a modeling and rendering unit 920, a compression unit 930, a feature extraction unit 940, a generation unit 950, and a reconstruction unit 960 which are sequentially communicatively connected.

[0103] The acquisition unit 910 may acquire a physical image of a packaged semiconductor laser.

[0104] The modeling and rendering unit 920 may model and render the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser.

[0105] The compression unit 930 may use a variational autoencoder to map the physical image and the lead defect rendering image to a latent space to obtain corresponding physical latent variables and lead defect rendering latent variables.

[0106] The feature extraction unit 940 may extract the physical object style feature and the defect content feature from the physical object potential variable and the lead defect rendering potential variable respectively.

[0107] The generating unit 950 may generate a physical style lead defect potential variable based on the physical style feature and the defect content feature.

[0108] The reconstruction unit 960 can use the decoder of the variational autoencoder to reconstruct the real-style lead defect latent variables to obtain a real-style lead defect image.

[0109] It should be noted that the acquisition unit 910, the modeling and rendering unit 920, the compression unit 930, the feature extraction unit 940, the generation unit 950 and the reconstruction unit 960 are respectively used to execute the aforementioned steps 210 to 260. Here, the specific execution logic of each module can be found in the description of steps 210 to 260 in the method 200 above, which will not be repeated here.

[0110] In addition, the embodiment of the present invention also discloses: A9, a method as described in any one of A1-A8, wherein obtaining a physical image of a packaged semiconductor laser comprises: obtaining a physical image of a packaged semiconductor laser acquired based on a predetermined viewing angle; modeling and rendering the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser, comprising: modeling the packaged semiconductor laser to generate a three-dimensional model of the packaged semiconductor laser; rendering the three-dimensional model of the packaged semiconductor laser based on the predetermined acquisition viewing angle to generate a lead defect rendering image of the packaged semiconductor laser. A10, a method as described in any one of A1-A9, wherein the lead defect types of the packaged semiconductor laser include lead missing, lead disconnection, lead tail wire, and lead re-welding. B13, a computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the method as described in any one of A1-A10. A14, a readable storage medium storing program instructions, when the program instructions are read and processed by a computing device, the computing device processes the method as described in any one of A1-A10.

[0111] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or some aspects or parts of the method and apparatus of the present invention may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0112] In the case where the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage element), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the defect image generation method of the packaged semiconductor laser of the present invention according to the instructions in the program code stored in the memory.

[0113] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0114] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0115] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0116] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0117] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0118] Unless otherwise specified, the use of ordinal numbers "first", "second", "third", etc. to describe common objects merely indicates that different instances of similar objects are involved and is not intended to imply that the objects so described must have a given order in time, space, order or in any other manner.

Claims

1. A method for generating a defect image of a packaged semiconductor laser, executed in a computing device, comprising: Acquire physical images of packaged semiconductor lasers; Modeling and rendering the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser; Using a variational autoencoder, mapping the physical image and the lead defect rendering image to a latent space to obtain corresponding physical latent variables and lead defect rendering latent variables; Extracting a physical object style feature and a defect content feature from the physical object latent variable and the lead defect rendering latent variable respectively; Based on the physical style features and defect content features, generating physical style lead defect potential variables; The decoder of the variational autoencoder is used to reconstruct the real-style lead defect latent variables to obtain a real-style lead defect image.

2. The method of claim 1, wherein: Extracting the physical object style features and defect content features from the physical object potential variables and the lead defect rendering potential variables respectively includes: The denoising diffusion implicit model inversion method is used to add noise to the physical object potential variables and the lead defect rendering potential variables respectively to generate corresponding physical object potential Gaussian noise and lead defect rendering potential Gaussian noise, and to collect physical object style features and defect content features.

3. The method of claim 2, wherein: The denoising diffusion implicit model inversion method is used to add noise to the physical object potential variable and the lead defect rendering potential variable respectively to generate corresponding physical object potential Gaussian noise and lead defect rendering potential Gaussian noise, and collect physical object style features and defect content features, including: Using a denoising diffusion implicit model inversion method, based on a predetermined time step, the physical latent variable and the lead defect rendering latent variable are respectively denoised to generate corresponding physical latent Gaussian noise and lead defect rendering latent Gaussian noise; Among them, in the process of adding noise to the physical latent variables at each time step, the physical style key-value features of the corresponding time step are collected respectively to obtain the physical style features; in the process of adding noise to the lead defect rendering latent variables at each time step, the defect content query features of the corresponding time step are collected respectively to obtain the defect content features.

4. The method according to any one of claims 1 to 3, wherein: Based on the physical style features and defect content features, physical style lead defect potential variables are generated, including: Using an adaptive instance normalization method, an initial latent noise is generated based on the style features of the physical object and the defect content features; The initial latent noise is denoised to inject the physical style feature into the defect content feature to generate the physical style lead defect latent variable.

5. The method of claim 4, wherein: Denoising the initial latent noise to inject the physical style feature into the defect content feature, including: In the process of denoising the initial potential noise, the physical object style key-value feature of each time step in the physical object style feature is injected into the defect content query feature of the corresponding time step.

6. The method of claim 5, wherein: The physical style key-value features of each time step in the physical style features are injected into the defect content query features of the corresponding time step, including: Performing weighted calculation on the stylized potential query features generated at each time step in the initial potential noise denoising process and the defect content query features at the corresponding time step to obtain weighted content query features at each time step; Based on the attention temperature scaling parameter, attention is calculated on the weighted content query features and object style key-value features at each time step.

7. The method according to any one of claims 4 to 6, wherein: Denoising the initial potential noise includes: The initial potential noise is denoised using a DDIM sampler.

8. The method according to any one of claims 1 to 7, wherein: Using a variational autoencoder, the physical image and the lead defect rendering image are mapped to a latent space to obtain corresponding physical latent variables and lead defect rendering latent variables, including: Using an encoder of a variational autoencoder, the physical image is converted into a corresponding first mean vector and a first variance vector, and the lead defect rendering image is converted into a corresponding second mean vector and a second variance vector; The first mean vector and the first variance vector are reparameterized to obtain a physical latent variable, and the second mean vector and the second variance vector are reparameterized to obtain a lead defect rendering latent variable.

9. A defect image generating device, deployed in a computing device, the device comprising: An acquisition unit, adapted to acquire a physical image of a packaged semiconductor laser; A modeling and rendering unit, adapted to model and render the packaged semiconductor laser to generate a lead defect rendering image of the packaged semiconductor laser; A mapping unit, adapted to map the physical object image and the lead defect rendering image to a latent space using a variational autoencoder to obtain corresponding physical object latent variables and lead defect rendering latent variables; A feature extraction unit, adapted to extract a physical object style feature and a defect content feature from the physical object potential variable and the lead defect rendering potential variable, respectively; A generating unit, adapted to generate a physical style lead defect potential variable based on the physical style feature and the defect content feature; The reconstruction unit is adapted to reconstruct the real-style lead defect latent variables using the decoder of the variational autoencoder to obtain a real-style lead defect image.

10. A computing device comprising: at least one processor; and A memory storing program instructions, wherein the program instructions are configured to be processed by the at least one processor, and the program instructions include instructions for processing the method according to any one of claims 1 to 8.

Images