Ultramicro bonding copper wire abnormal state detection method, system, device, equipment, medium and product

Abstract

The invention discloses an ultramicro bonding copper wire abnormal state detection method, system, device and equipment, a medium and a product, and relates to the field of copper alloy wire quality management and control, the method comprises the following steps: carrying out preliminary enhancement treatment on a low-quality sample set, and determining a preliminarily enhanced sample set; samples in the low-quality sample set are historical images of ultramicro bonding copper wire reflection light spots; performing data set distillation on the preliminarily enhanced sample set to determine a high-quality sample, and mixing the high-quality sample and the low-quality sample set to determine a high-quality sample set; constructing an MASA detection model according to the high-quality sample set; light spot reconstruction noise reduction processing is carried out on the real-time image of the copper wire reflection light spots, and the image after noise reduction is determined; the image after noise reduction is input into the MASA detection model, whether the ultramicro bonding copper wire is in an abnormal state or not is judged, and a final detection result is determined, the quality and scale of a sample set are improved, and the accuracy of copper wire detection is further improved.

Description

Technical Field

[0001] The present application relates to the field of copper alloy wire quality control, and in particular to a method, system, device, equipment, medium and product for detecting abnormal state of ultra-fine bonding copper wire. Background Art

[0002] Ultrafine bonding copper wire is an indispensable core connection material in semiconductor production. It is made by processing copper alloy materials into high-precision connection materials with a diameter of micron level, usually less than 0.05mm. During the winding process of ultrafine bonding copper wire, the copper wire often breaks due to rapid bending and shaking of the copper wire. Workers need to visually detect the location of the broken wire and manually handle it in a high temperature environment, resulting in slow production speed, low energy efficiency and high defect rate. How to realize wire break detection through automated technical means to improve production efficiency has become one of the industry's pain points and key technical challenges.

[0003] The current cutting-edge trend is mainly to use methods based on image detection. The main idea of ​​the image detection method is to obtain pictures through a camera, analyze them with various image detection algorithms, and obtain the detection results. Image-based detection methods are usually limited by the quality and scale of the sample set. The key lies in whether enough clear and high-quality images can be obtained. However, the copper wire itself has the characteristics of ultra-fine, strong reflection, and unevenness. At the same time, in the process of copper wire winding, the copper wire distribution is a cone structure, from sparse to dense, so that they are intertwined and blocked. Therefore, it is difficult for traditional industrial imaging systems to obtain clear and high-quality images. In addition, winding machines and wire racks are important equipment for copper wire production. They are expensive, limited in quantity, large in size, and inconvenient to move. Ultrafine bonding copper wires are high in fineness, low in strength, and fragile and easy to break. Therefore, the cost of changing the copper wire distribution and collecting data is very high. Ultimately, the quality and scale of the sample set are limited, which seriously affects the accuracy of detection. Summary of the invention

[0004] The purpose of this application is to provide a method, system, device, equipment, medium and product for detecting abnormal state of ultra-fine bonding copper wire to solve the problems of limited quality and scale of sample sets and low detection accuracy.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for detecting abnormal state of ultra-fine bonding copper wires, comprising:

[0007] Performing preliminary enhancement processing on the low-quality sample set to determine a preliminary enhanced sample set; the samples in the low-quality sample set are historical images of the ultra-fine bonding copper wire reflection spots; the ultra-fine bonding copper wire reflection spots are line spots;

[0008] Performing data set distillation on the initially enhanced sample set to determine high-quality samples, and mixing the high-quality samples with the low-quality sample set to determine a high-quality sample set;

[0009] Constructing a MASA detection model based on the high-quality sample set;

[0010] Performing spot reconstruction and noise reduction processing on the real-time image of the copper wire reflected light spot to determine the image after noise reduction;

[0011] The denoised image is input into the MASA detection model to determine whether the ultra-fine bonding copper wire is in an abnormal state and determine the final detection result.

[0012] In a second aspect, the present application provides an ultra-fine bonding copper wire abnormal state detection system, comprising:

[0013] A preliminary enhancement processing module is used to perform preliminary enhancement processing on a low-quality sample set to determine a preliminary enhanced sample set; the samples in the low-quality sample set are historical images of ultra-fine bonding copper wire reflection spots; the ultra-fine bonding copper wire reflection spots are line spots;

[0014] A high-quality sample set determination module, used to perform data set distillation on the initially enhanced sample set to determine high-quality samples, and mix the high-quality samples with the low-quality sample set to determine a high-quality sample set;

[0015] A MASA detection model building module, used to build a MASA detection model according to the high-quality sample set;

[0016] A noise reduction module is used to perform light spot reconstruction and noise reduction processing on the real-time image of the light spot reflected by the copper wire to determine the image after noise reduction;

[0017] The final detection result determination module is used to input the denoised image into the MASA detection model, determine whether the ultra-fine bonding copper wire is in an abnormal state, and determine the final detection result.

[0018] In a third aspect, the present application provides a device for detecting abnormal state of ultra-fine bonding copper wires, comprising:

[0019] Robotic arm, camera, array light source, fixed bracket, connecting network cable and industrial computer;

[0020] The array light source is fixed on the fixed bracket to irradiate the ultra-fine bonding copper wire with a conical spatial structure;

[0021] The camera is fixed on the mechanical arm and continuously photographs the ultra-fine bonding copper wire to obtain a real-time image of the reflected light spot of the ultra-fine bonding copper wire;

[0022] The camera and the robotic arm are connected and communicated with the industrial computer via a network cable; the industrial computer controls the robotic arm by calling the SDK interface of the robotic arm to change the joint angle and position to adjust the shooting angle and position of the camera;

[0023] The camera transmits the captured real-time image to the industrial computer through a network cable, and the industrial computer uses the ultra-fine bonding copper wire abnormal state detection method described in any one of claims 1 to 5 to perform abnormal state detection processing.

[0024] In a fourth aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described methods for detecting abnormal states of ultra-fine bonding copper wires.

[0025] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for detecting abnormal states of ultra-fine bonding copper wires.

[0026] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for detecting abnormal states of ultra-fine bonding copper wires.

[0027] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0028] The present application provides a method, system, device, equipment, medium and product for detecting abnormal state of ultra-fine bonded copper wire, collects a small amount of low-quality sample sets; rotates, crops and scales the low-quality sample sets, performs preliminary sample enhancement, and obtains a preliminary enhanced sample set. In practical applications, the synthetic minority over-sampling technique (Smote) can be used for preliminary sample enhancement; the above-mentioned preliminary enhanced sample set is subjected to data set distillation using the Minimax Diffusion method, further enhanced to obtain a small amount of high-quality samples, and then mixed with the above-mentioned preliminary enhanced sample set to obtain a small amount of higher-quality sample sets; uses the Matching Anything By Segmenting Anything (MASA) method to train based on the above-mentioned small amount of higher-quality sample sets to obtain a corresponding MASA detection model; obtains a real-time image of the copper wire reflection spot through a collection device, such as a high-definition camera, performs image preprocessing, and performs spot reconstruction and noise reduction. The denoised image is detected based on the above-mentioned MASA detection model to obtain the final detection result. This application improves the quality and size of the sample set by sample enhancement, reduces interference by using an image preprocessing method of spot reconstruction and noise reduction, thereby improving the accuracy of copper wire detection, effectively detecting whether ultra-fine bonding copper wires are abnormal, and improving the automation level of quality control in the production process of ultra-fine bonding copper wires. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flow chart of a method for detecting abnormal state of ultra-fine bonding copper wires provided in one embodiment of the present application;

[0031] Figure 2 The S1 flow chart provided in one embodiment of the present application;

[0032] Figure 3 The S2 flow chart provided in one embodiment of the present application;

[0033] Figure 4 The S3 flow chart provided in one embodiment of the present application;

[0034] Figure 5 S3 flow chart provided for another embodiment of the present application;

[0035] Figure 6 The S5 flow chart provided in one embodiment of the present application;

[0036] Figure 7 S1 flow chart provided for another embodiment of the present application;

[0037] Figure 8 A schematic diagram of an abnormal state detection device for ultra-fine bonding copper wires provided in one embodiment of the present application;

[0038] Fig. 9 This is a working principle diagram of an abnormal state detection device for ultra-fine bonding copper wire winding provided in an embodiment of the present application; wherein, Fig. 9 (a) is a schematic diagram of a real-time image of a light spot; Fig. 9 (b) is a schematic diagram of the detection effect of the YOLOX model trained based on a small number of low-quality sample sets; Fig. 9 (c) is a schematic diagram of the detection effect of the MASA model trained on a higher quality sample set after sample enhancement; Fig. 9 (d) is a schematic diagram of the detection effect after the MASA model is trained based on a higher quality sample set after sample enhancement and denoised through spot reconstruction. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0041] During the winding process of ultra-fine bonding copper wire, the copper wire is ultra-fine, highly reflective, uneven, and distributed in a conical structure, from sparse to dense, so that it is intertwined and blocked. Therefore, it is difficult for traditional industrial imaging systems to obtain clear and high-quality images. At the same time, winding machines and wire racks are expensive, limited in number, large in size, and inconvenient to move, while copper wires are highly fine, low in strength, and fragile and easy to break. Therefore, it is very costly to change the distribution of copper wires and collect data. Ultimately, the quality and scale of the sample set are limited, which seriously affects the accuracy of detection.

[0042] In response to the above problems, the present application proposes a method, system, device, equipment, medium and product for detecting abnormal status of ultra-fine bonding copper wires, which can effectively detect whether the ultra-fine bonding copper wires are abnormal in detection scenarios where the quality and scale of the sample set are limited, while improving the level of automation of quality control in the production process of ultra-fine bonding copper wires.

[0043] Example 1

[0044] The embodiment of the present application provides a method for detecting abnormal state of ultra-fine bonding copper wire, which is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, Figure 1 As shown, the method includes the following steps.

[0045] S1: Performing preliminary enhancement processing on a low-quality sample set to determine a preliminary enhanced sample set; the samples in the low-quality sample set are historical images of reflection spots of ultra-fine bonding copper wires; the reflection spots of ultra-fine bonding copper wires are line spots.

[0046] S2: performing data set distillation on the initially enhanced sample set to determine high-quality samples, and mixing the high-quality samples with the low-quality sample set to determine a high-quality sample set.

[0047] S3: Construct a MASA detection model based on the high-quality sample set.

[0048] S4: performing spot reconstruction and noise reduction processing on the real-time image of the light spot reflected by the copper wire to determine a noise-reduced image.

[0049] S5: Input the denoised image into the MASA detection model to determine whether the ultra-fine bonding copper wire is in an abnormal state, and determine the final detection result.

[0050] In an exemplary embodiment, before S1, it also includes: collecting a small number of low-quality sample sets, with a size of about 300. The sample sets are obtained by adjusting the distribution of copper wires and the shooting angle of the camera on different machine tools, and using a light source to illuminate the copper wires and shoot the light spots formed by the reflection of the copper wires.

[0051] In an exemplary embodiment, Figure 2 As shown, S1 can be replaced by the following steps.

[0052] S11: Using candy edge detection to process the low-quality sample set to determine a sample contour set.

[0053] S12: Based on the sample contour set, the number of sample contours and the brightness variance are counted, and the number of sample contours and the brightness variance are used as a measure of the number of light spots and the contrast, to screen out the minimum two categories of samples based on the number and the contrast. In practical applications, the minimum two categories of samples based on the number and the contrast are screened out by optics clustering.

[0054] S13: Based on the sample profile set, Hough transform is used to perform linear fitting clustering on the line spot, and the two smallest categories of samples based on direction and length are screened out. In practical applications, the average angle between the straight lines and the average length of the straight lines are calculated as a measure of the direction and length of the spot, and optics clustering is performed to select the two smallest categories based on direction and length according to the quantity.

[0055] S14: Based on the sample contour set, the ellipse fitting function of cv2 is used to perform ellipse fitting processing on the line spot, the width of the maximum ellipse is determined, and the width of the maximum ellipse is used as a measure of the cross-overlap degree of the spot, and the minimum two types of samples based on the cross-overlap degree of the spot are screened out. Among them, cv2 is the cv2 library, that is, the interface of the OpenCV library in Python; in actual applications, the minimum two types of samples based on the cross-overlap degree of the spot are screened out by optics clustering.

[0056] S15: Preprocessing is performed based on the minimum two types of samples based on quantity and contrast, the minimum two types of samples based on direction and length, and the minimum two types of samples based on the cross-overlap degree of light spots to determine a preliminary enhanced sample set; the preprocessing includes rotation from 0 to 360 degrees, random cropping and splicing, secondary interpolation scaling, and expansion of scale, with a total scale of approximately 800 images.

[0057] In an exemplary embodiment, Figure 3 As shown, S2 can be replaced by the following steps.

[0058] S21: dividing the initially enhanced sample set according to the blurriness and dimness of the light spot to determine a positive sample set with light spots and a negative sample set without light spots.

[0059] S22: Performing data set distillation on the positive sample set and the negative sample set using the Minimax Diffusion method to determine an enhanced positive sample set and an enhanced negative sample set, with a total size of about 100 images.

[0060] In practical applications, the Minimax Diffusion method is based on the formula L = L simple +λ r L r +λ d L d Calculate. Among them, λr and λ d is a hyperparameter of the weight, adjusted empirically,

[0061] For the formula Among them, ε is the real noise with ε~N(0,1), ε θ is the noise prediction function, i.e. the desired variable; z t =E(x), E is the encoding function of the variational auto-encoder (vae), x is the sample set; c = E c (y), E c is the embedding mapping function of the VAE encoder to encode the category information into the latent space, and y is the category label of the sample set, i.e., with light spot and without light spot.

[0062] For the formula σ is the cosine similarity function; is the original embedding predicted by subtracting the noise from the noisy embedding, have M is a storage helper that stores the true samples used in adjacent iterations, z m represents the real sample, N M Represents the batch size of real samples, a hyperparameter.

[0063] For the formula have D is another storage aid. The meanings of other parameters are the same as those for L. r The formula parameters have the same meaning.

[0064] S23: Mix the enhanced positive sample set and the enhanced negative sample set, and mark the enhanced positive sample set in the mixed sample set as the positive sample set, and mark the enhanced negative sample set as empty, to determine a high-quality sample set with a total size of about 900 samples.

[0065] In an exemplary embodiment, Figure 4 As shown, S3 can be replaced by the following steps.

[0066] S31: performing image preprocessing on the historical images in the high-quality samples to determine the preprocessed spot image; the image preprocessing includes mean filtering, Gaussian filtering, binarization processing and noise filtering.

[0067] S32: using a contour extraction function to perform contour extraction on the preprocessed light spot image to determine a contour set of the light spot.

[0068] S33: Using the ellipse fitting function of cv2 to perform ellipse fitting on the contour set of the light spot to determine the ellipse set.

[0069] S34: Perform kmeans clustering on the ellipse set to determine a minimum ellipse class, and determine the average minor axis radius length of the minimum ellipse class.

[0070] S35: Taking the ellipse region where the minimum ellipse class is located as the region where the target light spot is located, and determining the center point of the light spot.

[0071] S36: With the center point of the light spot as the center of the circle and the average minor axis radius length as the radius of the circle, a circular area is generated, and the circular area covers the elliptical area, the line light spot is reconstructed into a point light spot, and a MASA detection model is constructed; the point light spot is the image after noise reduction.

[0072] In another exemplary embodiment, Figure 5 As shown, S3 can be replaced by the following steps.

[0073] S41: Input the above-mentioned small amount of high-quality sample sets including positive sample sets and negative sample sets into yolox as custom data sets, create a corresponding configuration file, specify the data set path and set hyperparameters such as resolution, step size, and number of iterations, adjust the hyperparameters according to experience, and load yolox's own pre-trained model based on the coco data set, and train them together to obtain a training model of copper wire spot based on yolox.

[0074] S42: Use the above-mentioned yolox-based training model as a pre-trained model of the MASA (Matching Anything By Segmenting Anything) method, create a corresponding configuration file, specify the dataset path and set hyperparameters such as resolution, step size, and number of iterations. The hyperparameters are adjusted according to experience. At the same time, load the MASA pre-trained model based on the coco dataset and train them together to obtain the corresponding MASA detection model.

[0075] In an exemplary embodiment, Figure 6 As shown, S5 can be replaced by the following steps.

[0076] S51: calling the detection function of the model open API in the MASA detection model to detect the denoised image, and outputting the coordinates of the upper left corner point and the lower right corner point of the minimum circumscribed matrix of the target light spot.

[0077] S52: Mark the coordinates of the upper left corner point and the coordinates of the lower right corner point with a frame, and judge whether the ultra-fine bonding copper wire is in an abnormal state according to the number of light spots in the marked frame to determine the final detection result.

[0078] In an exemplary embodiment, Figure 7 As shown, S5 can be replaced by the following steps.

[0079] S61: Preprocessing the real-time image of the light spot by using mean filtering and Gaussian filtering, and performing binarization processing to filter out small and dim noise points, thereby obtaining a preprocessed light spot image.

[0080] S62: Use the contour extraction function to extract the contour of the pre-processed spot image to obtain the contour set of the spot. The copper wire reflection spot is actually a line spot, which is elliptical, so the ellipse fitting function of cv2 is used to perform ellipse fitting based on the contour set of the spot to obtain an ellipse set.

[0081] S63: Perform kmeans clustering on the above ellipse set, find the smallest ellipse class, that is, the non-overlapping line spot, and calculate its average minor axis radius length b. Other overlapping and intersecting ellipses remain unchanged without processing. The ellipse area is the area where the target spot is located. Use the grayscale centroid method according to the formula Calculate. Where, the image is I, the image resolution is MxN, I(x,y) represents the gray value of the image at position (x,y), C x is the x-coordinate of the image's centroid, C y is the y coordinate of the image center. x ,C y ).

[0082] S64: Taking the center point of the light spot (C x ,C y ) as the center of the circle, and the average minor axis radius length b as the radius of the circle, draw a circle, and use the regenerated circular area to cover the above elliptical area. The grayscale value of the center of the circle remains unchanged, and the grayscale value from the center of the circle to the circle is regenerated according to the Gaussian distribution. Finally, Gaussian blur is used for smoothing to make the edge smoother, so that the original line spot is reconstructed into a point spot, making it more focused, thereby achieving noise reduction. Thus, the image after noise reduction is obtained.

[0083] S65: Based on the above MASA detection model, specify the path to load, call the detection function of the model open API to detect the above denoised image, and the output of the detection model is the x, y coordinates of the upper left corner and lower right corner of the minimum external matrix of the identified target light spot. Mark the frame according to the x, y coordinates of these two points. Under normal circumstances, the light spots that can be detected in each detection point are fixed. If a break occurs, the number of light spots will decrease. Therefore, according to the statistical number of light spots at the detection point, it is judged whether the light spot is in an abnormal state to obtain the final detection result.

[0084] Example 2

[0085] Based on the same inventive concept, the embodiment of the present application also provides an ultra-fine bonded copper wire abnormal state detection system for implementing the ultra-fine bonded copper wire abnormal state detection method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more ultra-fine bonded copper wire abnormal state detection system embodiments provided below can refer to the limitations of the ultra-fine bonded copper wire abnormal state detection method above, and will not be repeated here.

[0086] In an exemplary embodiment, a system for detecting abnormal state of ultra-fine bonding copper wire is provided, comprising:

[0087] The preliminary enhancement processing module is used to perform preliminary enhancement processing on the low-quality sample set to determine the preliminary enhanced sample set; the samples in the low-quality sample set are historical images of the reflection spots of the ultra-fine bonding copper wire; the reflection spots of the ultra-fine bonding copper wire are line spots.

[0088] The high-quality sample set determination module is used to perform data set distillation on the initially enhanced sample set to determine high-quality samples, and mix the high-quality samples with the low-quality sample set to determine a high-quality sample set.

[0089] The MASA detection model building module is used to build a MASA detection model according to the high-quality sample set.

[0090] The noise reduction module is used to perform light spot reconstruction and noise reduction processing on the real-time image of the light spot reflected by the copper wire to determine the image after noise reduction.

[0091] The final detection result determination module is used to input the denoised image into the MASA detection model, determine whether the ultra-fine bonding copper wire is in an abnormal state, and determine the final detection result.

[0092] Example 3

[0093] The present application provides a device for detecting abnormal state of ultra-fine bonding copper wires, such as Figure 8 As shown, the ultra-fine bonding copper wire abnormal state detection device includes: a mechanical arm, a camera, an array light source, a fixed bracket, a connecting network cable and an industrial computer.

[0094] The array light source is fixed on the fixing bracket to illuminate the ultra-fine bonding copper wire with a conical space structure from the upper side. Further, the array light source is a high-brightness array light source, the position of the array light source is adjusted and fixed by the fixing bracket, and it starts to emit light after being plugged in.

[0095] The camera is fixed on the mechanical arm and continuously photographs the ultra-fine bonding copper wire from the upper side to obtain a real-time image of the reflected light spot of the ultra-fine bonding copper wire. Further, the camera is a high-definition camera.

[0096] The camera and the robotic arm are connected and communicated with an industrial computer via a network cable; the industrial computer controls the robotic arm by calling the SDK interface of the robotic arm, changes the joint angle and position, and adjusts the shooting angle and position of the camera, so that the real-time image of the reflected light spot is clearly visible.

[0097] The camera transmits the captured real-time image to the industrial computer through the network cable, and the industrial computer uses the ultra-fine bonding copper wire abnormal state detection method shown in Example 1 to perform abnormal state detection processing. Fig. 9 As shown in (a), it is a real-time image of a light spot obtained, such as Fig. 9 (b) in the figure shows the detection effect of the YOLOX model trained based on a small number of low-quality sample sets. Fig. 9 (c) in the figure shows the detection effect of the MASA model trained on a higher quality sample set after sample enhancement. Fig. 9 (d) shows the detection effect after the MASA model is trained based on a higher quality sample set after sample enhancement and denoised through spot reconstruction.

[0098] 1) During the winding process of ultra-fine bonding copper wire, the copper wire is ultra-fine, highly reflective, and uneven, and the distribution is in a conical structure, from sparse to dense, so that it is intertwined and blocked. Therefore, it is difficult for traditional industrial imaging systems to obtain clear and high-quality images. At the same time, the winding machine and the wire rack are expensive, limited in quantity, large in size, and inconvenient to move, while the copper wire has high fineness, low strength, and is fragile and easy to break. Therefore, the cost of changing the distribution of copper wires and collecting data is very high. Ultimately, the quality and scale of the sample set are limited, which seriously affects the accuracy of the detection. In response to the above problems, the present application can improve the balance by enhancing the samples of the corner cases (cornercase) that account for a small number of cases but have a large impact in the detection scenario where the quality and scale of the sample set are limited, and by using the minimax diffusion method, the existing small sample set is subjected to data set distillation, and by generating samples close to the farthest real sample, it is ensured that the newly generated data set after distillation is both representative and diverse for the original data set. This improves the size and quality of the sample set, and thus the accuracy of the detection model, effectively detecting whether the ultra-fine bonding copper wire is abnormal, while also improving the level of automation in the quality control of the ultra-fine bonding copper wire production process.

[0099] 2) The accuracy of image detection is directly related to the level of image noise reduction. This application converts the line light spot that was originally dim and blurred on both wings and only bright in the center into a smaller and more focused point light spot, reducing noise interference and the difficulty of recognition, thereby improving the accuracy of light spot detection.

[0100] Example 4

[0101] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store abnormal state detection data of ultra-fine bonding copper wires. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting abnormal state of ultra-fine bonding copper wires is implemented.

[0102] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0103] Example 5

[0104] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0105] Example 6

[0106] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0108] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0109] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting abnormal state of ultra-fine bonding copper wire, characterized in that: The method for detecting abnormal state of ultra-fine bonding copper wire comprises: Performing preliminary enhancement processing on the low-quality sample set to determine a preliminary enhanced sample set; the samples in the low-quality sample set are historical images of the ultra-fine bonding copper wire reflection spots; the ultra-fine bonding copper wire reflection spots are line spots; Performing data set distillation on the initially enhanced sample set to determine high-quality samples, and mixing the high-quality samples with the low-quality sample set to determine a high-quality sample set; Constructing a MASA detection model based on the high-quality sample set; Performing spot reconstruction and noise reduction processing on the real-time image of the copper wire reflected light spot to determine the image after noise reduction; The denoised image is input into the MASA detection model to determine whether the ultra-fine bonding copper wire is in an abnormal state and determine the final detection result.

2. The method for detecting abnormal state of ultra-fine bonding copper wire according to claim 1, characterized in that: Perform preliminary enhancement processing on the low-quality sample set and determine the preliminary enhanced sample set, which specifically includes: Using candy edge detection to process the low-quality sample set to determine a sample contour set; Based on the sample contour set, the number of sample contours and the brightness variance are counted, and the number of sample contours and the brightness variance are used as a measure of the number and contrast of light spots to screen out the minimum two types of samples based on the number and contrast; Based on the sample profile set, Hough transform is used to perform linear fitting clustering on the line spot, and the two smallest types of samples based on direction and length are screened out; Based on the sample contour set, the ellipse fitting function of cv2 is used to perform ellipse fitting processing on the line spot, the width of the maximum ellipse is determined, and the width of the maximum ellipse is used as a measure of the cross-overlap degree of the spot, and the minimum two types of samples based on the cross-overlap degree of the spot are screened out; Preprocessing is performed based on the minimum two types of samples based on quantity and contrast, the minimum two types of samples based on direction and length, and the minimum two types of samples based on the cross-overlap degree of light spots to determine a preliminary enhanced sample set; the preprocessing includes rotation, random cropping and splicing, quadratic interpolation scaling, and scale expansion.

3. The method for detecting abnormal state of ultra-fine bonding copper wire according to claim 1, characterized in that: Performing data set distillation on the initially enhanced sample set to determine high-quality samples, and mixing the high-quality samples with the low-quality sample set to determine a high-quality sample set, specifically includes: Dividing the initially enhanced sample set according to the blurriness and dimness of the light spot to determine a positive sample set with light spots and a negative sample set without light spots; Performing data set distillation on the positive sample set and the negative sample set using a Minimax Diffusion method to determine an enhanced positive sample set and an enhanced negative sample set; The enhanced positive sample set and the enhanced negative sample set are mixed, and the enhanced positive sample set in the mixed sample set is marked as a positive sample set, and the enhanced negative sample set is marked as empty, so as to determine a high-quality sample set.

4. The method for detecting abnormal state of ultra-fine bonding copper wire according to claim 1, characterized in that: The MASA detection model is constructed according to the high-quality sample set, specifically including: Performing image preprocessing on the historical images in the high-quality samples to determine the preprocessed spot image; the image preprocessing includes mean filtering, Gaussian filtering, binarization processing and noise filtering; Using a contour extraction function to extract the contour of the preprocessed light spot image to determine a contour set of the light spot; Using the ellipse fitting function of cv2 to perform ellipse fitting on the contour set of the light spot to determine the ellipse set; Performing kmeans clustering on the ellipse set to determine a minimum ellipse class, and determining an average minor axis radius length of the minimum ellipse class; Taking the elliptical area where the minimum ellipse class is located as the area where the target light spot is located, and determining the center point of the light spot; A circular area is generated with the center point of the light spot as the center of the circle and the average minor axis radius length as the radius of the circle. The circular area covers the elliptical area, the line light spot is reconstructed into a point light spot, and a MASA detection model is constructed; the point light spot is the denoised image.

5. The method for detecting abnormal state of ultra-fine bonding copper wire according to claim 4, characterized in that: Inputting the de-noised image into the MASA detection model to determine whether the ultra-fine bonding copper wire is in an abnormal state and determining the final detection result specifically includes: Calling the detection function of the model open API in the MASA detection model to detect the denoised image, and outputting the coordinates of the upper left corner point and the lower right corner point of the minimum circumscribed matrix of the target light spot; The coordinates of the upper left corner point and the coordinates of the lower right corner point are marked with a frame, and whether the ultra-fine bonding copper wire is in an abnormal state is judged according to the number of light spots in the marked frame to determine the final detection result.

6. A system for detecting abnormal state of ultra-fine bonding copper wire, characterized in that: include: A preliminary enhancement processing module is used to perform preliminary enhancement processing on the low-quality sample set and determine a preliminary enhanced sample set; The samples in the low-quality sample set are historical images of ultra-fine bonding copper wire reflection spots; the ultra-fine bonding copper wire reflection spots are line spots; A high-quality sample set determination module, used to perform data set distillation on the initially enhanced sample set to determine high-quality samples, and mix the high-quality samples with the low-quality sample set to determine a high-quality sample set; A MASA detection model building module, used to build a MASA detection model according to the high-quality sample set; A noise reduction module is used to perform light spot reconstruction and noise reduction processing on the real-time image of the light spot reflected by the copper wire to determine the image after noise reduction; The final detection result determination module is used to input the denoised image into the MASA detection model, determine whether the ultra-fine bonding copper wire is in an abnormal state, and determine the final detection result.

7. An ultra-fine bonding copper wire abnormal state detection device, characterized in that: The ultra-fine bonding copper wire abnormal state detection device comprises: a mechanical arm, a camera, an array light source, a fixing bracket, a connecting network cable and an industrial computer; The array light source is fixed on the fixed bracket to irradiate the ultra-fine bonding copper wire with a conical spatial structure; The camera is fixed on the mechanical arm and continuously photographs the ultra-fine bonding copper wire to obtain a real-time image of the reflected light spot of the ultra-fine bonding copper wire; The camera and the robotic arm are connected and communicated with the industrial computer via a network cable; the industrial computer controls the robotic arm by calling the SDK interface of the robotic arm to change the joint angle and position to adjust the shooting angle and position of the camera; The camera transmits the captured real-time image to the industrial computer through a network cable, and the industrial computer uses the ultra-fine bonding copper wire abnormal state detection method described in any one of claims 1 to 5 to perform abnormal state detection processing.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for detecting abnormal state of ultra-fine bonding copper wires according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting abnormal state of ultra-fine bonding copper wire according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting abnormal state of ultra-fine bonding copper wire according to any one of claims 1 to 6 is implemented.

Images