Flying wafer posture detection method, device and medium based on real-time vision

Abstract

The present invention discloses a flying wafer posture detection method, device and medium based on real-time vision. The method allocates and processes the time-consuming operations required for image processing in advance before the material moves to the field of view, triggers the photo taking after the timing, so as to ensure high repeatability of the photo taking timing. The time-consuming operations include memory copying. The high repeatability of the photo taking timing means that the posture of the wafer is within the set range in the field of view each time the photo taking is triggered in each solid crystal taking path. The present invention can ensure that the visual system should have real-time performance and compress the image transmission time as much as possible, and perform "advanced processing" on the visual related operations to avoid the mechanical arm movement pausing due to the long visual data processing time.

Description

Technical Field

[0001] The present invention relates to the field of real-time vision technology, and more specifically, to a flying wafer posture detection method, device and medium based on real-time vision. Background Art

[0002] In recent years, the market demand for assembly and processing has continued to expand and improve in the balance of "high speed and high precision" and "quality and quantity". However, in the actual processing and assembly process, in order to grasp quality control and improve quality, it is often necessary to add processes to form a fully closed-loop process flow, which often reduces the speed of equipment operation and leads to reduced efficiency. How to optimize product processes, achieve cost reduction and efficiency improvement while ensuring product quality improvement has become a pain point that companies have to solve.

[0003] In order to ensure the accuracy of the solid crystal, it is necessary to use a camera to capture the position of the wafer during the wafer transportation process, which often causes unnecessary pauses in the equipment. The so-called flying photography is to take pictures while the object is in motion, and obtain a clear and correct position of the object at the moment of taking the picture. Taking pictures on the fly can effectively reduce equipment pauses and improve production efficiency. The biggest advantage of applying flying photography technology is that it uses machine vision to achieve full closed-loop detection, and automatically triggers the actual position of the object at the end of the object during the transportation process. This can avoid pauses in taking pictures during the movement, effectively improve efficiency, and send the actual position to the controller to determine whether secondary correction or ejection of the position is required, effectively improving accuracy.

[0004] Regarding real-time processing, Xian Hao's "Design of a Single Cutting System for Rounded Corner Bags in Bag Making Machines Based on Machine Vision" uses a flying vision system to obtain the position correction value of the end material, effectively solving the problems of burrs in rounded corner bags, material loss in double cutting processes, mechanical wear, and environmental pollution. Tian Zhibin's "Research on Robot Flying Method Based on KALMAN Filtering and Deep Learning" can ensure that the calculation of the correction value is completed before the next beat of the robot, and ultimately achieve accurate robot flying positioning.

[0005] Xian Hao's "Design of a Single-cutting System for Rounded Corner Bags in Bag Making Machines Based on Machine Vision" uses a flying mode vision system to obtain the position correction value of the end material, effectively solving the problems of burrs in rounded corner bags, material loss in double-cutting processes, mechanical wear, and environmental pollution. However, its PLC uses a high-speed counter to hard-trigger the camera through the I / O port. The sending of trigger signals, hardware response, and information feedback all require milliseconds of time. For high-speed and high-precision application scenarios, too many time-consuming factors will increase the inaccuracy of real-time. Tian Zhibin's "Research on Robot Flying Methods Based on KALMAN Filtering and Deep Learning" can ensure that the calculation of the correction value is completed before the next beat of the robot, and ultimately achieve accurate robot flying positioning. However, it only guarantees the real-time nature of the operation and control through the RTE of codesys. For the real-time nature of the visual system, it cannot guarantee the real-time calculation of the correction value, which can easily cause pauses and discontinuities in the movement. Summary of the invention

[0006] The present invention is provided to solve the above problems existing in the prior art. Therefore, a flying wafer posture detection method, device and medium based on real-time vision are needed to ensure that the visual system should have real-time performance and compress the image transmission time as much as possible and perform "advanced processing" on visual related operations to avoid the visual data processing time being too long and causing the robot arm movement to stop.

[0007] According to a first aspect of the present invention, a method for detecting wafer posture by flying shot based on real-time vision is provided, the method comprising:

[0008] Before the material moves into the field of view, the time-consuming operations required for image processing are allocated in advance, and the photo is triggered after the timing to ensure high repeatability of the photo timing. The time-consuming operations include memory copying. The high repeatability of the photo timing means that the position of the chip is within the set range in the field of view each time the photo is triggered in each solid crystal path.

[0009] Furthermore, a set posture is set, and the deviation between the posture of the chip when the camera is triggered and the set posture is calculated. If the deviation is within a first threshold, it is determined that the posture of the chip when the camera is triggered is within a set range in the field of view. If the deviation is within a second threshold, correction is performed. If the deviation is within a third threshold, material is discarded.

[0010] Furthermore, the time-consuming operations required for image processing are allocated in advance before the material moves into the field of view, and the photo is triggered after the timing to ensure high repeatability of the photo-taking timing, which specifically includes:

[0011] Obtaining the standard posture during the flying shot, the standard posture during the flying shot means that the verticality of the center line drawn from the crystal midpoint from the crystal taking position to the flying shot position and then to the crystal fixing position and the center line of the crystal at the crystal taking position is within the error tolerance ε, the error tolerance ε means that even if there is an angle deviation when the crystal is placed during the curing process of the solder paste, it will be corrected to the center line position through the curing characteristics of the solder paste;

[0012] Determine the timing time according to the time when the chip moves to the flying position, and when the chip moves to the set advance point, send the timing time to the camera, and the camera takes a picture of the flying position after reaching the timing time;

[0013] According to the standard posture during the flying shot, determine the lateral coordinate x of the center of the wafer relative to the field of view, the longitudinal coordinate y of the center of the wafer relative to the field of view, and the deflection angle θ of the center line of the wafer relative to the field of view in the standard posture;

[0014] Obtain the camera cache image, and obtain the lateral coordinate x of the center of the chip in the cache image relative to the field of view according to the camera cache image. r , the longitudinal coordinate y of the chip center relative to the field of view r and the deflection angle θ of the wafer centerline relative to the field of view r ;

[0015] Construct the following judgment formula:

[0016] |θ r -θ|<=ε

[0017] If θ r If the judgment formula is met, then x r ,y r Take the difference: Δx = x r -x、Δy=y r -y, and send it to the operation control system;

[0018] If θ r If it does not meet the judgment formula, the material will be thrown away.

[0019] Furthermore, the fault tolerance ε is calculated by the following formula:

[0020]

[0021] Among them, ε i represents the error tolerance in the i-th experiment, and n represents the total number of experiments.

[0022] Furthermore, the time it takes for the wafer to move to the flying position is calculated by the following formula:

[0023]

[0024] Where l is the rod length, α is the arc, s is the total arc length of the wafer from the crystal taking position to the flying position, s a is the arc length of the acceleration phase, s c is the arc length of the constant speed stage, s, v c , a are known items in process design, t1 is the acceleration time, t2 is the constant speed time of the wafer from the flying position, and t is the total time taken for the wafer to move to the flying position.

[0025] Furthermore, the lateral coordinate x of the center of the chip relative to the field of view, the longitudinal coordinate y of the center of the chip relative to the field of view, and the deflection angle θ of the center line of the chip relative to the field of view in the standard posture are calculated by the following formula:

[0026]

[0027] Among them, x i represents the lateral coordinate of the center of the wafer relative to the field of view in the i-th experiment; y i represents the longitudinal coordinate of the center of the wafer relative to the field of view in the i-th experiment; θ i It represents the deflection angle of the center line of the chip relative to the field of view in the i-th experiment, and n represents the total number of experiments.

[0028] Furthermore, the camera cache image is obtained by the following method:

[0029] Build a dynamically scalable thread pool and divide the image into m images horizontally for asynchronous parallel transmission according to the number of remaining idle threads m on the CPU.

[0030] Furthermore, the method further comprises:

[0031] Calculate the exposure time t according to the accuracy requirements and the crystal movement speed e ;

[0032] exist In the case of , where v is the object's speed and e is the accuracy of the moving direction, the camera is allowed to take a single shot directly at the flying position;

[0033] exist In the case of , taking pictures in the flying position based on the shaking shutter coding exposure technology to obtain the captured image;

[0034] Determining the size of the smear in the captured image;

[0035] When the size of the smear is not within the set accuracy threshold, a deblurring operation is performed on the captured image based on a non-blind deconvolution method.

[0036] According to a second aspect of the present invention, there is provided a flying wafer posture detection device based on real-time vision, the device comprising:

[0037] The advance processing module is configured to allocate and process the time-consuming operations required for image processing in advance before the material moves into the field of view, and trigger the photo taking after the timing to ensure high repeatability of the photo taking timing. The time-consuming operations include memory copying. The high repeatability of the photo taking timing means that the position of the chip is within the set range in the field of view each time the photo taking is triggered in each solid crystal taking path.

[0038] According to a third aspect of the present invention, there is provided a readable storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the method as described above.

[0039] The present invention has at least the following beneficial effects:

[0040] 1. The present invention can ensure that the visual system should be real-time and compress the image transmission time as much as possible, and perform "advanced processing" on visual related operations to avoid the visual data processing time being too long and causing the robot arm movement to pause.

[0041] 2. The present invention can realize how to avoid or reduce the influence of motion smear on the photographing accuracy under the condition of high speed without speed reduction, so as to ensure the final crystal assembly accuracy.

[0042] 3. When using visual flying camera, the high-speed and high-precision die bonding process has high requirements for repeatability, speed and accuracy. Only when motion control and visual processing are both real-time and the visual images captured at high speed are not affected by motion smear can their accuracy be guaranteed. Combining the two can ensure the continuity of the robot arm's movement, the reliability of accuracy and high efficiency at high speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flow chart of a method for detecting wafer posture by flying camera based on real-time vision according to an embodiment of the present invention is shown.

[0044] Figure 2 FIG. 4 is a schematic diagram showing wafer movement according to an embodiment of the present invention.

[0045] Figure 3 A vt diagram is shown when the stop-shoot technique is used.

[0046] Figure 4 The vt diagram during speed reduction segmented operation is shown.

[0047] Figure 5 The vt diagram when running without deceleration is shown.

[0048] Figure 6 A flowchart of image processing steps in a flying wafer posture detection method based on real-time vision according to an embodiment of the present invention is shown.

[0049] Figure 7 A schematic diagram of an image taken using the Coded Exposure by Flutter Shutter technology according to an embodiment of the present invention is shown.

[0050] Figure 8 A structural diagram of a flying wafer posture detection device based on real-time vision according to an embodiment of the present invention is shown.

[0051] Fig. 9 A structural diagram of a flying wafer posture detection device based on real-time vision according to an embodiment of the present invention when an image processing module is provided is shown. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present invention. For the various steps described herein, if there is no necessity for a causal relationship between each other, the order in which they are described as examples herein should not be regarded as a limitation, and those skilled in the art should know that they can be adjusted in order, as long as the logic between them is not destroyed, resulting in the inability to implement the entire process.

[0053] The embodiment of the present invention provides a flying shot wafer posture detection method based on real-time vision. The method is applied to high-speed and high-precision die bonding process scenarios. For high-speed and high-precision die bonding process scenarios, the requirements for repeatability, speed and accuracy are relatively high. When using visual flying shots, the real-time performance of visual processing is equally important. Ensuring real-time performance can ensure the continuity of the robot arm movement and the possibility of speeding up.

[0054] If the vision system does not have real-time capability, it will usually face the following serious problems: individual data packets of GigE Vision or USB3Vision cameras will not be transmitted, resulting in corrupted or only partially transmitted images. The reaction time is affected by other applications, resulting in long and uncertain processing, which affects subsequent processes. Therefore, one of the purposes of the present invention is to ensure that the vision system should have real-time performance and compress the image transmission time as much as possible and perform "advanced processing" on vision-related operations to avoid the long visual data processing time causing the robot movement to stop.

[0055] like Figure 1As shown, the method includes step S100, which allocates the time-consuming operations required for image processing in advance before the material moves to the field of view, and triggers the photo taking after the timing to ensure high repeatability of the photo taking timing, wherein the time-consuming operations include memory copying, and the high repeatability of the photo taking timing means that the position of the chip is within the set range in the field of view each time the photo taking is triggered in each solid crystal taking path.

[0056] Step S100 can be summarized as an advance processing step, in which the method of judging whether the position of the wafer when the photo is triggered each time in each solid crystal path is within the set range in the field of view can be:

[0057] A set posture (standard posture) is set, and the deviation between the posture of the chip when the camera is triggered and the set posture is calculated. If the deviation is within the first threshold, it is determined that the posture of the chip when the camera is triggered is within the set range in the field of view. If the deviation is within the second threshold, correction is performed. If the deviation is within the third threshold, material is discarded.

[0058] It should be noted that the first threshold can be the set maximum allowable error, the second threshold is determined according to the tolerance of the standard posture, and the third threshold is to judge that the deviation between the current displacement and the standard posture is too large, resulting in it being unable to correspond to the standard posture through correction, at which time the material is thrown away.

[0059] In a specific embodiment, in order to avoid data packet loss, the camera module provides a large memory buffer and a real-time network or XHCI driver, and its response to the received GigE Vision or USB3 Vision image can occur in a real-time environment within a few microseconds. And it can seamlessly transmit and connect with EtherCAT bus communication device data.

[0060] By comparing the transmission parameters of USB3.0 and GigE as shown in Table 1 below, it can be seen that the transmission capacity of the USB3.0 protocol far exceeds that of GigE. Therefore, choosing USB3.0 can effectively shorten the transmission time.

[0061] Table 1. Transmission parameters of USB3.0 and GigE

[0062] type USB3.0 GigE bandwidth 500MB / S 125MB / S Transfer rate 5Gbps 1000Mbps

[0063] When reading the camera cache image, a dynamically scalable thread pool is constructed. According to the number of idle threads remaining in the CPU, m images are divided horizontally for asynchronous parallel transmission (in some cases, processing data in the horizontal direction can more effectively utilize the parallelism of the hardware, such as using the SIMD instruction set). Due to the difference in processing speed of each thread, each thread will be released autonomously after completion. This can further reduce the time consumption of image transmission.

[0064] The so-called "advanced processing" means that the time-consuming operations such as memory copy required for image processing are allocated in advance before the material moves into the field of view, and the photo is triggered after the timing, so as to avoid the time consumption caused by the hard trigger chain of the timer and ensure the high repeatability of the photo timing. The so-called "high repeatability of the photo timing" means that in each solid crystal removal path, due to the dual real-time nature of the motion and visual systems, the position of the chip each time the photo is triggered is almost the same in the field of view. If the position deviation occurs during a certain photo, it means that the picking deviation is caused by factors such as vibration, and correction is required. If the deviation is too large, the material is thrown away.

[0065] like Figure 2 As shown, assuming that the crystal retrieval position is set to 0° and the crystal fixing position is set to 90°, first take a picture of the standard position during flying. The so-called standard position refers to whether the verticality of the center line drawn from the crystal midpoint from the crystal retrieval position to the flying position and then to the crystal fixing position and the center line of the crystal at the crystal retrieval position is within the tolerance ε. The so-called tolerance ε means that even if there is a certain angle deviation when the crystal is placed during the curing process of the solder paste, it will be corrected to the center line position through its curing characteristics. This value is an empirical value tested through multiple experiments.

[0066]

[0067]

[0068] The rod length is l, α is the arc, s is the total arc length from the crystal taking position to the flying shooting position, s a is the arc length of the acceleration phase, s c is the arc length of the constant speed stage, s, v c , a are known items in the process design, t1 is the acceleration time, t2 is the partial constant speed time to the flying position, and the total time t required to move to the flying position can be calculated.

[0069] Since the operation control system is real-time, t is certain. And the visual system is also real-time. Both run in kernel mode. The data interaction trigger can be considered to have zero delay. When the object runs to a certain advance point of the flying position on the s path, a signal is sent to trigger the timing of the visual system. When the timing time is up, the flying position takes a picture.

[0070] Calibrate the x, y, and θ of the crystal standard pose relative to the field of view, and take the average value through multiple experiments:

[0071]

[0072] x refers to the lateral coordinate of the chip center relative to the field of view in the standard position, y refers to the longitudinal coordinate of the chip center relative to the field of view in the standard position, and θ refers to the deflection angle of the chip center line relative to the field of view in the standard position.

[0073] In the normal production process, the robot arm sucks the wafer from the wafer picking position, and then reaches the position that triggers the visual system to advance processing, and then pre-applies for threads and memory and opens up copy space. When reading the camera cache image, a dynamic scaling thread pool is used for transmission to further reduce time consumption. Image processing obtains x r ,y r ,θ r , first of all, r Make a judgment:

[0074] |θ r -θ|<=ε

[0075] If the above formula is met, then x r ,y r Take the difference: Δx = x r -x、Δy=y r -y. Send to the operation control system.

[0076] If the tolerance is not met, the material will be discarded and sent to the transportation control system.

[0077] In some embodiments, in order to ensure the accuracy of the visual images captured at high speed, the most important problem that must be overcome is the ghosting. The ghosting is formed when there is relative motion between the shooting target and the camera system at the moment of exposure. To solve this problem, the more traditional method is to adopt visual stop shooting, that is, the traditional visual positioning adopts the stop shooting method, that is, the workpiece moves along a predetermined trajectory. When it runs to the shooting point, the speed of the moving axis is reduced to 0 and stops to take pictures, collect images, calculate and process, and correct deviations. After the visual system processing is completed, it continues to complete the next trajectory movement. The vt diagram is shown in the figure Figure 3 shown.

[0078] In view of the problem of low production efficiency caused by traditional visual stop shooting, visual flying shooting is used in some applications with high production efficiency. That is, the mechanism moves along a predetermined trajectory. When it runs to the photo position, the motion axis does not stop moving, and the high-speed capture function is completed instantly. The moving mechanism does not stop during the whole process, thus saving the mechanism running time, meeting the efficiency, and ensuring the running accuracy. However, in the case of insufficient accuracy, it is necessary to reduce the speed and run in sections, but the speed does not drop to 0, as shown in the vt diagram Figure 4 shown.

[0079] While ensuring accuracy, there is no need to slow down. The vt diagram is as follows Figure 5 shown.

[0080] Therefore, after the advance processing, the flying chip posture detection method based on real-time vision also includes an image processing step to realize how to avoid or reduce the influence of motion smear on the shooting accuracy under high-speed and non-speed reduction conditions, thereby ensuring the final crystal assembly accuracy.

[0081] like Figure 6 As shown, the specific steps of image processing include:

[0082] Step S201, calculate the exposure time t according to the accuracy requirement and the crystal movement speed e .

[0083] Step S202, In the case of , v is the object's speed and e is the accuracy of the moving direction, the camera is allowed to take a single shot directly in the flying position.

[0084] If the camera itself supports this exposure time t e , you can directly shoot single shots without motion ghosting.

[0085] Step S203, In this case, the captured image is obtained by taking photos in the flying position based on the shaking shutter coding exposure technology.

[0086] It should be noted that the flutter shutter coded exposure technology refers to Coded Exposure by FlutterShutter technology. The so-called Coded Exposure by Flutter Shutter is divided into two parts: Flutter Shutter and Coded Exposure Photography. The significance of flutter shutter is that the alternating opening and closing of the shutter during the exposure time can retain more frequency domain information than keeping it open all the time, while the role of coded exposure is to obtain an imaging image through a certain regular shutter flutter. The tested 52-bit (10100001110000010100001100111101110101110010011001100111) coding form can enable the entire system to retain the widest frequency response, thereby achieving the highest signal-to-noise ratio and the best imaging effect.

[0087] like Figure 7 As shown in the figure, the images taken by Coded Exposure by Flutter Shutter technology retain as much frequency domain information as possible. Although the image may still be blurred, the degree of blur has been greatly optimized.

[0088] Step S204, determining the size of the smear in the captured image.

[0089] It should be noted that here, the smear can be extracted from the captured image based on the existing technology and the size of the smear can be calculated. Therefore, this embodiment does not describe in detail the process of determining the size of the smear in the captured image.

[0090] Step S205 : when the size of the smear is not within the set accuracy threshold, a deblurring operation is performed on the captured image based on a non-blind deconvolution method.

[0091] It should be noted that the non-blind deconvolution method is an existing image motion blur removal algorithm, which is not elaborated in detail in this embodiment.

[0092] For example, the image obtained by the Coded Exposure by Flutter Shutter technology continues to be judged whether there is motion ghosting. If so, it is determined whether the size of the ghosting is within the accuracy requirements. If not, the non-blind deconvolution method is used to deblur the image. The reason for using the non-blind deconvolution method is that the motion trajectory, speed, direction, etc. of the crystal when it moves through the field of view are all known, and the degree of blur can also be calculated by the object movement speed, exposure time, etc.

[0093] The above method can avoid or reduce motion smear.

[0094] The embodiment of the present invention also provides a flying wafer posture detection device based on real-time vision, such as Figure 8 As shown, the device 800 includes an advance processing module 801, which is configured to allocate and process the time-consuming operations required for image processing in advance before the material moves into the field of view, and trigger the photo taking after the timing to ensure high repeatability of the photo taking timing, wherein the time-consuming operations include memory copying, and the high repeatability of the photo taking timing means that the position of the chip is within the set range in the field of view each time the photo taking is triggered in each solid crystal taking path.

[0095] Before the material moves into the field of view, the time-consuming operations required for image processing are allocated in advance, and the photo is triggered after the timing to ensure high repeatability of the photo timing. The time-consuming operations include memory copying. The high repeatability of the photo timing means that the position of the chip is within the set range in the field of view each time the photo is triggered in each solid crystal path.

[0096] In some embodiments, the advanced processing module 801 is further configured to set a set posture, calculate the deviation between the posture of the chip when the camera is triggered and the set posture, if the deviation is within a first threshold, determine that the posture of the chip when the camera is triggered is within a set range in the field of view, if the deviation is within a second threshold, correct the deviation, and if the deviation is within a third threshold, discard the material.

[0097] In some embodiments, the advance processing module 801 is further configured to:

[0098] Obtaining the standard posture during the flying shot, the standard posture during the flying shot means that the verticality of the center line drawn from the crystal midpoint from the crystal taking position to the flying shot position and then to the crystal fixing position and the center line of the crystal at the crystal taking position is within the error tolerance ε, the error tolerance ε means that even if there is an angle deviation when the crystal is placed during the curing process of the solder paste, it will be corrected to the center line position through the curing characteristics of the solder paste;

[0099] Determine the timing time according to the time when the chip moves to the flying position, and when the chip moves to the set advance point, send the timing time to the camera, and the camera takes a picture of the flying position after reaching the timing time;

[0100] According to the standard posture during the flying shot, determine the lateral coordinate x of the center of the wafer relative to the field of view, the longitudinal coordinate y of the center of the wafer relative to the field of view, and the deflection angle θ of the center line of the wafer relative to the field of view in the standard posture;

[0101] Obtain the camera cache image, and obtain the lateral coordinate x of the center of the chip in the cache image relative to the field of view according to the camera cache image. r , the longitudinal coordinate y of the chip center relative to the field of view r and the deflection angle θ of the wafer centerline relative to the field of view r ;

[0102] Construct the following judgment formula:

[0103] |θ r -θ|<=ε

[0104] If θ r If the judgment formula is met, then x r ,y r Take the difference: Δx = x r -x、Δy=y r -y, and send it to the operation control system;

[0105] If θ r If it does not meet the judgment formula, the material will be thrown away.

[0106] In some embodiments, the advance processing module 801 is further configured to calculate the fault tolerance ε by the following formula:

[0107]

[0108] Among them, ε i represents the error tolerance in the i-th experiment, and n represents the total number of experiments.

[0109] In some embodiments, the advance processing module 801 is further configured to calculate the time for the wafer to move to the flying position by the following formula:

[0110]

[0111] Where l is the rod length, α is the arc, s is the total arc length of the wafer from the crystal taking position to the flying position, s a is the arc length of the acceleration phase, s c is the arc length of the constant speed stage, s, v c , a are known items in process design, t1 is the acceleration time, t2 is the constant speed time of the wafer from the flying position, and t is the total time taken for the wafer to move to the flying position.

[0112] In some embodiments, the advanced processing module 801 is further configured to calculate the lateral coordinate x of the center of the wafer relative to the field of view, the longitudinal coordinate y of the center of the wafer relative to the field of view, and the deflection angle θ of the center line of the wafer relative to the field of view in the standard posture by the following formula:

[0113]

[0114] Among them, x i represents the lateral coordinate of the center of the wafer relative to the field of view in the i-th experiment; y i represents the longitudinal coordinate of the center of the wafer relative to the field of view in the i-th experiment; θ i It represents the deflection angle of the center line of the chip relative to the field of view in the i-th experiment, and n represents the total number of experiments.

[0115] In some embodiments, the advanced processing module 801 is further configured to obtain the camera cache image by the following method:

[0116] Build a dynamically scalable thread pool and divide the image into m images horizontally for asynchronous parallel transmission according to the number of remaining idle threads m on the CPU.

[0117] In some embodiments, Fig. 9 As shown, the apparatus 800 further includes an image processing module 802, and the image processing module 802 is configured to:

[0118] Calculate the exposure time t according to the accuracy requirements and the crystal movement speed e ;

[0119] exist In the case of , where v is the object's speed and e is the accuracy of the moving direction, the camera is allowed to take a single shot directly at the flying position;

[0120] exist In the case of , taking pictures in the flying position based on the shaking shutter coding exposure technology to obtain the captured image;

[0121] Determining the size of the smear in the captured image;

[0122] When the size of the smear is not within the set accuracy threshold, a deblurring operation is performed on the captured image based on a non-blind deconvolution method.

[0123] It should be noted that the various device structures described in this embodiment belong to the same technical concept as the previously described method, and achieve the same technical effect through the same principle, which will not be repeated here.

[0124] An embodiment of the present invention further provides a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method described in any of the above embodiments.

[0125] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present invention with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of this application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0126] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the present invention. This should not be interpreted as a feature of an invention that is not claimed for protection being necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of the embodiments of a specific invention. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the full scope of the equivalent forms of the attached claims and these claims.

Claims

1. A flying wafer posture detection method based on real-time vision, characterized in that: The method comprises: Before the material moves to the field of view, the time-consuming operations required for image processing are allocated in advance, and the photo is triggered after the timing to ensure high repeatability of the photo timing. The time-consuming operations include memory copying. The high repeatability of the photo timing means that the position of the wafer is within the set range in the field of view each time the photo is triggered in each solid crystal removal path; The time-consuming operations required for image processing are allocated in advance before the material moves to the field of view, and the photo is triggered after the timing to ensure high repeatability of the photo-taking timing, specifically including: Obtaining the standard posture during the flying shot, the standard posture during the flying shot means that the verticality of the center line drawn from the crystal midpoint from the crystal taking position to the flying shot position and then to the crystal fixing position and the center line of the crystal at the crystal taking position is within the error tolerance ε, the error tolerance ε means that even if there is an angle deviation when the crystal is placed during the curing process of the solder paste, it will be corrected to the center line position through the curing characteristics of the solder paste; Determine the timing time according to the time when the chip moves to the flying position, and when the chip moves to the set advance point, send the timing time to the camera, and the camera takes a picture of the flying position after reaching the timing time; According to the standard posture during the flying shot, determine the lateral coordinate x of the center of the wafer relative to the field of view, the longitudinal coordinate y of the center of the wafer relative to the field of view, and the deflection angle θ of the center line of the wafer relative to the field of view in the standard posture; Obtain the camera cache image, and obtain the lateral coordinate x of the center of the chip in the cache image relative to the field of view according to the camera cache image. r , the longitudinal coordinate y of the chip center relative to the field of view r and the deflection angle θ of the wafer centerline relative to the field of view r ; Construct the following judgment formula: |θ r -θ|<=ε If θ r If the judgment formula is met, then x r ,y r Take the difference: Δx = x r -x、Δy=y r -y, and send it to the operation control system; If θ r If the judgment formula is not met, the material is thrown away; The fault tolerance ε is calculated by the following formula: Among them, ε i represents the error tolerance in the i-th experiment, and n represents the total number of experiments.

2. The method according to claim 1, characterized in that A set posture is set, and the deviation between the posture of the chip when the camera is triggered and the set posture is calculated. If the deviation is within the first threshold, it is determined that the posture of the chip when the camera is triggered is within the set range in the field of view. If the deviation is within the second threshold, correction is performed. If the deviation is within the third threshold, material is discarded.

3. The method according to claim 1, characterized in that The time it takes for the chip to move to the flying position is calculated using the following formula: Where l is the rod length, α is the arc, s is the total arc length of the wafer from the crystal taking position to the flying position, s a is the arc length of the acceleration phase, s c is the arc length of the constant speed stage, s, v c , a are known items in process design, t1 is the acceleration time, t2 is the constant speed time of the wafer from the flying position, and t is the total time taken for the wafer to move to the flying position.

4. The method according to claim 1, characterized in that: The lateral coordinate x of the center of the chip relative to the field of view, the longitudinal coordinate y of the center of the chip relative to the field of view, and the deflection angle θ of the center line of the chip relative to the field of view in the standard posture are calculated by the following formula: Among them, x i represents the lateral coordinate of the center of the wafer relative to the field of view in the i-th experiment; y i represents the longitudinal coordinate of the center of the wafer relative to the field of view in the i-th experiment; θ i It represents the deflection angle of the center line of the chip relative to the field of view in the i-th experiment, and n represents the total number of experiments.

5. The method according to claim 1, characterized in that Get the camera cache image by the following method: Build a dynamically scalable thread pool and divide the image into m images horizontally for asynchronous parallel transmission according to the number of remaining idle threads m on the CPU.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Calculate the exposure time t according to the accuracy requirements and the crystal movement speed e ; exist In the case of , where v is the object's speed and e is the accuracy of the moving direction, the camera is allowed to take a single shot directly at the flying position; exist In the case of , taking pictures in the flying position based on the shaking shutter coding exposure technology to obtain the captured image; Determining the size of the smear in the captured image; When the size of the smear is not within the set accuracy threshold, a deblurring operation is performed on the captured image based on a non-blind deconvolution method.

7. A flying wafer posture detection device based on real-time vision, characterized in that: The device comprises: The advance processing module is configured to allocate and process the time-consuming operations required for image processing in advance before the material moves to the field of view, and trigger the photo taking after the timing to ensure high repeatability of the photo taking timing, wherein the time-consuming operations include memory copying, and the high repeatability of the photo taking timing means that the position of the wafer when the photo taking is triggered each time in each die bonding path is within the set range in the field of view; The advanced processing module is further configured to obtain a standard posture during flying, wherein the standard posture during flying indicates that the verticality of the center line drawn from the crystal midpoint from the crystal taking position to the flying position and then to the crystal fixing position and the center line of the crystal at the crystal taking position is within the error tolerance ε, wherein the error tolerance ε means that even if there is an angle deviation when the crystal is placed during the curing process of the solder paste, the error will be corrected to the center line position through the curing characteristics of the solder paste; Determine the timing time according to the time when the chip moves to the flying position, and when the chip moves to the set advance point, send the timing time to the camera, and the camera takes a picture of the flying position after reaching the timing time; According to the standard posture during the flying shot, determine the lateral coordinate x of the center of the wafer relative to the field of view, the longitudinal coordinate y of the center of the wafer relative to the field of view, and the deflection angle θ of the center line of the wafer relative to the field of view in the standard posture; Obtain the camera cache image, and obtain the lateral coordinate x of the center of the chip in the cache image relative to the field of view according to the camera cache image. r , the longitudinal coordinate y of the chip center relative to the field of view r and the deflection angle θ of the wafer centerline relative to the field of view r ; Construct the following judgment formula: |θ r -θ|<=ε If θ r If the judgment formula is met, then x r ,y r Take the difference: Δx = x r -x、Δy=y r -y, and send it to the operation control system; If θ r If the judgment formula is not met, the material is thrown away; The fault tolerance ε is calculated by the following formula: Among them, ε i represents the error tolerance in the i-th experiment, and n represents the total number of experiments. 8 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, executes the method according to claim 1 .

Images