Full-automatic lead frame and cardboard visual recognition system and its control method

Abstract

The present invention provides a full-automatic lead frame and cardboard visual recognition system and its control method, belonging to the field of artificial intelligence technology in the semiconductor packaging industry. The full-automatic lead frame and cardboard visual recognition system includes a frame, and an identification mechanism, a total transmission mechanism, a sorting mechanism, a sub-transmission mechanism and an industrial control computer are arranged in the frame; the present invention uses robot intelligent sorting to replace operator manual sorting and dust removal, which effectively saves manpower and labor intensity, ensures the sorting accuracy rate, and improves the sorting efficiency. The control method applied to the above system takes multi-angle and all-round pictures of the material images through the camera component, and identifies the material type and obtains the material position information through image recognition and deep learning algorithms; further, the two materials after sorting are separately transmitted and processed respectively, realizing the assembly line function and saving labor and time costs.

Description

Technical Field

[0001] The invention belongs to the technical field of artificial intelligence in the semiconductor packaging industry, and specifically relates to a full-automatic lead frame and cardboard visual recognition system and a control method thereof. Background Art

[0002] In the lead frame packaging industry, there are cardboard boards in the materials after the quality inspection of lead frame products. These cardboard boards need to be processed and recycled. However, there are also lead frames of different models in the materials. Therefore, the materials need to be sorted before collecting the cardboard boards.

[0003] The existing technology in this field separates the lead frames and cardboard boards through manual operation, and then takes the cardboard boards to a dust removal machine for dust removal. After the dust removal is completed, the cardboard boards are collected. In the above process, one operator is required for sorting, one operator for feeding, and one operator for receiving. Therefore, three operators are required to complete the processing and recycling of the cardboard boards; and in this working mode, it takes at least 2 hours to collect 10,000 cardboard boards. To sum up, the existing method of manually recycling cardboard boards has many drawbacks, which are manifested as high labor consumption, long working hours, and unstable sorting accuracy in actual use. Summary of the Invention

[0004] The purpose of the invention is to provide a full-automatic lead frame and cardboard visual recognition system and a control method thereof, which can intelligently sort the materials after the quality inspection of the lead frames automatically and dust and collect the cardboard boards in the materials, so as to solve the problems of high labor consumption, long working hours, and unstable sorting accuracy existing in the traditional method of manually recycling cardboard boards.

[0005] The invention is realized by adopting the following technical solutions:

[0006] A full-automatic lead frame and cardboard visual recognition system includes a frame, in which an identification mechanism is arranged for identifying lead frames and cardboard boards of different models and obtaining the position information of the lead frames and cardboard boards; a general transmission mechanism for transmitting the materials to the identification area of the identification mechanism; a sorting mechanism for sorting the lead frames and cardboard boards; a sub-transmission mechanism for respectively transmitting the sorted lead frames and cardboard boards to the corresponding next working positions; and an industrial control computer for receiving and processing the signals sent by the identification mechanism and controlling the actions of the sorting mechanism and the sub-transmission mechanism.

[0007] In the above solution, for the materials transported to the sorting position, the identification mechanism automatically identifies lead frames and cardboard of different models. The identification mechanism sends the identification results to the industrial control computer in the form of signals. The industrial control computer receives and processes the signals through the built-in control system, and then sends signals to control the sorting mechanism to respectively grab and place lead frames and cardboard of different models. The sorted lead frames and cardboard reach the corresponding stacking and collection positions through their respective transmission mechanisms. For the above process, an operator can easily complete it, and the identification and sorting of materials are stably achieved through the identification mechanism and the sorting mechanism, ensuring the sorting accuracy rate and at the same time accelerating the sorting efficiency.

[0008] Furthermore, the identification mechanism includes an overhead camera assembly and a side-view camera assembly; the overhead camera assembly is located above the total transmission mechanism, and the side-view camera assembly is located at the end of the total transmission mechanism. The side-view camera assembly is a binocular camera.

[0009] In the above solution, in addition to identifying the type of materials, the identification mechanism can also obtain the position information of the materials, so that the sorting mechanism can make corresponding adjustments according to the material position during grasping. The overhead camera assembly can obtain the position information of the materials in the XY direction, and the side-view camera assembly can obtain the height position information of the materials; both the overhead camera assembly and the side-view camera assembly include industrial cameras, lenses and strip combination light sources; the difference is that the camera of the side-view camera assembly is a binocular camera, which is built by two monocular cameras (taking the exposed surface of the camera lens as the front view surface, the two monocular cameras are the left camera and the right camera respectively). The ranging accuracy of the binocular camera is relatively high, and it can accurately obtain the height position information of the materials, thus ensuring that the sorting mechanism can stably grab the materials.

[0010] Furthermore, the total transmission mechanism includes a V-belt conveyor, which is used to convey material boxes; a V-belt conveyor baffle is provided at the end of the V-belt conveyor.

[0011] In the above solution, the material box is used to hold the materials to be sorted, and the V-belt conveyor is driven by a motor. When the material box is conveyed to the position of the baffle of the V-belt conveyor, it means that the material box has reached the recognition area of the recognition mechanism. At this time, the V-belt conveyor stops running. After all the materials in the material box have been recognized, the V-belt conveyor starts running again to convey the next material box to the recognition area. The start and stop of the V-belt conveyor are automatically realized. When the material box touches or leaves the contact sensor provided on the surface of the baffle of the V-belt conveyor, the contact sensor sends the corresponding signal to the single-chip microcomputer chip, and then the single-chip microcomputer chip controls the motor to shut down or start, so that the V-belt conveyor stops or starts running. The single-chip microcomputer chip receives the sensor signal and controls the motor switch through the program principle in the existing technology, which will not be elaborated here. In addition, those skilled in the art can use any other existing technology that can achieve the same technical effect to replace the above contact sensor and single-chip microcomputer chip.

[0012] Further, the V-belt conveyor is divided into several sections.

[0013] In the above solution, by dividing the V-belt conveyor into several sections, when there is a material box reaching the baffle position on the last section of the conveyor, the V-belt conveyor does not need to stop running as a whole, and the conveyors of the remaining sections can normally convey the material boxes, so that the storage capacity of the material boxes can be effectively increased, the operator can be avoided from frequently loading materials, and the labor intensity can be reduced. Each section of the V-belt conveyor is driven by a single motor separately, and there is no sudden change in the structure at the connection of every two sections of the V-belt conveyor, ensuring the smoothness of the conveyance of the material boxes.

[0014] Further, the sorting mechanism includes a sorting robot, and the sorting robot places the materials on the corresponding sub-conveying mechanism after sucking them with a suction tool.

[0015] In the above solution, the sorting robot is the execution mechanism of the sorting action and performs actions according to the category information and position information obtained by the recognition mechanism. The sorting robot contacts and sucks the materials through the suction tool, which is convenient to use, has a fast suction speed, and does not damage the materials, that is, it avoids the problem that mechanical grippers and the like may cause scratches on the surface of the cardboard.

[0016] Further, the sub-conveying mechanism includes a waste conveying mechanism and a cardboard conveying mechanism. The waste conveying mechanism includes a belt conveyor, and the belt conveyor is used to convey the waste box containing the lead frame to the palletizing position. A centering device and a belt conveyor baffle are provided at the end of the belt conveyor. The cardboard conveying mechanism includes a cardboard conveyor, and the cardboard conveyor is used to convey the cardboard to the position where the cardboard receiving tray is located. A dust removal mechanism is provided on the cardboard conveyor.

[0017] In the above solution, the waste box on the belt conveyor is the material box on the V-belt conveyor. The first waste box is manually placed on the belt conveyor, and the subsequent material boxes are transferred to the belt conveyor by the sorting robot from the empty material boxes on the V-belt conveyor. When the recognition mechanism recognizes that the materials in the material box on the V-belt conveyor have been sorted, it means that the waste box on the belt conveyor is full. At this time, the industrial control computer controls the belt conveyor to operate and convey the waste box to the end of the conveyor. When the waste box reaches the position of the baffle of the belt conveyor, the conveyor automatically stops operating, and the centering device acts to place the waste box in the middle of the belt for subsequent palletizing of the waste box. At the same time, the sorting robot sucks and places the empty material box on the V-belt conveyor on the belt conveyor as a new waste box. The palletizing of the waste box is realized by a robot or an automatic robotic arm in the prior art. The principle of the belt conveyor stopping operation is the same as that of the V-belt conveyor stopping operation, so the above two points will not be elaborated here. The cardboard conveyor can directly convey cardboard. When the cardboard passes through the dust removal mechanism on the conveyor, the dust and impurities on the surface of the cardboard will be removed, and then the cardboard reaches and automatically falls into the cardboard receiving tray. The operator only needs to collect the cardboard in the cardboard receiving tray.

[0018] Further, the dust removal mechanism includes an electrostatic dust removal component. The electrostatic dust removal component includes two dust collection film rolls, which are respectively placed above and below the cardboard conveyor, and rubber rollers are provided between the two dust collection film rolls and the cardboard conveyor.

[0019] In the above solution, when the cardboard passes through the electrostatic dust removal component, the dust and impurities on the surface of the cardboard first adhere to the rubber rollers and then are transferred to the dust collection film rolls, so that the dust and impurities will no longer be transferred to the surface of the cardboard and will not flow to the dust-free area of the production line.

[0020] A control method for a full-automatic lead frame and cardboard visual recognition system is applied to the visual recognition system described above, and specifically includes the following steps:

[0021] S1: The system starts, and the V-belt conveyor conveys the material box to the sorting position at the end of the V-belt conveyor. The material box contains the lead frame and cardboard to be sorted.

[0022] S2: The recognition mechanism performs type recognition and position information acquisition on both the lead frame and cardboard in the material box.

[0023] S3: The recognition mechanism sends a signal to the industrial control computer. The industrial control computer receives and processes the signal, and then the industrial control computer controls the sorting robot to perform corresponding actions according to the different types of materials.

[0024] S4: According to the instruction of the industrial control computer, the sorting robot moves to the corresponding working position, and then the suction tool on the sorting robot opens to the corresponding width and sucks the lead frame or cardboard.

[0025] S5: The sorting robot places the lead frame into the waste box on the belt conveyor and places the cardboard on the cardboard conveyor.

[0026] S6: After the waste box is filled, the industrial control computer controls the belt conveyor to start running and conveys the waste box to the stacking position at the end of the belt conveyor; when the recognition mechanism recognizes the cardboard, the industrial control computer controls the cardboard conveyor to start running, and conveys the cardboard to the cardboard receiving tray after dust removal by the dust removal mechanism.

[0027] In the above solution, the operator only needs to place the material box containing the lead frame and cardboard on the head section of the V-belt conveyor, and then the system automatically runs steps S1-S6 to complete the sorting of the lead frame and cardboard and dust removal of the cardboard. Finally, the operator can collect the dust-removed cardboard collected in the cardboard receiving tray. Compared with the traditional manual sorting method that requires at least three operators, the present invention only needs one operator to complete the whole operation, reducing the labor intensity by 80% and saving 67% of the manpower. The annual manpower cost savings can reach 200,000 yuan; and 10,000 pieces of cardboard can be processed in one hour using this system, doubling the efficiency compared with the prior art.

[0028] Further, in S2, the type recognition performed by the recognition mechanism can specifically identify different models of the two types of materials respectively, and the position information obtained by the recognition mechanism includes the XY-direction position information of the material and the height position information of the material.

[0029] Among them, the material type recognition and the acquisition of the XY-direction position information are realized by the overhead camera assembly, and specifically include the following sub-steps:

[0030] S21-1: The camera takes a picture to obtain an image.

[0031] S21-2: Convert the image to a grayscale image and then perform morphological processing, and finally identify the material type and XY-direction position information through threshold and area judgment.

[0032] S21-3: Send the type and position information in S21-2 to the industrial control computer.

[0033] The acquisition of the material height position information is realized by the side-view camera assembly, and specifically includes the following sub-steps:

[0034] S22-1: Establish camera parameters, calculate the internal and external parameters through the camera calibration algorithm to eliminate the influence of the radial distortion and tangential distortion of the camera; the specific method of calculating using the camera calibration algorithm can adopt one of the prior arts to calculate the internal parameters including the focal length f, the imaging origin cx and cy, 5 distortion coefficients (k1, k2, k3, p1, p2), and the external parameters including the rotation matrix R and the translation vector T.

[0035] S22-2: Binocular correction and matching. Using internal and external parameters, undistortion and row alignment are performed on the two views captured by the binocular camera, so that the imaging coordinate origins of the two views are the same, the optical axes of the two cameras are parallel, the imaging planes of the two views are coplanar, and the epipolar lines are row-aligned.

[0036] S22-3: Calculate depth information. Using a deep learning object detection algorithm, identify and calibrate the dark area of the view obtained by the left camera in the binocular camera to obtain the object detection frame. Then, perform disparity calculation on this view through a stereo matching method, and obtain the depth information of the object according to the conversion relationship from disparity to depth.

[0037] S22-4: Calculate height information. Using the depth information of the object, correct the top and bottom edges of the object detection frame to accurately frame the object to obtain the image coordinate system. Then, calculate the object height information using the conversion relationships from the image coordinate system to the camera coordinate system and from the camera coordinate system to the world coordinate system.

[0038] Furthermore, in S6, the judgment of the waste box being full is realized by the recognition mechanism. Specifically, when the overhead camera component detects that the material box at the end of the V-belt conveyor is empty, it means that the waste box on the belt conveyor is full.

[0039] The beneficial effects achieved by the present invention are as follows:

[0040] (1) The fully automatic lead frame and cardboard visual recognition system realizes the automatic separation of the lead frame and the cardboard through the recognition mechanism and the sorting mechanism, and dust removal is performed on the cardboard through the dust removal mechanism. Compared with the traditional sorting method, the present invention uses robot intelligent sorting to replace operator manual sorting and dust removal, effectively saving manpower and labor intensity while ensuring the sorting accuracy rate and improving the sorting efficiency.

[0041] (2) The control method of the fully automatic lead frame and cardboard visual recognition system captures the material images from multiple angles and in all directions through the camera component, and identifies the material type and obtains the material position information through image recognition and deep learning algorithms. Furthermore, the two sorted materials are transported separately and processed individually, realizing the assembly line function and saving labor and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic internal structure diagram of the recognition system in an embodiment of the present invention;

[0043] Figure 2 is a schematic overall external shape diagram of the recognition system in an embodiment of the present invention;

[0044] Figure 3 is a schematic structure diagram of the V-belt conveyor in an embodiment of the present invention;

[0045] Figure 4 is Figure 3 The enlarged view of location A in

[0046] Figure 5 is the structural schematic diagram of the manipulator suction cup in the embodiment of the present invention;

[0047] Figure 6 is Figure 5 the structural schematic diagram of the bottom surface of the shown manipulator suction cup;

[0048] Figure 7 is the structural schematic diagram of the electrostatic dust removal device in the embodiment of the present invention;

[0049] Figure 8 is Figure 7 the schematic diagram of the connection relationship between the rubber roller and the dust collection rubber roll in the shown electrostatic dust removal device;

[0050] Figure 9 is the structural schematic diagram of the belt conveyor in the embodiment of the present invention;

[0051] In the figure: 1. Frame; 2. V-belt conveyor; 3. Industrial control computer; 4. Identification mechanism; 5. Belt conveyor; 6. Sorting robot; 7. Dust removal mechanism; 8. Cardboard receiving tray; 9. Side view camera assembly; 10. V-belt conveyor baffle; 11. Top view camera assembly; 12. V-belt conveyor body; 13. V-belt conveyor guiding device; 14. Material box; 15. Lead frame; 16. Suction cup; 17. Linear slide rail 1; 18. Suction cup mounting bracket; 19. Servo motor; 20. Manipulator mounting plate; 21. Cross steering box; 22. Lead screw; 23. Cardboard conveyor; 24. Cardboard; 25. Linear slide rail 2; 26. Electrostatic dust removal assembly; 27. Rubber roller; 28. Dust collection rubber roll; 29. Belt conveyor baffle; 30. Centering device; 31. Belt conveyor body; 32. Waste box. Detailed implementation manners

[0052] To clearly illustrate the solutions in the present invention, the following further explanations are made with reference to the drawings:

[0053] Embodiment 1:

[0054] Please refer to Figures 1 to 2 , a full-automatic lead frame and cardboard vision recognition system, including a frame 1, in which an identification mechanism 4, a total transmission mechanism, a sorting mechanism, a sub-transmission mechanism and an industrial control computer 3 are provided, and the industrial control computer 3 is located below the total transmission mechanism. This recognition system is used to recognize the materials after the quality inspection of the lead frame 15 products. The materials in this embodiment include three kinds of lead frames 15 with different widths, namely type A (width 62 mm), type B (width 70 mm), type C (width 78 mm), and cardboard 24 of type D (width 70 mm).

[0055] The working principle and process of this embodiment are as follows:

[0056] The identification mechanism 4 is used to identify lead frames 15 and cardboard 24 of different models and obtain the position information of the lead frames 15 and cardboard 24; the general transmission mechanism is used to transport materials into the identification area of the identification mechanism 4; the sorting mechanism is used to sort the lead frames 15 and cardboard 24; the sub-transmission mechanism is used to transport the sorted lead frames 15 and cardboard 24 to the corresponding next working positions respectively; the industrial control computer 3 is used to receive and process the signals sent by the identification mechanism 4 and control the actions of the sorting mechanism and the sub-transmission mechanism.

[0057] For the materials transported to the position to be sorted, that is, the identification area of the identification mechanism 4, the identification mechanism 4 automatically identifies the material types (A, B, C, D); then the identification mechanism 4 sends the identification result to the industrial control computer 3 in the form of a signal, and the industrial control computer 3 receives and processes the signal through the built-in control system; subsequently, the industrial control computer 3 sends a signal to control the sorting mechanism to respectively grab and place the aforementioned lead frames 15 and cardboard 24, and the sorted lead frames 15 and cardboard 24 reach the corresponding stacking and collection positions through their respective sub-transmission mechanisms respectively; finally, the lead frames 15 are automatically stacked by the stacking robot externally connected to the system, and the operator collects the cardboard 24.

[0058] Embodiment 2:

[0059] Please refer to Figures 3 to 9 , on the basis of Embodiment 1, the identification mechanism 4 includes a side-view camera assembly 9 and a top-view camera assembly 11; the general transmission mechanism includes a V-belt conveyor 2 and the components used to achieve the driving or connection functions thereon; the sorting mechanism includes a sorting robot 6, and the specific form in this embodiment is a manipulator suction cup; the sub-transmission mechanism is divided into a waste transmission mechanism and a cardboard 24 transmission mechanism. The waste transmission mechanism includes a belt conveyor 5 and its components, and the cardboard 24 transmission mechanism includes a cardboard 24 conveyor 23 and the dust removal mechanism 7 and other components thereon. Among them, the specific connection methods between and within each mechanism are as described below:

[0060] The side-view camera assembly 9 is arranged at the end of the V-belt conveyor 2 through a support frame, and the top-view camera assembly 11 is arranged above the end of the V-belt conveyor 2 through a support frame. The side-view camera assembly 9 and the top-view camera assembly 11 can capture the material images in the material box 14 at the very end on the V-belt conveyor 2. Both the side-view camera assembly 9 and the top-view camera assembly 11 include a Hikvision industrial camera, a lens, and a bar-shaped combined light source; among them, the industrial camera in the side-view camera assembly 9 is a binocular camera, and this binocular camera is built by two monocular cameras (taking the exposed surface of the camera lens as the front view surface, and the two monocular cameras are the left camera and the right camera respectively).

[0061] The V-belt conveyor 2 includes a V-belt conveyor body 12, V-belt conveyor guiding devices 13 are arranged on both sides of the V-belt conveyor body 12, a V-belt conveyor baffle 10 is arranged at the end of the V-belt conveyor body 12, and a material box 14 is conveyed on the V-belt conveyor body 12; the V-belt conveyor body 12 is divided into three sections, each section is driven by a motor. Among them, the driving wheel of the first section and the driven wheel of the second section of the V-belt conveyor body 12 are installed on the same shaft, the driving wheel is connected to the shaft by a key, the driven wheel is a loose pulley and is connected to the shaft through a bearing, and there is no sudden change in the structure at the joints of the first and second sections and the second and third sections of the V-belt conveyor body 12.

[0062] The manipulator suction cup is composed of a manipulator mounting plate 20, a servo motor 19, a cross steering box 21, a lead screw 22, a suction cup mounting bracket 18, a suction cup 16, a linear slide rail 17 and a solenoid valve, etc. The guide rail of the linear slide rail 17 and the servo motor 19 are fixed on the manipulator mounting plate 20, the output shaft of the servo motor 19 is connected to the input shaft of the cross steering box 21, and the two output shafts of the cross steering box 21 are two lead screws 22; the slider of the linear slide rail 17 and four suction cups 16 are fixed on the suction cup mounting bracket 18, and a long hole is opened on the suction cup mounting bracket 18 for manually adjusting the X-direction position of the suction cup 16.

[0063] The belt conveyor 5 includes a belt conveyor body 31, a centering device 30 is arranged at the tail of the belt conveyor body 31, and a belt conveyor baffle 29 is arranged at the end of the belt conveyor body 31; a waste box 32 is conveyed on the belt conveyor body 31. An electrostatic dust removal component 26 is connected to the cardboard conveyor 23 through a linear slide rail 25, and cardboard 24 is conveyed on the cardboard conveyor 23; the electrostatic dust removal component 26 includes a rubber roller 27 and an integrated film 28; the rubber roller 27 and the integrated film 28 are of a drawer-type double-sided design, that is, drawers are arranged above and below the cardboard conveyor 23, and an integrated dust collection film 28 is placed in each of the two drawers, and rubber rollers 27 are connected to the surfaces of the two integrated dust collection films 28 close to the cardboard conveyor 23.

[0064] The working principle and working process of this embodiment are as follows:

[0065] The function of this system is to automatically separate the lead frame 15 (which can be one of lead frame A (width 62 mm), lead frame B (width 70 mm), and lead frame C (width 78 mm), and the same applies hereinafter) and the cardboard 24 (which is cardboard D (width 70 mm), and the same applies hereinafter) in the material box 14; collect the cardboard 24 into the cardboard receiving tray 8 after electrostatic dust removal, and then it is taken away manually; put the lead frame 15 into the waste box 32 and convey it to the end of the belt conveyor 5, and then it is automatically palletized by the palletizing robot external to the system. When using this system, the operator only needs to place the material box 14 containing the lead frame 15 and the cardboard 24 at the head end of the V-belt conveyor 2, and then collect the cardboard in the cardboard receiving tray 8. Among them:

[0066] The operator first places the material box 14 at the head end of the V-belt conveyor 2, that is, the first section of the conveyor. Subsequently, the material box 14 is conveyed to the sorting position by the three-section conveyor. At this time, the motor corresponding to the third section of the conveyor stops working, the third section of the conveyor stops running, and the first and second sections of the conveyor keep running to facilitate the continuous placement of the material box 14; after the material box 14 on the second section of the conveyor is full, the motor corresponding to the second section of the conveyor stops working, and the second section of the conveyor stops running; the first section of the conveyor can place two material boxes 14. If the first section stops running after placing the material box 14, it means that the material boxes 14 on the first and second sections of the conveyor are full, and the operator stops placing. Through the three-section V-belt conveyor 2, the storage capacity of placing materials at one time can be effectively increased, avoiding the operator from frequently interrupting and starting to feed again, thus reducing the labor intensity; and because there is no sudden change at the joints of each section of the conveyor, the smoothness of the conveyance of the material box 14 is ensured.

[0067] The overhead camera assembly 11 is used to identify the lead frame 15 or the cardboard 24, and further judge the specific models of the two materials. In addition, the overhead camera assembly 11 can also obtain the position of the material in the XY direction; the side camera assembly 9 is used to detect the height position of the material. The above two camera assemblies both detect the topmost material in the material box 14 each time, and transmit the identified and obtained information to the industrial control computer 3 for processing.

[0068] After the recognition mechanism transmits the identified and obtained information to the industrial control computer 3, the industrial control computer 3 processes and sends a signal to control the action of the manipulator suction tool. The specific implementation process is as follows: The servo motor 19 starts, drives the lead screw 22 to rotate through the cross steering box 21, so that the suction cup mounting bracket 18 moves along the Y direction; the suction cup mounting bracket 18 moves to the specified position along the linear slide rail 17, that is, the manipulator suction tool opens the corresponding width to make the suction cup 16 move to the appropriate position; the vacuum pipeline solenoid valve is opened, so that the suction cup 16 generates suction to lift the material; finally, according to the material type identified by the recognition mechanism, the material is placed on the corresponding sub-conveying mechanism.

[0069] If the material identified by the identification mechanism is of type A, B, or C (i.e., the lead frame 15), the manipulator suction cup will place it into the waste box 32 on the belt conveyor 5; when the materials in the material box 14 at the end of the V-shaped conveyor 2 are sorted out, the belt conveyor 5 operates to convey the waste box 32 to the end of the conveyor. When the waste box 32 reaches the position of the baffle plate, the conveyor stops rotating, and the centering device 30 acts to place the waste box 32 in the middle position of the belt, so that the palletizing robot externally connected to the system can automatically palletize the waste box 32; at the same time, the manipulator suction cup sucks and places the empty material box 14 sorted out at the end of the V-shaped conveyor 2 onto the belt conveyor 5, and then the manipulator suction cup continues the new round of material suction operation.

[0070] If the material identified by the identification mechanism is of type D (i.e., the cardboard 24), the manipulator suction cup will place it onto the cardboard conveyor 23; the cardboard conveyor 23 operates. When the cardboard 24 passes through the electrostatic dust removal assembly 26, the dust and impurities on the upper and lower surfaces of the cardboard 24 first adhere to the rubber roller 27, and then are transferred to the dust collection film 28; after the cardboard 24 passes through the electrostatic dust removal assembly 26, it reaches the end of the cardboard conveyor 23 and automatically falls into the cardboard receiving tray 8, and finally the operator can collect the cardboard 24 by himself. The aforementioned cleaning method enables the dust and impurities not to be transferred to the surface of the cardboard 24 anymore, nor to flow to other dust-free areas of the system, and through the drawer-type dust collection device with double-sided and double-time cleaning, it is convenient for the operator to replace the dust collection film and maintain the equipment clean.

[0071] In the above-described working principle and working process, the start and stop of the operation of each section of the V-belt conveyor 2 and the stop of the operation of the belt conveyor 5 are automatically realized. Taking the V-belt conveyor 2 as an example, when the material box 14 touches or leaves the contact sensor provided on the surface of the baffle plate 10 of the V-belt conveyor, the contact sensor sends the corresponding signal to the single-chip microcomputer chip, and the single-chip microcomputer chip then controls the corresponding motor to be turned off or started, so that the V-belt conveyor 2 stops or starts operating. Among them, the single-chip microcomputer chip receives the sensor signal and controls the motor switch through the program principle in the prior art, which will not be elaborated here; in addition, those skilled in the art can adopt any other prior art that can achieve the same technical effect to replace the above-mentioned contact sensor and single-chip microcomputer chip. The principle of the belt conveyor 5 stopping operating is the same as the principle of the V-belt conveyor stopping operating, and is realized through the baffle plate 29 of the belt conveyor and the control system in the same way as above.

[0072] Embodiment 3:

[0073] Please refer to Figures 1 to 9 , a control method for a full-automatic lead frame and cardboard vision recognition system, which is applied to the recognition system described in Embodiment 1 and Embodiment 2, and specifically includes the following steps:

[0074] S1: The system starts. The V-belt conveyor 2 transports the material box 14 to the sorting position at the end of the V-belt conveyor 2. Inside the material box 14 are the lead frames 15 and cardboard 24 to be sorted.

[0075] S2: The identification mechanism 4 identifies the types of both the lead frames 15 and cardboard 24 in the material box 14 and obtains their position information.

[0076] S3: The identification mechanism 4 sends a signal to the industrial control computer 3. The industrial control computer 3 receives and processes the signal, and then the industrial control computer 3 controls the manipulator suction cup to perform corresponding actions according to the different types of materials.

[0077] S4: According to the instruction of the industrial control computer 3, the manipulator suction cup moves to the corresponding working position, and then the manipulator suction cup opens to the corresponding width and sucks the lead frame 15 or cardboard 24.

[0078] S5: The manipulator suction cup places the lead frame 15 into the waste box 32 on the belt conveyor 5 and places the cardboard 24 on the cardboard conveyor 23.

[0079] S6: After the waste box 32 is filled, the industrial control computer 3 controls the belt conveyor 5 to start running and transports the waste box 32 to the stacking position at the end of the belt conveyor 5. When the identification mechanism 4 identifies the cardboard 24, the industrial control computer 3 controls the cardboard conveyor 23 to start running and transports the cardboard 24 to the cardboard receiving tray 8 after dust removal by the electrostatic dust removal component 26. The judgment of the waste box 32 being filled is realized by the identification mechanism 4. Specifically, when the top-down camera assembly 11 detects that the state of the material box 14 at the end of the V-belt conveyor 2 is empty, it means that the waste box 32 on the belt conveyor 5 is filled.

[0080] Among them: In the above S2, the type identification performed by the identification mechanism 4 can specifically identify the different models of the two types of materials, and the final result of the identification by the identification mechanism 4 is one of types A, B, C, and D. The position information obtained by the identification mechanism 4 includes the XY-direction position information and the height position information of the material. The material type identification and the XY-direction position information acquisition are realized by the top-down camera assembly 11, and specifically include the following sub-steps:

[0081] S21-1: The camera takes a picture to obtain an image.

[0082] S21-2: The image is converted into a grayscale image and then subjected to morphological processing. Finally, the material type and XY-direction position information are identified through threshold and area judgment.

[0083] S21-3: The type and position information obtained in S21-2 are sent to the industrial control computer 3.

[0084] The acquisition of the height position information of the material is achieved through the side-view camera assembly 9, which specifically includes the following sub-steps:

[0085] S22-1: Establish camera parameters, calculate the internal and external parameters through the camera calibration algorithm to eliminate the influence of radial and tangential distortions of the camera; the specific calculation method of the camera calibration algorithm used in this embodiment is to take 32 checkerboards in the actual scene to calculate the internal parameters including the focal length f, the imaging origin cx and cy, 5 distortion coefficients (k1, k2, k3, p1, p2), and the external parameters including the rotation matrix R and the translation vector T.

[0086] S22-2: Binocular rectification and matching, use the internal and external parameters to eliminate distortions and row alignment of the two views taken by the binocular camera, so that the imaging coordinate origins of the two views are the same, the optical axes of the two cameras are parallel, the imaging planes of the two views are coplanar, and the epipolar lines are row-aligned;

[0087] S22-3: Calculate depth information, use the deep learning object detection algorithm of the yolov5 network and the pytorch framework 1 to identify and calibrate the dark area of the view obtained by the left camera in the binocular camera to obtain the target detection frame, and then calculate the disparity of the view through the stereo matching method, and obtain the depth information of the target according to the conversion relationship between the disparity and the depth;

[0088] S22-4: Calculate height information, use the depth information of the target to correct the top and bottom edges of the target detection frame, accurately frame the target to obtain the image coordinate system, and then calculate the target height information using the conversion relationship from the image coordinate system to the camera coordinate system and from the camera coordinate system to the world coordinate system.

[0089] In the above process, the operator only needs to place the material box 14 containing the lead frame 15 and the cardboard 24 at the head end of the V-belt conveyor 2, and then the system automatically runs steps S1-S6 to complete the sorting of the lead frame 15 and the cardboard 24, and dust the cardboard 24. Finally, the operator can collect the dusted cardboard 24 collected in the cardboard receiving tray 8. Compared with the traditional manual sorting method that requires at least three operators, the present invention only requires one operator to complete the whole operation, reducing the labor intensity by 80% and saving 67% of the manpower. The annual manpower cost savings can reach 200,000 yuan; and the system under this control method can process 10,000 pieces of cardboard 24 in one hour, doubling the efficiency compared with the prior art.

[0090] Of course, the above content is only the preferred embodiment of the present invention and cannot be considered as limiting the scope of the embodiments of the present invention. The present invention is not limited to the above examples. Equivalent changes and improvements made by those of ordinary skill in the art within the essence of the present invention shall fall within the scope covered by the patent of the present invention.

Claims

1. An automatic lead frame and cardboard visual recognition system, characterized in that, Comprising a frame (1), within which are provided: An identification mechanism (4) for identifying lead frames (15) and cardboard (24) of different models and obtaining the position information of the lead frames (15) and cardboard (24); A general transmission mechanism for transporting materials into the identification area of the identification mechanism (4); A sorting mechanism for sorting the lead frames (15) and cardboard (24); A sub-transmission mechanism for respectively transporting the sorted lead frames (15) and cardboard (24) to the corresponding next working positions; An industrial control computer (3) for receiving and processing the signals sent by the identification mechanism (4) and controlling the actions of the sorting mechanism and the sub-transmission mechanism; The general transmission mechanism includes a V-belt conveyor (2) for transporting a material box (14); a V-belt conveyor baffle (10) is provided at the end of the V-belt conveyor (2); The V-belt conveyor (2) is divided into several sections. The V-belt conveyor (2) includes a V-belt conveyor body (12), V-belt conveyor guiding devices (13) are provided on both sides of the V-belt conveyor body (12), a V-belt conveyor baffle (10) is provided at the end of the V-belt conveyor body (12), and the material box (14) is transported on the V-belt conveyor body (12); among them, the V-belt conveyor body (12) is divided into three sections, each section is driven by a motor. The first-stage driving wheel and the second-stage driven wheel of the V-belt conveyor body (12) are installed on the same shaft, the driving wheel is connected to the shaft by a key, the driven wheel is a loose pulley and is connected to the shaft through a bearing, and there is no sudden change in the structure at the joints between the first and second sections and between the second and third sections of the V-belt conveyor body (12); The sub-transmission mechanism includes a waste transmission mechanism and a cardboard transmission mechanism; the waste transmission mechanism includes a belt conveyor (5) for transporting a waste box (32) containing lead frames (15) to the stacking position, and a centering device (30) and a belt conveyor baffle (29) are provided at the end of the belt conveyor (5); the cardboard transmission mechanism includes a cardboard conveyor (23) for transporting the cardboard (24) to the position where the cardboard receiving tray (8) is located, and a dust removal mechanism (7) is provided on the cardboard conveyor (23); The identification mechanism (4) includes an overhead camera assembly (11) and a side-view camera assembly (9); among them, the overhead camera assembly (11) is located above the general transmission mechanism, the side-view camera assembly (9) is located at the end of the general transmission mechanism, and the side-view camera assembly (9) is a binocular camera; The sorting mechanism includes a sorting robot (6), and the sorting robot (6) places the materials on the corresponding sub-transmission mechanism after sucking them with a suction device; The dust removal mechanism (7) includes an electrostatic dust removal assembly (26), and the electrostatic dust removal assembly (26) includes two dust collection films (28). The two dust collection films (28) are respectively placed above and below the cardboard conveyor (23), and rubber rollers (27) are provided between the two dust collection films (28) and the cardboard conveyor (23).

2. A control method for a full-automatic lead frame and cardboard visual recognition system, applied to the visual recognition system described in claim 1, characterized in that, Specifically, it includes the following steps: S1: The system starts, and the V-belt conveyor (2) conveys the material box (14) to the sorting position at the end of the V-belt conveyor (2). The material box (14) contains the lead frame (15) and cardboard (24) to be sorted. S2: The identification mechanism (4) performs type identification and obtains position information for both the lead frame (15) and cardboard (24) in the material box (14). S3: The identification mechanism (4) sends a signal to the industrial control computer (3). The industrial control computer (3) receives and processes the signal, and then the industrial control computer (3) controls the sorting robot (6) to perform corresponding actions according to the different types of materials. S4: According to the instruction of the industrial control computer (3), the sorting robot (6) moves to the corresponding working position, and then the suction cup on the sorting robot (6) opens to the corresponding width and sucks the lead frame (15) or cardboard (24). S5: The sorting robot (6) places the lead frame (15) into the waste box (32) on the belt conveyor (5), and places the cardboard (24) on the cardboard conveyor (23). S6: When the waste box (32) is full, the industrial control computer (3) controls the belt conveyor (5) to start running and conveys the waste box (32) to the palletizing position at the end of the belt conveyor (5). When the identification mechanism (4) identifies the cardboard (24), the industrial control computer (3) controls the cardboard conveyor (23) to start running, and conveys the cardboard (24) to the cardboard receiving tray (8) after dust removal by the dust removal mechanism (7).

3. The control method of the full-automatic lead frame and cardboard visual recognition system according to claim 2, wherein: In the above S2, the type identification performed by the identification mechanism (4) can specifically identify the different models of the two types of materials. The position information obtained by the identification mechanism (4) includes the XY-direction position information and the height position information of the materials. Among them, the material type identification and the acquisition of XY-direction position information are realized by the top-view camera assembly (11), and specifically include the following sub-steps: S21-1: The camera takes a picture and obtains an image. S21-2: The image is converted into a grayscale image and then subjected to morphological processing. Finally, the material type and XY-direction position information are identified through threshold and area judgment. S21-3: Send the type and position information in S21-2 to the industrial control computer (3). The acquisition of the material height position information is realized by the side-view camera assembly (9), and specifically includes the following sub-steps: S22-1: Establish camera parameters, and calculate the internal and external parameters through the camera calibration algorithm to eliminate the influence of radial and tangential distortion of the camera. S22-2: Binocular correction and matching. Use the internal and external parameters to eliminate distortion and row alignment for the two views taken by the binocular camera, so that the imaging coordinate origins of the two views are the same, the optical axes of the two cameras are parallel, the imaging planes of the two views are coplanar, and the epipolar lines are row-aligned. S22-3: Calculate the depth information. Use the deep learning object detection algorithm to identify and calibrate the dark area of the view obtained by one of the cameras in the binocular camera to obtain the object detection frame. Then, perform disparity calculation on this view through the stereo matching method, and obtain the depth information of the object according to the conversion relationship from disparity to depth. S22-4: Calculate the height information. Use the depth information of the target to correct the top and bottom edges of the target detection frame, accurately frame the target to obtain the image coordinate system, and then calculate the target height information using the conversion relationships from the image coordinate system to the camera coordinate system and from the camera coordinate system to the world coordinate system.

4. The control method of the full-automatic lead frame and cardboard vision recognition system according to claim 2, wherein: In the above S6, the judgment of the completion of the waste box (32) being filled is realized by the recognition mechanism (4). Specifically, when the overhead camera assembly (11) detects that the material box (14) at the end of the V-belt conveyor (2) is in an empty state, it means that the waste box (32) on the belt conveyor (5) is filled.

Images