Online visual detection device based on ultra-depth-of-field image fusion
By combining a robotic arm and an ultra-depth-of-field camera into an online visual inspection device, the problems of inspection accuracy and cost of thin-film integrated circuits or MEMS devices have been solved, realizing efficient online visual inspection and automated sorting.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- NINGBO INST OF TECH ZHEJIANG UNIV ZHEJIANG
- Filing Date
- 2025-05-14
- Publication Date
- 2026-04-17
AI Technical Summary
In existing technologies, the detection accuracy requirements of thin-film integrated circuits or MEMS devices are high, but ordinary cameras have insufficient pixels, and ultra-depth-of-field cameras are expensive and not suitable for online production lines. In addition, the parts to be inspected are scattered, making it impossible to achieve efficient visual inspection.
An online visual inspection device based on ultra-depth-of-field image fusion is adopted, which combines a robotic arm and an ultra-depth-of-field camera. The position of the part to be inspected is determined by the visual positioning camera, the robotic arm controls the ultra-depth-of-field camera to take pictures, and the inspection is realized by image fusion technology. Combined with sorting components, automated sorting is performed.
It enables online visual inspection of multiple production lines, reduces equipment investment costs, and improves inspection accuracy and efficiency, making it suitable for automated inspection on production lines.
Smart Images
Figure CN224127950U_ABST
Abstract
Description
Technical Field
[0001] This utility model relates to the field of visual inspection technology, and in particular to an online visual inspection device based on ultra-depth image fusion. Background Technology
[0002] Thin-film integrated circuits are integrated circuits made by using thin films of metal, semiconductor, metal oxide, mixed metal phase, alloy or insulating dielectric with a thickness of less than 1 micrometer, for all components such as transistors, diodes, resistors, capacitors and inductors and their interconnecting leads. They are fabricated through processes such as vacuum evaporation, sputtering and electroplating.
[0003] Larger circuit boards are generally inspected visually using a camera. However, thin-film integrated circuits or MEMS devices are very small and require high inspection accuracy. Ordinary cameras do not have enough pixels. In the current technology, ultra-depth-of-field cameras are expensive. The conditions of the workshop production line are limited, and the parts to be inspected are scattered. Generally, ultra-depth-of-field cameras are installed on fixed brackets in the laboratory to take pictures of thin-film integrated circuits for visual inspection. This method is not very practical and cannot be used for online production inspection on the production line. Utility Model Content
[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an online visual inspection device based on ultra-depth-of-field image fusion.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] An online visual inspection device based on ultra-depth-of-field image fusion includes a gantry and an inspection conveyor belt. A visual positioning camera is provided on the lower side of the gantry and is correspondingly arranged with the inspection conveyor belt. A robotic arm is provided on one side of the inspection conveyor belt, and a lifting platform is provided on the lower side of the robotic arm. A transfer plate is provided at one end of the robotic arm, and an ultra-depth-of-field camera is fixed on the transfer plate. The ultra-depth-of-field camera includes a CCD industrial camera, a long-focal-length electric zoom lens, and a ring light source.
[0007] A sorting conveyor is provided on one side of the detection conveyor belt, and a sorting assembly is provided on one side of the gantry. The sorting assembly includes a slide rail and sorting grippers. Both the detection conveyor belt and the sorting conveyor belt are located under the slide rail. The sorting grippers include a telescopic cylinder, a gripping controller, and gripping fingers.
[0008] Preferably, the detection conveyor belt is parallel to the sorting conveyor belt and the ends are staggered. The detection conveyor belt is provided in two sets, which are respectively set on both sides of the robotic arm.
[0009] Preferably, the slide rail is provided with a sliding seat, and a driving device is provided between the sliding seat and the slide rail; the telescopic cylinder is fixedly connected to the sliding seat, and the output shaft of the telescopic cylinder is fixedly connected to the clamping controller.
[0010] Preferably, at least two sets of clamping fingers are provided and symmetrically arranged on the lower side of the clamping controller, and the clamping fingers are made of rubber.
[0011] Preferably, the ultra-depth-of-field camera is connected to an industrial computer via a data cable, and the visual positioning camera, robotic arm, and sorting components are all connected to the industrial computer via data cables.
[0012] Preferably, the gantry is provided in two sets, and the two sets of gantry and the two sets of detection conveyor belts are distributed in a grid pattern, with the robotic arm located between the two sets of gantry.
[0013] The beneficial effects of this utility model are as follows:
[0014] Compared with existing technologies, this utility model combines a robotic arm and a super depth-of-field camera. The position of the part to be inspected is determined by the visual positioning camera, and the robotic arm controls the movement of the super depth-of-field camera to take pictures. It can be applied to online visual inspection of multiple production lines, maximizing the role of the super depth-of-field camera, reducing equipment investment costs, and improving inspection accuracy and efficiency. Attached Figure Description
[0015] Figure 1 This is a first-view three-dimensional structural diagram of an online visual inspection device based on super-depth-of-field image fusion proposed in this utility model.
[0016] Figure 2 This is a schematic diagram of the second perspective three-dimensional structure of an online visual inspection device based on super-depth-of-field image fusion proposed in this utility model.
[0017] Figure 3 for Figure 2 A magnified schematic diagram of the partial structure at point A in the middle;
[0018] Figure 4 This is a three-dimensional structural schematic diagram of Embodiment 3 of an online visual inspection device based on ultra-depth image fusion proposed in this utility model;
[0019] Figure 5 This is a schematic diagram of the main view structure of Embodiment 3 of the online visual inspection device based on super-depth image fusion proposed in this utility model;
[0020] Figure 6 This is a three-dimensional structural diagram of the robotic arm and ultra-depth-of-field camera of an online visual inspection device based on ultra-depth-of-field image fusion proposed in this utility model.
[0021] In the diagram: 1. Gantry; 2. Vision positioning camera; 3. Slide rail; 4. Robotic arm; 41. CCD industrial camera; 42. Long-focal-length electric zoom lens; 43. Ring light source; 44. Adapter plate; 5. Inspection conveyor belt; 6. Lifting platform; 7. Sorting conveyor belt; 8. Sorting gripper; 81. Sliding seat; 82. Telescopic cylinder; 83. Clamping controller; 84. Grip finger. Detailed Implementation
[0022] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present utility model or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of the present utility model without creative effort are within the scope of protection of the present utility model.
[0023] Reference Figure 1-6 An online visual inspection device based on ultra-depth-of-field image fusion includes a gantry 1 and an inspection conveyor belt 5. A visual positioning camera 2 is provided on the lower side of the gantry 1, and the visual positioning camera 2 is correspondingly set with the inspection conveyor belt 5. A robotic arm 4 is provided on one side of the inspection conveyor belt 5, and a lifting platform 6 is provided on the lower side of the robotic arm 4. The lifting platform 6 is used to adjust the height of the robotic arm 4 to adapt to different production line conveyor belts. An adapter plate 44 is provided at the end of the robotic arm 4, and an ultra-depth-of-field camera is fixed on the adapter plate 44. The ultra-depth-of-field camera includes a CCD industrial camera 41, a long-focal electric zoom lens 42, and a ring light source 43. The ultra-depth-of-field camera, the visual positioning camera 2, and the robotic arm 4 are all connected to an industrial computer via data cables.
[0024] The objects to be inspected are generally thin-film integrated circuits or MEMS devices, which are placed on the inspection conveyor belt 5. The positioning accuracy of the inspection conveyor belt 5 is not as good as that of a dedicated inspection stage, so YOLO vision positioning is required to determine the position of the object to be inspected. Then, the robotic arm 4 controls the ultra-depth camera to approach and take pictures. The telephoto motorized zoom lens 42 focuses on different inspection positions on the upper, middle and lower parts of the circuit board to take clear pictures of the local parts. Through computer deep learning or other image fusion technology, the clear parts are fused into a single image for subsequent visual inspection.
[0025] A sorting conveyor belt 7 is provided on one side of the inspection conveyor belt 5, and a sorting component is provided on one side of the gantry 1. The sorting component is connected to an industrial computer via a data cable. The sorting component includes a slide rail 3 and a sorting gripper 8. The slide rail 3 is fixedly installed on one side of the gantry 1. Both the inspection conveyor belt 5 and the sorting conveyor belt 7 are located under the slide rail 3. The sorting gripper 8 slides on the slide rail 3, grips the circuit board, and moves it back and forth between the inspection conveyor belt 5 and the sorting conveyor belt 7 to sort the circuit boards to be inspected and the circuit boards that have been inspected. This facilitates the transport of circuit boards by the inspection conveyor belt 5 and the sorting conveyor belt 7, and realizes automated inspection.
[0026] As needed, multiple sets of sorting conveyor belts 7 can be set up. After inspection, the sorting grippers 8 will separate qualified products from defective products and place them on the corresponding sorting conveyor belts 7 to be transported to different workstations for convenient processing in subsequent processes.
[0027] The sorting gripper 8 includes a telescopic cylinder 82, a gripping controller 83, and gripping fingers 84. A sliding seat 81 is provided on the slide rail 3, and a driving device is provided between the sliding seat 81 and the slide rail 3. The driving device controls the sliding seat 81 to slide on the slide rail 3. The driving device is generally a lead screw structure or a gear and rack structure, driven by a motor to achieve smooth movement of the sliding seat 81. The outer shell of the telescopic cylinder 82 is fixedly connected to the sliding seat 81, and the output shaft of the telescopic cylinder 82 is fixedly connected to the gripping controller 83, driving the gripping controller 83 to move up and down to adjust the height. At least two sets of gripping fingers 84 are provided and symmetrically arranged on the lower side of the gripping controller 83. The gripping fingers 84 are made of rubber to ensure stable and reliable gripping of the workpiece to be inspected, while avoiding damage to the workpiece to be inspected.
[0028] Example 1: One set of detection conveyor belt 5 and one set of sorting conveyor belt 7 are set. The detection conveyor belt 5 and the sorting conveyor belt 7 are parallel and staggered at their ends. The slide rail 3 is set above the overlapping part of the detection conveyor belt 5 and the sorting conveyor belt 7 to ensure that the items to be detected can be clamped and transported.
[0029] Example 2, as follows Figure 1 , Figure 2 As shown, there are two sets of inspection conveyor belts 5, which are respectively set on both sides of the robotic arm 4. The sorting conveyor belt 7 is set between the two sets of inspection conveyor belts 5, in a triangular arrangement. With two sets of inspection conveyor belts 5 and two inspection production lines, the robotic arm 4 controls the ultra-depth-of-field camera to switch back and forth for shooting, reducing equipment investment costs and improving the convenience of inspection operation.
[0030] Example 3, as follows Figure 4 , Figure 5 As shown, there are two sets of gantry 1. The two sets of gantry 1 and the two sets of inspection conveyor belts 5 are distributed in a grid pattern. The robotic arm 4 is located between the two sets of gantry 1, equipped with more inspection stations. The inspection conveyor belts 5 move in two directions to improve inspection efficiency. Multiple production lines are reasonably arranged for online monitoring, maximizing the utilization of the working time of the ultra-depth-of-field camera.
[0031] In this embodiment, the thin-film integrated circuit or MEMS device to be tested is placed on the testing conveyor belt 5. The testing conveyor belt 5 moves until the visual positioning camera 2 determines the position of the device to be tested. The industrial computer controls the robotic arm 4 to aim the ultra-depth camera at the device to be tested according to YOLO visual positioning technology, and takes pictures. The industrial computer then performs image fusion and testing.
[0032] Based on the test results, the sorting gripper 8 separates and picks up qualified and unqualified products, places them on the sorting conveyor belt 7, and transports them separately. Qualified products enter the next process, while unqualified products are repaired or scrapped.
[0033] The above description is only a preferred embodiment of the present utility model, but the protection scope of the present utility model is not limited thereto. Any equivalent substitutions or changes made by those skilled in the art within the technical scope disclosed in the present utility model, based on the technical solution and the inventive concept of the present utility model, should be included within the protection scope of the present utility model.
[0034] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0035] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps described in these embodiments do not limit the scope of this invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.
Claims
1. An online visual inspection apparatus based on super-zoom image fusion, comprising a portal frame (1) and an inspection conveyor belt (5), characterized in that, A visual positioning camera (2) is provided on the lower side of the gantry (1). The visual positioning camera (2) is correspondingly set with the detection conveyor belt (5). A robotic arm (4) is provided on one side of the detection conveyor belt (5). A lifting platform (6) is provided on the lower side of the robotic arm (4). A transition plate (44) is provided at one end of the robotic arm (4). A super depth-of-field camera is fixed on the transition plate (44). The super depth-of-field camera includes a CCD industrial camera (41), a long-focal electric zoom lens (42), and a ring light source (43). The detection conveyor belt (5) is provided with a sorting conveyor belt (7) on one side, and the gantry (1) is provided with a sorting component on one side. The sorting component includes a slide rail (3) and a sorting gripper (8). The detection conveyor belt (5) and the sorting conveyor belt (7) are both located under the slide rail (3). The sorting gripper (8) includes a telescopic cylinder (82), a gripping controller (83), and a gripping finger (84).
2. The online visual inspection apparatus based on super-depth-of-field image fusion according to claim 1, characterized in that, The detection conveyor belt (5) is parallel to the sorting conveyor belt (7) and the ends are staggered. The detection conveyor belt (5) has two sets, which are respectively set on both sides of the robotic arm (4).
3. The online visual inspection apparatus based on super-zoom image fusion according to claim 2, characterized in that, The slide rail (3) is provided with a sliding seat (81), and a driving device is provided between the sliding seat (81) and the slide rail (3); The telescopic cylinder (82) is fixedly connected to the sliding seat (81), and the output shaft of the telescopic cylinder (82) is fixedly connected to the clamping controller (83).
4. The online visual inspection device based on super-depth image fusion according to claim 3, characterized in that, The clamping fingers (84) are provided in at least two sets and are symmetrically arranged on the lower side of the clamping controller (83). The clamping fingers (84) are made of rubber.
5. The online visual inspection apparatus based on super-zoom image fusion according to claim 4, characterized in that, The ultra-depth-of-field camera is connected to an industrial computer via a data cable. The visual positioning camera (2), the robotic arm (4), and the sorting components are all connected to the industrial computer via data cables.
6. The online visual inspection apparatus based on super-zoom image fusion according to claim 5, characterized in that, The gantry (1) is provided in two sets, and the two sets of gantry (1) and the two sets of detection conveyor belts (5) are distributed in a grid pattern. The robotic arm (4) is located between the two sets of gantry (1).