Measuring device and measuring system

The structured light of different colors is generated synchronously by multiple projection components arranged around the surrounding, and combined with the filter array and sensor, the blind spot problem and the long photography time in the prior art are solved, and efficient three-dimensional image acquisition is achieved.

CN222069603UActive Publication Date: 2024-11-26MATRIXTIME ROBOTICS (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202420230780.8
Authority / Receiving Office
CN · China
Patent Type
Utility models(China)
Current Assignee / Owner
Filing Date
2024-01-31
Publication Date
2024-11-26
Estimated Expiration
2034-01-31

AI Technical Summary

Technical Problem

The existing structured light 3D imaging technology has blind spots when using dual structured light, which causes some convex materials to fail to present striped images intact, which is easy to pass through detection and miss inspection; and when using four structured light, the equipment takes a long time to take pictures, reducing the production capacity of the whole machine. .

Method used

Using multiple projection components arranged around the board, the structured light projection light paths of different colors are synchronized, and reflected light signals are captured through filter arrays and sensors, color images are generated, and point cloud reconstruction is carried out through processing devices to achieve complete acquisition of three-dimensional images. .

Benefits of technology

It reduces the time for taking structured light, reduces blind spots, improves equipment efficiency, and realizes complete high-speed image acquisition on the surface of the chip product.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN222069603U_ABST
    Figure CN222069603U_ABST
Patent Text Reader

Abstract

The utility model relates to the technical field of automatic measurement, discloses a three-dimensional measurement method, and particularly relates to a measurement device and a measurement system. Comprising a projection unit and an imaging unit, the projection unit comprises a plurality of projection parts arranged around the imaging unit, the view field of each projection part can cover the surface of the object to be detected, and each projection part is used for generating structured light with different colors and forming a corresponding projection light path. According to the embodiment of the invention, by synchronously generating a plurality of structured light with different colors and forming a plurality of projection light paths, the structured light photographing time is shortened, the efficiency of the whole equipment is improved, and dead angles of the structured light during imaging of part of materials are reduced. Complete high-speed image acquisition of the surface of a chip product is realized on the whole.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automatic measurement technology, in particular to a three-dimensional measurement method, and in particular to a measurement device and a measurement system. Background Art

[0002] Structured light 3D imaging technology has the following advantages: High precision: Structured light 3D imaging technology can achieve sub-millimeter level measurement accuracy, which can meet the needs of high-precision measurement. High speed: Structured light 3D imaging technology can quickly obtain three-dimensional information of the surface of an object, and can complete the scanning and measurement of large-area objects in a short time. Non-contact: Structured light 3D imaging technology is a non-contact measurement method that can avoid the risk of damage or contamination of the surface of the object. Wide range of applications: Structured light 3D imaging technology is suitable for objects of different shapes, sizes and materials, including metals, plastics, ceramics, glass and other materials.

[0003] However, the structured light 3D imaging in the prior art also has disadvantages, and the disadvantages are explained separately according to the structured light imaging arrangement method. The structured light imaging arrangement methods mainly include dual structured light and quad structured light. When dual structured light is used, some raised materials will have blind spots, and the stripe image cannot be presented intact, and there will inevitably be blind spots with point clouds. For product measurement, it is easier to pass inspection and miss inspection. When quad structured light is used, although the blind spots of the point cloud can be removed, the stripe device projection device requires a certain amount of time. When four projection devices are used, the shooting time of the equipment will be greatly increased, reducing the production capacity of the entire equipment. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a measuring device and a measuring system, which can provide high-speed structured light without blind spots to achieve 3D imaging.

[0005] In the first aspect, an embodiment of the present invention provides a measuring device, comprising: a projection unit, used to synchronously generate multiple projection light paths and project them onto the surface of an object to be detected, and reflect the multiple projection light paths through the surface of the object to be detected to form multiple reflection light paths; an imaging unit, used to receive the multiple reflection light paths, and capture the light signal corresponding to each reflection light path and generate an image; the projection unit comprises multiple projection components arranged around the imaging unit, and the field of view of each projection component can cover the surface of the object to be detected, and each projection component is used to generate structured light of different colors and form corresponding projection light paths.

[0006] In some specific implementations, the plurality of projection components have the same circumferential length.

[0007] In some specific implementations, a plurality of the projection components are used to generate stripe structured lights of different colors.

[0008] In some specific implementations, the plurality of projection components include a first projection component that generates red structured light, a second projection component that generates green structured light, a third projection component that generates blue structured light, and a fourth projection component that generates white structured light.

[0009] In some specific implementations, the imaging unit includes a filter array and a sensor, and the reflected light enters the sensor through the filter array.

[0010] In some specific implementations, the filter array is a mosaic color filter array, which is arranged with red filters, green filters and blue filters; the red filter, the green filter and the blue filter respectively receive red light, green light and blue light in the reflected light path.

[0011] In some specific implementations, the sensor is used to convert the red light, green light, blue light and white light into corresponding multiple electrical signals.

[0012] In some specific implementations, the accuracy of the projection component is less than one quarter of the accuracy of the imaging component.

[0013] In a second aspect, a measuring system is provided, comprising the measuring device described in any one of the above items, and also comprising a controller connected to the measuring device, wherein the controller is used to control the first projection component, the second projection component and the third projection component to start synchronous projection.

[0014] In some specific implementations, the measurement system further includes a processing device, and the processing device is used to perform point cloud reconstruction on the electrical signals corresponding to a plurality of different colors to obtain a reconstructed three-dimensional image.

[0015] In some specific implementations, the processing device includes a synthesis unit and a reconstruction unit; the synthesis unit is used to interpolate and process the multiple electrical signals to obtain the color value of the current pixel, and determine the estimated value of the missing color channel to synthesize a color image; the reconstruction unit is used to extract channels of the color image based on the RGB three channels, respectively extract the black and white images corresponding to the three channels, and reconstruct the point clouds of the three black and white images respectively, and merge the reconstructed point clouds.

[0016] The embodiments of the present invention bring the following beneficial effects: by synchronously generating multiple structured lights with different colors to form multiple projection light paths, the structured light shooting time is reduced, the efficiency of the whole device is improved, and the blind spots when the structured light is used to image some materials are reduced. Overall, complete high-speed image acquisition of the surface of chip products is achieved.

[0017] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0018] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of the structure of a measuring device provided in an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the architecture of a measurement system provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the architecture of a processing device provided in an embodiment of the present invention.

[0023] Icons: 100 - measuring device; 110 - projection unit; 120 - imaging unit; 111 - first projection component; 112 - second projection component; 113 - third projection component; 114 - fourth projection component;

[0024] 200 - measurement system; 210 - controller; 220 - processing device; 221 - synthesis unit; 222 - reconstruction unit. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The detection light source and imaging device provided in the embodiment of the present application are applied to chip inspection and measurement, wherein the chip is a chip product after printing and packaging, and the three-dimensional shape contour of the chip product is obtained by structured light projection. Among them, the structured light three-dimensional vision is based on the principle of optical triangulation. Specifically, an optical projector is set to transmit a certain pattern of structured light to the surface of the object, and a three-dimensional image of a light strip modulated by the surface shape of the object to be measured is formed on the surface. The three-dimensional image is detected by a camera at another position, thereby obtaining a two-dimensional distorted image of the light strip. The degree of distortion of the light strip depends on the relative position between the optical projector and the camera and the shape contour (height) of the object surface. Intuitively, the displacement (or offset) displayed along the light strip is proportional to the height of the object surface, the kink represents the change of the plane, and the discontinuity shows the physical gap of the surface. When the relative position between the optical projector and the camera is constant, the three-dimensional shape contour of the object surface can be reproduced by the distorted two-dimensional light strip image coordinates. The structured light three-dimensional vision system is composed of an optical projector, a camera, and a computer system.

[0027] According to the different light beam patterns projected by the optical projector, the structured light pattern can be divided into point structured light pattern, line structured light pattern, multi-line structured light pattern, surface structured light pattern, phase method, etc.

[0028] Surface structured light mode: When surface structured light is used, a two-dimensional structured light pattern is projected onto the surface of the object, so that three-dimensional contour measurement can be achieved without scanning, and the measurement speed is very fast. The most commonly used method in surface structured light is to project grating stripes onto the surface of the object. When the projected structured light pattern is more complex, in order to determine the correspondence between the surface points of the object and its image pixels, the projected pattern needs to be encoded, so this type of method is also called coded structured light measurement method. Pattern coding is divided into spatial domain coding and time domain coding. The spatial domain coding method only needs one projection to obtain the depth map of the object, which is suitable for dynamic measurement, but the current resolution and processing speed cannot meet the requirements of real-time three-dimensional measurement, and the decoding requirements are very high. Time domain coding requires multiple different projected coding patterns to be combined and decoded, which is easier to achieve decoding. The main coding methods are binary coding, two-dimensional grid pattern coding, random pattern coding, color coding, grayscale coding, neighborhood coding, phase coding and hybrid coding.

[0029] Phase method: In recent years, the phase-based grating projection three-dimensional profile measurement technology has made great progress. The grating pattern is projected onto the surface of the object to be measured. The grating stripes are deformed by the modulation of the object height. The deformed stripes can be interpreted as spatial carrier signals modulated in both phase and amplitude. Collecting the deformed stripes and demodulating them can obtain the phase change containing height information, and finally the height is calculated according to the principle of triangulation. This type of method is also called the phase method. The theoretical basis of the three-dimensional profile measurement technology based on phase measurement is also optical triangulation, but it is different from the profilometry of optical triangulation. It does not directly find and judge the image points after the height change of the object, but indirectly achieves it through phase measurement. Due to the involvement of phase information, this type of method is very different from the pure optical triangulation method.

[0030] In the prior art, there are two main arrangements for the projection generating components when imaging with structured light solutions. The first is dual structured light, that is, placing two projection generating components, and the second is quad structured light, that is, placing four projection generating components. When using dual structured light, some raised materials will have blind spots, and the stripe image cannot be presented intact, and there will inevitably be blind spots with point clouds. For product measurement, it is easier to pass inspection and miss inspection. When using quad structured light, although the blind spots of the point cloud can be removed, the projection generating components require a certain amount of time. When using four projection generating components, the shooting time of the equipment will be greatly improved, reducing the uph (production capacity) of the entire equipment.

[0031] Based on this background technology, an embodiment of the present application provides a high-speed structured light 3D measurement solution without blind spots, specifically a measurement device that can achieve complete high-speed image acquisition of the surface of a chip product.

[0032] See also Figure 1 , the measuring device provided in the embodiment of the present application specifically includes a projection unit and an imaging unit.

[0033] The projection unit is used to synchronously generate a projection light path and project it onto the surface of the object to be detected, and reflect the projection light path through the surface of the object to be detected to form a reflection light path.

[0034] The imaging unit is used to receive the reflected light path, capture the light signal in the reflected light path and generate an image.

[0035] Specifically, the projection unit in the embodiment of the present application includes a plurality of projection components arranged around the imaging unit, wherein the projection field of each projection component can cover the surface of the object to be detected, that is, the surface of the chip, and each projection component is used to generate structured light of different colors. The color of the structured light generated by each projection component is monochromatic light.

[0036] Among them, in the embodiment of the present application, the number of projection components in the projection unit is four, that is, the projection unit of the measurement device is a four-structured light structure, generating four structured lights that can be received by the chip surface. That is, the projection unit includes a first projection component, a second projection component, a third projection component and a fourth projection component, wherein the above projection components are used to generate four monochromatic structured lights of different colors.

[0037] Specifically, the structured light colors of the above multiple projection components can be further explained in the embodiments of the present application. The first projection component can generate red structured light, the second projection component can generate green structured light, the third projection component can generate blue structured light, and the fourth projection component can generate white structured light.

[0038] In the embodiment of the present application, the first projection component, the second projection component, the third projection component and the fourth projection component are arranged around the imaging device, and a ring arrangement is formed between the projection components.

[0039] And in the embodiment of the present application, in order to achieve uniform structured light projection by each projection component, the first projection component, the second projection component, the third projection component and the fourth projection component have the same circumferential length, that is, the above four projection components are arranged evenly. Among them, in order to ensure the standardization of the projection pattern position of the projected structured light on the chip surface, the above four projection components are respectively placed at the twelve o'clock direction, the three o'clock direction, the six o'clock direction and the nine o'clock direction of the imaging device, that is, the above four projection components are respectively placed at 0°, 90°, 180° and 270° relative to the imaging device. Specifically, the projection components placed at the 0° position, the 90° position and the 180° position are the first projection component, the second projection component and the third projection component, and the fourth projection component is placed at the 270° position.

[0040] Among them, the projection patterns generated by the first projection component, the second projection component, the third projection component and the fourth projection component are stripe patterns, that is, the first projection component, the second projection component, the third projection component and the fourth projection component respectively perform red stripe projection, green stripe projection, blue stripe projection and white stripe projection, and the projection pattern structures of the above projection components are the same and the only difference is the color. In other embodiments, other existing patterns can also be selected for the projection pattern of the projection component, but because of the convenience of the stripe pattern in obtaining chip morphology information, this embodiment gives priority to the projection component of the stripe pattern, and gives priority to the DLP optical machine for implementation, and other implementation methods are no longer described in the embodiments of this application.

[0041] In the embodiment of the present application, in order to realize the stripe pattern corresponding to multiple colors reflected from the chip surface, the imaging device is provided with a filter array on the basis of the necessary sensor, and the reflected light of the multi-color stripe pattern reflected from the chip surface enters the sensor after passing through the filter array to realize imaging. Among them, for the color camera principle based on the color perception mechanism of the human eye, by setting the combination of the filter array and the sensor, light signals of different wavelengths can be captured, and the final color pattern of the straight line can be synthesized.

[0042] Among them, the filter array is a mosaic color filter array (Bayer), in which red filters, green filters and blue filters are arranged, and are arranged at each pixel position of the sensor based on the arrangement of the mosaic color green array. When the light passes through the filter array, each pixel can only receive light signals of a specific wavelength, that is, the pixel of the red filter can only receive red light, the pixel of the green filter can only receive green light, and the pixel of the blue filter can only receive blue light. The sensor converts the received red light, green light, blue light and white light into corresponding multiple electrical signals, and processes and amplifies them through the circuit set in the sensor to finally complete the imaging.

[0043] In order to avoid the problem of poor accuracy of the sensor in extracting a single image, in the embodiment of the present application, the accuracy of the projection component is set to be less than one quarter of the accuracy of the imaging component.

[0044] In the embodiment of the present application, the measuring device performs synchronous projection through the first projection component, the second projection component, and the third projection component to obtain a projection pattern containing three colors, and receives the corresponding color light signals through the filter array, and finally converts the light signals corresponding to different colors into multiple electrical signals through the sensor to achieve the final imaging. The whole process only requires one projection, which solves the time cost and production capacity cost problems caused by multiple projections of color images in the prior art, and improves the overall imaging efficiency.

[0045] On the other hand, an embodiment of the present application further provides a measurement system, including the above-mentioned measurement device and a controller and a processing device electrically connected to the measurement device.

[0046] The controller is electrically connected to the first projection component, the second projection component and the third projection component, and is used to simultaneously issue opening and closing commands to the first projection component, the second projection component and the third projection component, so that the first projection component, the second projection component and the third projection component can start and close the projection synchronously, thereby realizing the synchronization of the above three projection components.

[0047] The processing device is used to reconstruct the point cloud of the multiple electrical signals converted by the sensor to obtain a reconstructed three-dimensional image.

[0048] Specifically, the processing device includes a synthesis unit and a reconstruction unit.

[0049] The synthesis unit is used to interpolate the multiple electrical signals to obtain the color value of the current pixel, and determine the estimated value of the missing color channel to synthesize a color image. In the embodiment of the present application, the interpolation process can be implemented using the solution in the prior art, which will not be repeated in this process.

[0050] The reconstruction unit is used to extract channels of the color image based on the RGB three channels, extract black and white images corresponding to the three channels respectively, reconstruct point clouds of the three black and white images respectively, and merge the reconstructed point clouds. Similarly, the image processing method in the prior art can be used for the channel extraction process and the point cloud reconstruction process, which will not be described in detail in this process.

[0051] In summary, the measurement system provided in the embodiment of the present application can realize the synchronous projection opening and closing of the first projection component, the second projection component and the third projection component by setting a controller, thereby achieving the technical purpose of one projection, solving the problems of time cost and production capacity cost caused by multiple projections of color images in the prior art, and improving the overall imaging efficiency.

[0052] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A measuring device, characterized in that: include: A projection unit, used for synchronously generating a plurality of projection light paths and projecting them onto the surface of the object to be detected, and reflecting the plurality of projection light paths from the surface of the object to be detected to form a plurality of reflection light paths; An imaging unit, used to receive the plurality of reflected light paths, capture the light signal corresponding to each of the reflected light paths and generate an image; The projection unit includes a plurality of projection components arranged around the imaging unit, and the field of view of each projection component can cover the surface of the object to be detected, and each projection component is used to generate structured light of different colors and form a corresponding projection light path.

2. The measuring device according to claim 1, characterized in that: The plurality of projection components have the same circumferential length.

3. The measuring device according to claim 2, characterized in that: The plurality of projection components are used to generate stripe structured lights of different colors.

4. The measuring device according to claim 2 or 3, characterized in that: The plurality of projection components include a first projection component that generates red structured light, a second projection component that generates green structured light, a third projection component that generates blue structured light, and a fourth projection component that generates white structured light.

5. The measuring device according to claim 4, characterized in that: The imaging unit includes a filter array and a sensor, and the reflected light enters the sensor via the filter array.

6. The measuring device according to claim 5, characterized in that: The filter array is a mosaic color filter array, which is arranged with a red filter, a green filter and a blue filter; the red filter, the green filter and the blue filter receive red light, green light and blue light in the reflected light path respectively.

7. The measuring device according to claim 6, characterized in that: The sensor is used for converting the red light, green light, blue light and white light into corresponding multiple electrical signals.

8. The measuring device according to claim 1, characterized in that: The accuracy of the projection component is less than one quarter of the accuracy of the imaging unit.

9. A measurement system, characterized in that: It comprises the measuring device according to any one of claims 1 to 8, and also comprises a controller connected to the measuring device, wherein the controller is used to control the first projection component, the second projection component and the third projection component to start synchronous projection.

10. The measuring system according to claim 9, characterized in that: The measurement system further includes a processing device, which is used to perform point cloud reconstruction on the electrical signals corresponding to a plurality of different colors to obtain a reconstructed three-dimensional image.

11. The measuring system according to claim 10, characterized in that: The processing device includes a synthesis unit and a reconstruction unit; The synthesis unit is used to interpolate and process the plurality of electrical signals to obtain the color value of the current pixel, and determine the estimated value of the missing color channel to synthesize a color image; The reconstruction unit is used to perform channel extraction on the color image based on the RGB three channels, respectively extract black and white images corresponding to the three channels, respectively perform point cloud reconstruction on the three black and white images, and merge the reconstructed point clouds.