Three-dimensional imaging method based on image stacking and application thereof
Through the three-dimensional imaging method based on image stacking, clear areas on different heights of focal planes are extracted and images are stacked, which solves the problems of imaging accuracy and detection speed in the prior art, and realizes the construction of a three-dimensional image model with high precision and rapid detection.
Patent Information
- Application Number
- CN202410943701.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-05
AI Technical Summary
The existing three-dimensional imaging technology is difficult to have both imaging accuracy and detection speed when detecting complex packaging fields, especially when it is impossible to focus clearly on objects to be detected with height difference.
Using a three-dimensional imaging method based on image stacking, by acquiring the images that are focused and reflected on the focal plane of the measured object at different heights, extracting clear areas and stacking images, generating a target three-dimensional image model, combining the lighting module, optical imaging module and image acquisition Module, using digital micromirror array to achieve real-time adjustment of focal plane height.
While achieving high imaging accuracy, it speeds up detection speed, adapts to different target surfaces and accuracy requirements, and generates an accurate three-dimensional image model.
Smart Images

Figure CN118898685A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of display screen brightness and chromaticity detection, and in particular to a three-dimensional imaging method based on image stacking and a three-dimensional imaging device based on image stacking. Background Art
[0002] With the development of semiconductor technology, wafers in the packaging field are becoming more and more complex, from the initial plane to the highly integrated wafer-level WLP, 2.5D / three-dimensional, system-level packaging technology and high-performance FlipChip and lead interconnect packaging technology. Therefore, in computer vision detection, three-dimensional imaging technology is an important research direction, and complex structures need to be presented through three-dimensional imaging technology for detection and measurement.
[0003] Due to the presence of grooves, RDL lines, bumps and other complex structures on the surface of complex wafers in the field of advanced packaging, conventional 2D detection methods have to meet accuracy requirements, and the measurement depth range is very small, and it is impossible to clearly focus on such objects with height differences. In addition, existing 3D imaging solutions such as binocular stereo vision, laser triangulation, and structured light 3D imaging have the problem of difficulty in achieving both imaging accuracy and detection speed. Summary of the invention
[0004] In order to overcome the defects of the above-mentioned prior art, an embodiment of the present invention provides a three-dimensional imaging method based on image stacking and its application, which can achieve high imaging accuracy while having a faster detection speed in the wafer three-dimensional imaging detection process.
[0005] Specifically, the present invention provides a three-dimensional imaging method based on image stacking, including: obtaining a first image formed after illumination light is focused and reflected on a first height focal plane of a measured object; adjusting the focusing height of the illumination light, obtaining a second image formed after illumination light is focused and reflected on a second height focal plane of the measured object that is different from the first height; extracting clear areas on the first image and the second image, and stacking the clear areas according to the imaging heights of the measured object corresponding to the first image and the second image to obtain a target three-dimensional image.
[0006] In one embodiment of the present invention, extracting clear areas on the first image and the second image includes: acquiring pixel features on the first image and the second image; and setting a preset threshold according to the pixel features to select pixel points that meet the clear area.
[0007] In one embodiment of the present invention, before stacking the images of the clear area, it also includes: selecting any one of the first image or the second image to calibrate a reference point, calibrating each other image according to the reference point, and obtaining the translation coordinate value and pixel scaling value of each other image relative to the reference point.
[0008] In one embodiment of the present invention, the illumination light illuminates the object to be measured from at least two different directions simultaneously to obtain the corresponding first image and second image.
[0009] In one embodiment of the present invention, the focal plane at the second height includes a plurality of focal planes at different heights, and the illumination light is focused on the focal plane at the second height to obtain a corresponding plurality of second images; and clear areas on the first image and each of the second images are extracted for image stacking to obtain the target three-dimensional image.
[0010] In addition, an embodiment of the present invention also proposes a three-dimensional imaging system based on image stacking, comprising: an illumination module for illuminating the object to be measured; an optical imaging module for acquiring light reflected by the object to be measured for imaging; and an image acquisition module for acquiring the image formed by the optical imaging module; wherein the three-dimensional imaging system based on image stacking is used to implement the three-dimensional imaging method based on image stacking described in any one of the above embodiments.
[0011] In one embodiment of the present invention, the three-dimensional imaging system based on image stacking also includes: an imaging height adjustment module, which is used to pre-adjust the relative position of the optical imaging module and the object to be measured, so that the focal plane of the target height on the object to be measured is clearly imaged in the image acquisition module.
[0012] In one embodiment of the present invention, the three-dimensional imaging system based on image stacking also includes: a light adjustment module, which is arranged between the optical imaging module and the image acquisition module, and is used to adjust in real time the focal plane height of the optical imaging module for clearly imaging the object under test in the image acquisition module.
[0013] In one embodiment of the present invention, the light adjustment module is a digital micromirror array, which includes a plurality of small reflectors and micromotors connected to the small reflectors, and the micromotors control the movement and rotation of the small reflectors to achieve focusing.
[0014] Furthermore, an embodiment of the present invention also proposes a three-dimensional imaging device based on image stacking, including: a first image acquisition module, used to acquire a first image formed after the illumination light is focused and reflected on a focal plane at a first height of the object to be measured; a second image acquisition module, used to acquire a second image formed after the illumination light is focused and reflected on a focal plane at a second height of the object to be measured that is different from the first height; a three-dimensional image acquisition module, used to extract clear areas on the first image and the second image, and stack the clear areas according to the imaging heights of the object to be measured corresponding to the first image and the second image to obtain a target three-dimensional image.
[0015] Furthermore, an embodiment of the present invention also proposes an electronic device, comprising: a memory and one or more processors connected to the memory, the memory storing a computer program, and the processor being used to execute the computer program to implement the three-dimensional imaging method based on image stacking as described in any one of the above embodiments.
[0016] Still another embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the three-dimensional imaging method based on image stacking as described in any one of the above embodiments.
[0017] It can be seen from the above that the above embodiments of the present invention can have the following beneficial effects:
[0018] By acquiring images formed by focusing and reflecting on focal planes at different heights on the object under test, the clear parts of the images at different focus positions are extracted, and stacked according to the corresponding heights of the object to generate a three-dimensional image model, thereby realizing the construction of a three-dimensional image model based on two-dimensional imaging. Compared with existing two-dimensional or three-dimensional imaging solutions, it can adapt to different target surface requirements and accuracy requirements, and has both higher imaging accuracy and faster detection speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0020] Figure 1 A flowchart of a three-dimensional imaging method based on image stacking provided by an embodiment of the present invention;
[0021] Figure 2 A schematic structural diagram of a three-dimensional imaging system based on image stacking provided by an embodiment of the present invention;
[0022] Figure 3 A schematic diagram of the image stacking principle provided by an embodiment of the present invention;
[0023] Figure 4 A schematic structural diagram of a three-dimensional imaging device based on image stacking provided by an embodiment of the present invention;
[0024] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;
[0025] Figure 6 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0026] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described below with reference to the accompanying drawings and in combination with the embodiments.
[0027] In order to enable ordinary technicians in the field to better understand the technical solution of the present invention, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all should belong to the protection scope of the present invention.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are applicable to distinguishing similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition. The terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or are proprietary to these processes, methods, products or devices.
[0029] It should also be noted that the division of multiple embodiments in the present invention is only for the convenience of description and should not constitute a special limitation. The features in various embodiments can be combined and referenced to each other without contradiction.
[0030] like Figure 1As shown, the first embodiment of the present invention proposes a three-dimensional imaging method based on image stacking, for example, including: step S1, obtaining a first image formed after the illumination light is focused and reflected on a focal plane at a first height of the object to be measured; step S2, adjusting the focusing height of the illumination light, and obtaining a second image formed after the illumination light is focused and reflected on a focal plane at a second height of the object to be measured that is different from the first height; step S3, extracting clear areas on the first image and the second image, and stacking the clear areas according to the imaging heights of the object to be measured corresponding to the first image and the second image to obtain a target three-dimensional image.
[0031] In this embodiment, combined with Figure 2 As shown, for example, the object to be measured is illuminated by the lighting module, and the optical imaging module focuses the illumination light on the focal plane of the first height of the object to be measured and the reflected light is clearly imaged in the image acquisition module. Of course, the height of the focal plane where the light is clearly imaged in the image acquisition module can be set by adjusting the optical imaging module, and the present application is not limited to this first height.
[0032] In one embodiment, the focal plane at the second height includes, for example, several focal planes at different heights, and after the illumination light is focused on the focal plane at the second height, several corresponding second images are obtained; the clear areas on the first image and each of the second images are extracted for image stacking to obtain the target three-dimensional image. In this way, multiple images obtained by collecting clear images of focal planes at different heights of the measured object can be stacked, and the number of images can be unlimited to accurately obtain the required target three-dimensional image.
[0033] In one embodiment, for example, by acquiring pixel features such as pixel size, brightness and chromaticity values on the first image and the second image, a preset threshold is set according to these pixel features to select pixel points that meet the clear area requirements, thereby extracting clear areas on the first image and the second image.
[0034] In one embodiment, if Figure 3 As shown, image stacking fitting is to extract the clear areas of images at different focus positions, and then stack them according to the height corresponding to the focal plane on the object being measured when the light is focused. During stacking, the pixel size of each focus image changes. For example, any first image or second image is selected to calibrate the reference point, and each other image is calibrated according to the reference point to obtain the shiftx, shifty translation value and pixelsize value of each other image relative to the horizontal plane of the reference point, and then the images are stacked after calibration.
[0035] In one embodiment, for example, multiple light sources are used for illumination, and the illumination light illuminates the object to be measured from at least two different directions or angles simultaneously, so that the morphological details of the three-dimensional object to be measured can be clearly photographed.
[0036] In summary, the three-dimensional imaging method based on image stacking proposed in the first embodiment of the present invention obtains the image formed by focusing and reflecting on focal planes at different heights on the object to be measured, extracts the clear parts of the image at different focus positions, and stacks them according to the height corresponding to the object to generate a three-dimensional image model, thereby realizing the construction of a three-dimensional image model based on two-dimensional imaging. Compared with the existing two-dimensional or three-dimensional imaging solutions, it can adapt to different target surface requirements and accuracy requirements, and has both higher imaging accuracy and faster detection speed.
[0037] In addition, if Figure 2 As shown, the second embodiment of the present invention further proposes a three-dimensional imaging system based on image stacking, for example, including: an illumination module, an optical imaging module and an image acquisition module. Among them, the illumination module is used to illuminate the object to be measured; the optical imaging module is used to obtain the light reflected by the object to be measured for imaging; the image acquisition module is used to acquire the image formed by the optical imaging module.
[0038] In one embodiment, the three-dimensional imaging system based on image stacking, for example, also includes: an imaging height adjustment module, which is connected to the optical imaging module and is used to pre-adjust the relative position of the optical imaging module and the object to be measured so that the focal plane of the target height on the object to be measured is clearly imaged in the image acquisition module.
[0039] In one embodiment, the three-dimensional imaging system based on image stacking, for example, also includes: a light adjustment module, which is arranged between the optical imaging module and the image acquisition module, and is used to adjust in real time the focal plane height of the optical imaging module for clearly imaging the object under test in the image acquisition module.
[0040] Specifically, for example, the distance between the object to be measured and the optical imaging module is first adjusted to a suitable position through the imaging height adjustment module and then fixed. This position, for example, is a point at which half the thickness of the object to be measured can be clearly imaged on the image acquisition module, and at this time, the adjustment interval of the light adjustment module is in the middle of the range. Then, the object to be measured is illuminated by the lighting module, and the reflected light is received and imaged by the image acquisition module. Furthermore, the focus position is changed through the light adjustment module, and the image acquisition module will take images each time the focus position is changed, and then the images taken are stacked and fitted to generate a three-dimensional image model.
[0041] In one embodiment, the light adjustment module is, for example, a digital micromirror array, which includes a plurality of small reflectors and a micro-motor connected to the small reflectors, and the micro-motor controls the movement and rotation of the small reflectors to achieve focusing. The microlens array group can split an entire light beam into many small light spots, and changing the angle and optical path of each small light spot can achieve fine-tuning of the focal length to achieve focusing.
[0042] It is worth mentioning that the three-dimensional imaging method based on image stacking implemented by the three-dimensional imaging system based on image stacking proposed in the second embodiment of the present invention is as described in the first embodiment, so it will not be described in detail here. Optionally, the various modules and the above-mentioned other operations or functions in the second embodiment are respectively for implementing the method described in the first embodiment, and the three-dimensional imaging system based on image stacking provided in this embodiment has the same beneficial effects as the three-dimensional imaging method based on image stacking provided in the first embodiment, and for the sake of brevity, it will not be repeated here.
[0043] like Figure 4 As shown, the third embodiment of the present invention further proposes a three-dimensional imaging device 20 based on image stacking, for example, including: a first image acquisition module 201, a second image acquisition module 202 and a three-dimensional image acquisition module 203.
[0044] The first image acquisition module 201 is used to acquire a first image formed by focusing and reflecting the illumination light on a focal plane at a first height of the object to be measured. The second image acquisition module 202 is used to acquire a second image formed by focusing and reflecting the illumination light on a focal plane at a second height of the object to be measured that is different from the first height. The three-dimensional image acquisition module 203 is used to extract clear areas on the first image and the second image, and stack the clear areas according to the imaging heights of the object to be measured corresponding to the first image and the second image to obtain a target three-dimensional image.
[0045] It is worth mentioning that the three-dimensional imaging method based on image stacking implemented by the three-dimensional imaging device based on image stacking proposed in the third embodiment of the present invention is as described in the first embodiment, so it will not be described in detail here. Optionally, the various modules and the above-mentioned other operations or functions in the third embodiment are respectively for implementing the method described in the first embodiment, and the three-dimensional imaging device based on image stacking provided in this embodiment has the same beneficial effects as the three-dimensional imaging method based on image stacking provided in the first embodiment, and for the sake of brevity, it will not be repeated here.
[0046] like Figure 5As shown, the fourth embodiment of the present invention further proposes an electronic device 30, for example, including: at least one processing unit 31, and at least one storage unit 32, wherein the storage unit 32 stores a computer program, and when the computer program is executed by the processing unit 31, the processing unit 31 executes the method as described in the first embodiment, and the electronic device 30 provided in this embodiment has the same beneficial effects as the method described in the first embodiment.
[0047] The fifth embodiment of the present invention also provides a computer-readable storage medium 40 on which a computer program is stored. When the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium provided in this embodiment has the same beneficial effects as the three-dimensional imaging method based on image stacking provided in the first embodiment.
[0048] Among them, computer-readable storage media may include, but are not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.
[0049] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0050] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0051] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0052] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0053] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0054] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.
[0055] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct the relevant hardware. The program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0056] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.
[0057] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0058] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A three-dimensional imaging method based on image stacking, characterized in that: include: Acquire a first image formed after the illumination light is focused and reflected on a focal plane at a first height of the measured object; Adjusting the focusing height of the illumination light to obtain a second image formed by focusing and reflecting the illumination light on a focal plane at a second height of the object being measured that is different from the first height; The clear areas on the first image and the second image are extracted, and the clear areas are stacked according to the imaging heights of the measured object corresponding to the first image and the second image to obtain a target three-dimensional image.
2. The three-dimensional imaging method based on image stacking according to claim 1, characterized in that: The extracting clear areas on the first image and the second image includes: Acquire pixel features on the first image and the second image; A preset threshold is set according to the pixel characteristics to select pixel points that meet the clear area.
3. The three-dimensional imaging method based on image stacking according to claim 1, characterized in that: Before stacking the images of the clear area, the method further comprises: Select any one of the first image or the second image to calibrate a reference point, calibrate each other image according to the reference point, and obtain the translation coordinate value and pixel scaling value of each other image relative to the reference point.
4. The three-dimensional imaging method based on image stacking according to claim 1, characterized in that: The illumination light illuminates the object to be measured from at least two different directions simultaneously to obtain the corresponding first image and second image.
5. The three-dimensional imaging method based on image stacking according to claim 1, characterized in that: The focal plane at the second height includes a plurality of focal planes at different heights, and the illumination light is focused on the focal plane at the second height to obtain a corresponding plurality of second images; and clear areas on the first image and each of the second images are extracted for image stacking to obtain the target three-dimensional image.
6. A three-dimensional imaging system based on image stacking, characterized in that: include: An illumination module, used for illuminating the object to be measured; An optical imaging module, used to obtain light reflected by the object to be measured for imaging; An image acquisition module, used for acquiring the image formed by the optical imaging module; Wherein, the three-dimensional imaging system based on image stacking is used to implement the three-dimensional imaging method based on image stacking described in any one of claims 1 to 5.
7. The three-dimensional imaging system based on image stacking according to claim 6, characterized in that: Also includes: The imaging height adjustment module is used to pre-adjust the relative position between the optical imaging module and the object to be measured so that the focal plane at the target height on the object to be measured can be clearly imaged in the image acquisition module.
8. The three-dimensional imaging system based on image stacking according to claim 7, characterized in that: Also includes: The light adjustment module is arranged between the optical imaging module and the image acquisition module, and is used for adjusting in real time the focal plane height of the optical imaging module for clearly imaging the measured object in the image acquisition module.
9. The three-dimensional imaging system based on image stacking according to claim 8, characterized in that: The light adjustment module is a digital micromirror array, which includes a plurality of small reflectors and micromotors connected to the small reflectors, and the micromotors control the movement and rotation of the small reflectors to achieve focusing.
10. A three-dimensional imaging device based on image stacking, characterized in that: include: A first image acquisition module, used to acquire a first image formed after the illumination light is focused and reflected on a focal plane at a first height of the measured object; A second image acquisition module, used to acquire a second image formed after the illumination light is focused and reflected on a focal plane at a second height of the measured object that is different from the first height; The three-dimensional image acquisition module is used to extract the clear areas on the first image and the second image, and to stack the clear areas according to the imaging heights of the measured objects corresponding to the first image and the second image to obtain a target three-dimensional image.
Citation Information
Patent Citations
Machine Vision Inspection System and Method for Performing High-Speed Focus Height Measurement Operations
CN104236463A
3D imaging system
CN112422944A
Confocal camera control method and device, and medium
CN117896629A
Determination of depth values of a surface area of a workpiece
EP4227636A1
Automatic focusing by mirror translation
US20070053673A1
