Three-dimensional topography surface capturing device
By designing a portable 3D morphological surface capture device, using photometric stereo imaging technology to capture 3D morphological surface images of large or immovable samples in two operating modes, solving the problem that the prior art is difficult to deal with large or immovable samples. Achieve independent and portable high-quality image capture and metadata generation.
Patent Information
- Application Number
- CN202280094414.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-31
- Publication Date
- 2024-12-06
AI Technical Summary
Existing physical-based rendering (PBR) technologies are used in three-dimensional topographic surface capture devices that are difficult to handle large or immovable samples and usually require communication or conduction connections with the main computing device to function properly.
A portable 3D topographic surface capture device is designed, which can operate in two independent operating modes: one is the placement of the sample in a tray, and the other is the device attached to the mounting to abut the sample, using photometric stereoscopic imaging The technology generates 3D topographic surface images without communicating or conducting connections with other devices. The device includes a removable tray, control circuit, light source and imaging subassembly, capable of capturing images and synthesizing images under different lighting conditions.
3D topographic surface image capture of large or immovable samples is achieved, providing portable and independent capture capabilities, enabling high-quality 3D topographic surface images and metadata without external power supply and communication connections.
Smart Images

Figure CN119096111A_ABST
Abstract
Description
Background Art
[0001] Physically based rendering (PBR) is a method of synthesizing a three-dimensional (3D) topographic surface image of a sample. For example, PBR can employ photometric stereo imaging techniques. In photometric stereo imaging, two-dimensional (2D) images of a sample are captured under different lighting conditions. From these multiple 2D images, a 3D topographic surface image of the sample can be synthesized. In addition, surface reflectance metadata can be synthesized for image rendering of surface reflectance, where the orientation and color of the changing illumination source are correctly accounted for. BRIEF DESCRIPTION OF THE DRAWINGS
[0002] Figure 1A and 1B are diagrams depicting two different example modes of operation of a three-dimensional (3D) topographic surface capture device.
[0003] Figure 2A , 2B and 2C are available in Figure 1A and 1B Perspective, exploded perspective, and cross-sectional views of an example 3D topography surface capture device operating in two different modes.
[0004] Figure 3 Yes, you can Figure 2A , 2B Block diagram of an example circuit employed in a 3D topography surface capture device of 2C.
[0005] Figure 4 is Figure 1A and 1B Use in two different modes Figure 2A , 2B Flowchart of an example method of a 3D topographic surface capture device of 2C.
[0006] Figure 5 is a block diagram of an example 3D topography surface capture device. DETAILED DESCRIPTION
[0007] As mentioned in the background, physically based rendering (PBR) provides synthesis of three-dimensional (3D) topographic surface images of samples, and is achieved via photometric stereo imaging technology. Existing 3D topographic surface capture devices using photometric stereo imaging technology include a chamber in which the sample is placed. Therefore, the lighting conditions experienced by the sample can be strictly controlled, and multiple two-dimensional (2D) images are captured under different lighting conditions, thereby synthesizing the 3D topographic surface image of the sample. These devices are usually under the control of a main computing device.
[0008] While such 3D topography surface capture devices work well enough for samples that are small and portable enough to be placed in a chamber, they are generally not usable for large and / or immovable samples. For example, samples may be so large that they cannot be placed in the chamber of even the largest 3D topography surface capture devices. Furthermore, even small samples that can fit in a chamber may not be movable to be placed within the chamber. For example, priceless artifacts in a museum may not be allowed to be physically manipulated to be placed within the chamber of an existing 3D topography surface capture device.
[0009] The technology described herein relates to a 3D topographic surface capture device that alleviates these and other shortcomings of existing such capture devices. The capture device described is portable and can be operated independently in two modes without requiring a communication or conductive connection with any other device. In the first mode, a sample is placed in a tray, which is then attached to the bottom of the device. The light source and imaging subassemblies inside the device are then controlled to generate a 3D topographic surface image of the sample using photometric stereo imaging techniques.
[0010] In the second mode, the tray is removed from the bottom of the 3D topographic surface capture device. The capture device can be attached to a mount so that the bottom of the device can be placed in such a way as to abut the sample. The light source and imaging subassembly are then controlled as previously described to generate a 3D topographic surface image of the sample using photometric stereo imaging techniques. Then, in this second mode, the sample does not have to be moved inside or to the capture device, and can therefore be large and / or immovable. Instead, the capture device is moved to the location of the sample.
[0011] Figure 1A and 1B FIG. 1 shows an exemplary operation of the 3D topography surface capture device 100 in two different modes. Figure 1A In an example mode of operation, the sample 104 is placed within the tray 102 of the capture device 100, and the tray 102 is then attached to the bottom of the upper portion 101 of the device 100. Thus, the sample 104 is located within an internal cavity or volume of the capture device 100, thereby providing a controlled environment in which photometric stereo imaging techniques can be used to synthesize a 3D topographic surface image of the sample 104 from the captured 2D topographic surface image of the sample 104.
[0012] exist Figure 1BIn an example mode of operation, the tray 102 has been detached and removed from the bottom of the upper portion 101 of the 3D topographic surface capture device 100. A mount 152 (such as a tripod in this example) has been attached to the other end of the upper portion 101 of the capture device 100 and moved or adjusted so that the device 100 can be placed adjacent to and abutting the sample 154. Thus, a portion of the sample 154 is effectively located within the internal cavity or volume of the capture device 100, and since the sample 154 is in a controlled environment, the photometric stereo imaging technique can again be used to capture a 2D topographic surface image of the sample 154, from which a 3D topographic surface image of the sample 154 can be synthesized.
[0013] The 3D topographic surface capture device 100 is portable, allowing it to be used to capture 3D topographic surface images of samples (such as sample 154) that are too large and / or immovable to be moved to the location of the capture device 100 and placed in the tray 102. The capture device 100 can be completely self-contained, including sufficient components to perform photometric stereo imaging to generate and store 3D topographic surface images of the sample without requiring any other device, such as a host computing device. The stored images can then be transmitted to other devices.
[0014] Figure 2A , 2B 2C show a perspective view, an exploded perspective view, and a cross-sectional view of an example 3D topography surface capture device 100, respectively. Figure 2A In FIG. 1 , the tray 102 is removably attached to the upper portion 101 of the capture device 100, and the cover 202 covers and hides the internal components of the upper portion 101 from view. Figure 2B In FIG. 1 , the tray 102 is shown separated from the upper portion 101 of the capture device 100 and the cover 202 has also been removed from the upper portion 101. Figure 2C In the embodiment, the tray 102 is again removably attached to the upper part 101 of the capture device, and although Figure 2C The cover 202 is shown in FIG. 2 , but the cross-sectional nature of the figure shows internal components that would otherwise be hidden by the cover 202 .
[0015] The 3D topography surface capture device 100 includes a housing 222, which may also be referred to as a frame, and which is covered by the cover 202. The housing 222 extends from the bottom of the upper portion 101 of the capture device 100 to the top of the upper portion 101 of the device 100. The housing 222 may be constructed as a single integrated fixture, or as multiple parts attached to each other. The housing 222 may be made of rigid plastic, metal, or other materials.
[0016] The 3D topographic surface capture device 100 includes a tray 102 that can be removably attached to the bottom end of the housing 222. The tray 102 can be made of the same or different material as the housing 222. The tray 102 can be removably attached to the housing 222 magnetically or mechanically. As an example of the latter, when the tray 102 is aligned with the housing 222 and pushed against the housing 222, the tray 102 can have a fork that snaps into a corresponding hole in the housing 222. As another example, when the tray 102 is aligned with the housing 222, the tray 102 can have a protrusion that fits into a corresponding groove in the housing 222, wherein subsequent rotation of the tray 102 locks the tray 102 in place. As a third example, the tray 102 can be fixed to the housing 222 using screws.
[0017] The tray 102 includes a sample surface 226. Figure 1A During use of the 3D topography surface capture device 100 in the operating mode of the tray 102, the sample 104 is fixed to the sample surface 226. The sample 104 can be fixed to the sample surface 226 of the tray 102 in a variety of different ways. As an example, the sample 104 can be adhesively fixed to the sample surface 226. As another example, the sample 104 can instead be held in place with a physical clamp that forms part of the tray 102.
[0018] The 3D topography surface capture device 100 includes a light source 224 mounted around the perimeter of a housing 222. The light source 224 can be in the form of a group of light emitting diodes (LEDs) with a top mounted heat sink and can be linearly polarized. The light source 224 projects light directionally into an internal cavity or volume of the housing 222 to illuminate the sample 104 or 154. The light source 224 can be controlled individually. For example, LEDs oriented in the same direction relative to the housing 222 form a light source 224, which can be controlled independently of light sources 224 formed by LEDs oriented in other directions relative to the housing 222.
[0019] The 3D topography surface capture device 100 includes an imaging subassembly 242 mounted at the top end of the housing 222. The imaging subassembly 242 includes an image capture sensor 242A and one or more polarization state observers 242B, which are directed inwardly into the interior cavity or volume of the housing 222 and thus toward the sample 104 or 154. The image capture sensor 242A can be of the type currently available in smartphones, which are not typically used in 3D topography surface capture devices. As an example, the image capture sensor 242A can be an autofocus wide-angle camera device with autofocus and optical image stabilization (OIS). The polarization state observer 242B can be a mechanical linear polarizer (LP) mechanism, such as a rotatable polarizer wheel, or a solid-state liquid crystal (LC) phase retarder with LP. The polarization state observer 242B polarizes the light captured by the image capture sensor 242A.
[0020] The 3D topography surface capture device 100 includes mounting hardware 204 mounted to or as part of the housing 222 and an output indicator 206 visible through the cover 202. The mounting hardware 204 may be in the form of threaded holes to attach the housing 222, and therefore the capture device 100, to a mount 152 such as a tripod to fixedly position the bottom end of the housing 222 so that it is positioned above the housing 222. Figure 1B In an operational mode of abutting the sample 154. In one embodiment, the output indicator 206 may be an LED.
[0021] The 3D topography surface capture device 100 includes circuitry 244 mounted to the housing 222 . Figure 3 2 shows a block diagram of an example of such circuit 244 in one embodiment. Specifically, circuit 244 may include control circuit 302, and may also include vibration sensor 304, wireless transceiver 306 and / or storage device 308. Control circuit 302 is a circuit that controls imaging subassembly 242 and light source 224 to capture images under different lighting conditions. Figure 1A The first mode of the sample 104 or Figure 1B The control circuit 302 then synthesizes or generates - and the capture device 100 thus effectively captures - a 3D topographic surface image of the sample 104 or 154 in question using photometric stereo imaging techniques.
[0022] For example, to capture images of the sample 104 or 154 under different lighting conditions, the control circuit 302 may selectively turn on different light sources 224, either individually or in different groups, and, via controlling the polarization state observer 242B, cause the image capture sensor 242A to capture 2D topographic surface images of the sample 104 or 154 at different polarizations. Once the necessary images under such different lighting conditions have been captured, the control circuit 302 may then synthesize the captured 2D images into a 3D topographic surface image of the sample 104 or 154. A given lighting condition is controlled by two factors: the intensity of the light source 224 and which light source 224 is turned on when the image is captured, and the polarization of the polarization state observer 242B through which the image capture sensor 242A captures the image.
[0023] When synthesizing a 3D topographic surface image of the sample 104 or 154 from a plurality of captured 2D topographic surface images, the control circuit 302 may take into account the orientation and color of the varying illumination light source (i.e., the set of light sources 224). The control circuit 302 may also generate metadata from the captured 2D topographic surface images by calculation. For example, such metadata may be about the surface reflection characteristics of the sample, and / or about the transmitted light characteristics of the sample. In this case, the control circuit 302 may also take into account the orientation and color of the varying illumination light source.
[0024] The control circuit 302 may be implemented as a processor and a memory or other non-transitory computer-readable data storage medium storing program code executable by the processor. In the case where the processor is a special-purpose processor, the processor and the memory may be integrated within an application-specific integrated circuit (ASIC). The processor may alternatively be a general-purpose processor such as a central processing unit (CPU), in which case the memory may be a separate semiconductor or other type of volatile or non-volatile memory.
[0025] The vibration sensor 304 detects vibrations of the 3D topographic surface capture device 100. The vibration sensor 304 may be in the form of a triaxial or other type of accelerometer. The vibrations experienced by the capture device 100 may negatively affect the accuracy and precision of the captured 2D image, resulting in a synthetically degraded 3D topographic surface image of the sample. In one embodiment, when the vibration sensor 304 detects vibrations greater than a threshold that can affect the capture of the 3D topographic surface image, the output indicator 206 is illuminated to indicate this, so as to notify the user of the capture device 100. In the same or different embodiments, the control circuit will control the light source 224 and the illumination subassembly 242 to capture a 2D image only in response to detected vibrations less than the threshold.
[0026] The wireless transceiver 306 allows other devices to communicate wirelessly with the 3D topography surface capture device 100. For example, the wireless transceiver 306 can be a Wi-Fi, Bluetooth, or NFC transceiver, thereby allowing another device to communicate directly or indirectly with the capture device 100. Examples of direct wireless interconnection include a peer-to-peer Bluetooth connection, an NFC connection, a Wi-Fi direct connection, or a dedicated (ad hoc) Wi-Fi network. Examples of indirect wireless interconnection include the capture device 100 joining a wireless local area network (WLAN) to communicate with another device, which is directly connected to another network such as the Internet through the WLAN or via WLAN communication.
[0027] As described above, the 3D topography surface capture device 100 can be used in Figure 1A In the mode, you can Figure 1B The capture device 100 may operate independently in a mode without requiring a communication or conductive connection with any other device to capture a 3D topographic surface image. However, a communication connection with another device via the wireless transceiver 306 may allow the capture device 100 to operate instead of or also under the control of another device. A communication connection with another device via the wireless transceiver 306 may also allow the capture device 100 to send a synthesized 3D topographic surface image and / or computationally generated metadata to another device.
[0028] The storage device 308 may be a solid state drive (SSD) or another type of storage device. The storage device 308 stores the 3D topographic surface images. For example, when the imaging subassembly 242 captures images under different lighting conditions, the imaging subassembly 242 may store the captured 2D images on the storage device 308, either directly or through the control circuit 302. Thereafter, the control circuit 302 retrieves the captured 2D images from the storage device 308 to synthesize the 3D topographic surface images, and then stores them on the storage device 308 together with any computationally generated metadata.
[0029] Back to reference Figure 2A , 2B 2C, the 3D topographic surface capture device 100 may include an input mechanism 248 accessible through the cover 202. The input mechanism 248 may be in the form of a physical or capacitive button and receive an image capture initiation input from a user. The control circuit 302 may thus initiate control of the light source 224 and the imaging subassembly 242 to capture a 2D topographic surface image, and then synthesize a 3D topographic surface image in response to the input mechanism 248 receiving such an image capture initiation input.
[0030] The 3D topography surface capture device 100 may include a connector 250 accessible through the cover 202. The connector 250 may be a universal serial bus (USB), RJ45, or other type of connector. The connector 250 allows the capture device 100 to communicate with another device in a wired manner, as opposed to the wireless manner provided by the wireless transceiver 306. For example, the connector 250 may be connected to one end of a cable, with the other end of the cable connected to another device. As another example, the connector 250 may provide a wired connection to a network, to which another device is connected in a wired and / or wireless manner.
[0031] The 3D topography surface capture device 100 includes a battery 246. The battery 246 can be charged through the connector 250, via a different connector accessible through the cover 202, or inductively (i.e., wirelessly). The battery 246 provides for portable use of the capture device 100 without having to plug the device 100 into an external power source such as a wall outlet. The battery 246 powers the various components of the capture device 100 that have been described, such as the light source 224, the imaging subassembly 242, and the circuitry 244.
[0032] Figure 4 Shown in Figure 1B in the operating mode, then in Figure 1A Example method 400 of using the 3D topography surface capture device 100 in the operating mode of Figure 1B The capture device 100 is operated in the capture mode, and assuming that the tray 102 is currently attached to the housing 222 of the capture device 100, the tray 102 is first removed from the housing 222 (402). A mount 152, such as a tripod, can be attached to the mounting hardware 204 of the capture device 100 (404), and then the capture device 100 is positioned so that the bottom end of the housing 222 is placed adjacent to the sample 154 (406).
[0033] Then, the control circuit 302 of the 3D topographic surface capture device 100 controls the light source 224 and the imaging subassembly 242 so that the final Figure 1B A 2D topographic surface image of the sample 154 is captured in the operating mode of the capture device 100 (408). For example, the user can actuate the input mechanism 248 to initiate image capture. As another example, such image capture initiation input can be provided via another device that is communicatively connected to the capture device 100 via the wireless transceiver 306 or the connector 250. A 3D topographic surface image can then be synthesized from the captured 2D image and other metadata can be generated by calculation.
[0034] Then, in order to Figure 1AThe 3D topography surface capture device 100 is operated in the operating mode of FIG. 4 . If the mounting hardware 204 of the capture device 100 is currently attached to the mounting member, the mounting hardware 204 can be first removed from the mounting member 152 (410). Assuming that the tray 102 is currently separated from the housing 222, the sample 104 is placed (e.g., fixed) to the sample surface 226 of the tray 102 (412), and the tray 102 is removably attached to the housing 222 (414). Then, the control circuit 302 of the 3D topography surface capture device 100 controls the light source 224 and the imaging subassembly 242 so that in Figure 1A In the operating mode of the present invention, a 2D topographic surface image (416) of the sample 154 is captured, which can be similar to the Figure 1B 3D topographic surface images can then be similarly synthesized and other metadata generated computationally from the captured 2D images.
[0035] Figure 5 A block diagram of an example 3D topographic surface capture device 100 is shown. The capture device 100 includes a housing 222 and a light source 224 mounted around the perimeter of the housing 222. The capture device 100 includes an imaging subassembly 242, which includes a polarization state observer 242B and an image capture sensor 242A, which are mounted to the top end of the housing 222 and pointed inwardly into the housing 222. The capture device 100 includes a control circuit 302 to control the light source 224 and the imaging subassembly 242 to capture a 3D topographic surface image of the sample 104 or 154 at the bottom end of the housing 222.
[0036] The 3D topography surface capture device 100 includes a tray 102 that is removably attached to the bottom end of the housing 222 to allow the 3D topography surface capture device 100 to be removably attached to the bottom end of the housing 222. Figure 1A The first mode and Figure 1B The capture device 100 can be switchably configured between a first mode and a second mode. In the first mode, the tray 102 is attached to the bottom end of the housing 222, and the sample 104 is attached to the tray 102. In the second mode, the tray 102 is removed from the bottom end of the housing 222, and the bottom end of the housing 222 is placed adjacent to the sample 154.
[0037] Therefore, especially in Figure 1BIn the mode, the capture device 100 can therefore be used to capture 2D topographic surface images of samples 154 that are too large to fit in the device 100 (i.e., on the tray 102 of the device 100) and / or cannot be moved. The capture device 100 can operate completely independently to capture such images, and then synthesize 3D topographic surface images and generate other metadata without having to be communicatively or conductively connected to any other device and without having to be plugged into an external power source. Thus, the capture device 100 provides a portable and independent means by which to synthesize 3D topographic surface images using photometric stereo imaging and other physically based rendering (PBR) techniques.
Claims
1. A three-dimensional (3D) topographic surface capture device, comprising: case; a light source mounted around the perimeter of the housing; an imaging subassembly, the imaging subassembly including a polarization state observer and an image capture sensor, the polarization state observer and the image capture sensor being mounted to a top end of the housing and pointing inwardly into the housing; a control circuit for controlling the light source and the imaging subassembly to capture two-dimensional (2D) topographic surface images from a plurality of illumination orientations of the sample at the bottom end of the housing and synthesize a 3D topographic surface image of the sample from the 2D topographic surface images; and A tray removably attached to the bottom end of the housing to switchably configure the 3D topography surface capture device between the following modes: a first mode, wherein the tray is attached to a bottom end of the housing and a sample is attached to the tray; and A second mode, wherein the tray is removed from the bottom end of the housing and the bottom end of the housing is placed around the sample.
2. The 3D topography surface capturing device according to claim 1, further comprising: Mounting hardware is mounted to or is a portion of the housing to attach the housing to the mount to fixedly position a bottom end of the housing to abut a sample in the second mode.
3. The 3D topography surface capturing device according to claim 2, further comprising: A vibration sensor is mounted to the housing to detect vibration of the 3D topography surface capture device.
4. The 3D topography surface capturing device according to claim 3, further comprising: Output indicator that indicates whether the detected vibration is greater than the threshold.
5. The 3D topography surface capturing device according to claim 3, wherein: The control circuit controls the light source and the imaging subassembly to capture a 2D topographic surface image of the sample only in response to detected vibrations that are less than a threshold value.
6. The 3D topography surface capture device according to claim 1, further comprising one or more of the following: An input mechanism for receiving an image capture initiation input, wherein: The control circuit initiates control of the light source and the imaging subassembly in response to the image capture initiation input to capture a 2D topographic surface image of the sample; a storage device mounted to the housing, wherein the control circuit stores a 2D topographic surface image of the sample; a battery mounted to the housing for powering the light source, the imaging subassembly, and the control circuit; a wireless transceiver mounted to the housing for wireless communication with a device other than the 3D topography surface capture device; and A connector for communicating with a device other than the 3D topography surface capturing device in a wired manner.
7. The 3D topographic surface capturing device according to claim 6, wherein: The 3D topographic surface capture device is capable of operating independently in both the first mode and the second mode without requiring a communication or conductive connection with any other device to capture a 2D topographic surface image of the sample.
8. The 3D topographic surface capturing device according to claim 1, wherein: The control circuit is further configured to generate metadata about surface reflectance characteristics of the sample from the 2D topographic surface image by calculation.
9. The 3D topographic surface capturing device according to claim 1, wherein: The control circuit is further configured to generate metadata about the transmitted light characteristics of the sample from the 2D topographic surface image by calculation.
10. A method comprising: removing a tray removably attached to a bottom end of a housing of a three-dimensional (3D) topographic surface capture device, the 3D topographic surface capture device comprising a light source and an imaging subassembly mounted around a perimeter of the housing, the imaging subassembly comprising a polarization state observer and an image capture sensor mounted to a top end of the housing and pointing inwardly into the housing; placing the bottom end of the housing around the sample; and In the mode of the 3D topographic surface capture device, the control circuit of the 3D topographic surface capture device controls the light source and the imaging subassembly to capture a two-dimensional (2D) topographic surface image of the sample at the bottom end of the shell, and synthesizes a 3D topographic surface image of the sample from the 2D topographic surface image.
11. The method according to claim 10, further comprising: attaching mounting hardware of the 3D topography surface capture device to a mount, the mounting hardware being mounted to or being a part of the housing, Wherein, placing the bottom end of the housing adjacent to the sample comprises moving or adjusting the mounting member so that the bottom end of the housing abuts the sample.
12. The method according to claim 10, wherein: The mode is the second mode, and the method further comprises: Place different samples in a tray; removably attaching the tray to a bottom end of the housing; and In a first mode of the 3D topographic surface capturing device, the control circuit is controlled to control the light source and the imaging subassembly to capture 3D topographic surface images of different samples at the bottom end of the housing.
13. The method according to claim 10, wherein: The 3D topography surface capture device further comprises: An input mechanism for receiving an image capture start input; A storage device for storing a 3D topographic surface image of the sample; a battery mounted to power the light source, the imaging subassembly, and the control circuit; and A communication mechanism is used to communicate with a device other than the 3D topography surface capturing device.
14. The method according to claim 10, wherein: The 3D topographic surface capture device is capable of operating independently in the described mode without requiring a communication or conductive connection with any other device to capture the 3D topographic surface image of the sample.
15. The method according to claim 10, wherein: The control circuit also generates one or both of the following from the 2D topographic surface image: metadata regarding the surface reflectance properties of the sample; and Metadata about the transmitted light properties of the sample from the 2D topographic surface image.
