A method and apparatus for three-dimensional imaging

Abstract

Embodiments of the present application provide a three-dimensional imaging method and device, the three-dimensional imaging method comprising: obtaining N groups of light signals, each group of light signals comprising M light signals, each light signal corresponding to a pixel of a light modulation device, the M light signals in the same group of light signals corresponding to M pixels of a detector device respectively, N and M being positive integers greater than or equal to 2, at least one group of light signals in the N groups of light signals being obtained at a different time from the remaining groups of light signals; determining a three-dimensional image based on the N groups of light signals, the three-dimensional image comprising N sub-three-dimensional images, the N sub-three-dimensional images corresponding to the N groups of light signals respectively. Through the scheme, embodiments of the present application obtain corresponding N sub-three-dimensional images based on N groups of light signals having a sequence in acquisition time, thereby combining the N sub-three-dimensional images to form a three-dimensional image, so that a three-dimensional image with high spatial resolution can be obtained at a low cost under existing process conditions.

Description

Technical Field

[0001] This application relates to the field of three-dimensional imaging technology, and more specifically, to a method and apparatus for three-dimensional imaging. Background Technology

[0002] As the eyes of various intelligent devices, 3D sensing systems can acquire complete geometric information of 3D scenes in the real world, thereby using images with depth information to achieve accurate digitization of scenes and realize functions such as high-precision recognition, positioning, reconstruction, and scene understanding.

[0003] As the most commonly used technical solution in the field of three-dimensional sensing systems, time of flight (TOF) is a depth measurement technology that measures the round-trip flight time of an actively emitted light pulse between the transmitting device, the target object, and the receiving device, and obtains accurate distance information based on the speed of light.

[0004] However, due to limitations in manufacturing processes or the size of readout processing circuits, photodetectors used in TOF technology to measure time of flight, such as indirect time of flight (i-TOF) image sensors and single photon avalanche diode (SPAD) arrays, cannot achieve high-pixel imaging at a lower cost. Summary of the Invention

[0005] This application provides a method and apparatus for three-dimensional imaging, which can acquire high spatial resolution three-dimensional images at a lower cost under existing process conditions.

[0006] In a first aspect, a method for three-dimensional imaging is provided, comprising: acquiring N sets of optical signals, each set of optical signals including M optical signals, each optical signal corresponding one-to-one with a pixel of an optical modulation device, the M optical signals in the same set of optical signals respectively corresponding to M pixels of a detection device, N and M being positive integers greater than or equal to 2, and at least one set of optical signals in the N sets of optical signals being acquired at a different time than the other sets of optical signals; and determining a three-dimensional image based on the N sets of optical signals, the three-dimensional image including N sub-three-dimensional images, the N sub-three-dimensional images corresponding one-to-one with the N sets of optical signals.

[0007] Through this scheme, the embodiments of this application obtain N corresponding sub-3D images based on N sets of optical signals that have a sequential acquisition time. These N sub-3D images are then combined to form a 3D image, thereby enabling the acquisition of high spatial resolution 3D images at a lower cost under existing process conditions.

[0008] In conjunction with the first aspect, in some possible implementations of the first aspect, acquiring N sets of optical signals includes: determining N sets of modulation signals, each set of modulation signals being used to open the optical path corresponding to a set of pixels in the optical modulator; using the N sets of modulation signals to open the optical path corresponding to the pixels in the optical modulator, and acquiring the N sets of optical signals.

[0009] By acquiring N sets of optical signals in a sequential manner using modulation, and then obtaining N corresponding sub-3D images based on these N sets of optical signals, and combining these N sub-3D images to obtain a single 3D image, this embodiment of the application can obtain a high-resolution 3D image under existing process conditions.

[0010] In conjunction with the first aspect, in some possible implementations of the first aspect, the acquisition time of each of the N groups of optical signals is different.

[0011] By acquiring a set of optical signals in a time period, the embodiments of this application can acquire N sets of optical signals in N time periods, acquire N independent sub-3D images based on the N sets of optical signals, and combine the N independent sub-3D images to obtain a 3D image with a resolution of N*M, thereby improving the resolution of the 3D image.

[0012] In conjunction with the first aspect, in some possible implementations of the first aspect, the pixels of the optical modulator are arranged in M ​​matrices, each matrix including N pixels, each pixel corresponding to one optical signal in each group of optical signals. The use of N groups of modulation signals to open the optical path corresponding to the pixel in the optical modulator and to acquire the N groups of optical signals includes: using the N groups of modulation signals to simultaneously open the optical path corresponding to the pixel in the M matrices corresponding to one group of modulation signals, and sequentially opening the optical path corresponding to the pixel in the M matrices corresponding to the N groups of modulation signals.

[0013] In the time sequence of acquiring only one set of optical signals each time, the M pixels corresponding to the N sets of modulation signals in the M matrices will be turned on simultaneously, and only the optical path corresponding to the pixel of the corresponding modulation signal in each matrix will be turned on, while the optical paths corresponding to the other pixels in each matrix will remain closed. In this way, the embodiments of this application can reduce the influence of stray light and avoid crosstalk between different sets of optical signals, thereby further improving the resolution of the three-dimensional image.

[0014] In conjunction with the first aspect, in some possible implementations of the first aspect, the ratio N of the pixels of the optical modulation device to the pixels of the detection device is a square number.

[0015] By controlling the pixel ratio of the optical modulation device and the detector device to a flat number, the embodiments of this application can control the aspect ratio of the final three-dimensional image to remain unchanged.

[0016] In conjunction with the first aspect, in some possible implementations of the first aspect, each optical signal is obtained by focusing the signal light of the corresponding external region through a lens.

[0017] In conjunction with the first aspect, in some possible implementations of the first aspect, the optical modulation device is any one of the following: a liquid crystal light valve, a digital micromirror device, and a silicon-based liquid crystal.

[0018] Secondly, a three-dimensional imaging apparatus is provided, comprising: an optical modulation device for acquiring N sets of optical signals, each set of optical signals including M optical signals, each optical signal corresponding one-to-one with a pixel of the optical modulation device, the M optical signals in the same set of optical signals respectively corresponding to M pixels of a detection device, N and M being positive integers greater than or equal to 2, and at least one set of optical signals in the N sets of optical signals being acquired at a different time than the other sets of optical signals; and a detection device for determining a three-dimensional image based on the N sets of optical signals, the three-dimensional image including N sub-three-dimensional images, the N sub-three-dimensional images corresponding one-to-one with the N sets of optical signals.

[0019] In conjunction with the second aspect, in some possible implementations of the second aspect, the device further includes: a control module, configured to determine N sets of modulation signals, each set of modulation signals being used to open the optical path corresponding to a set of pixels in the optical modulator; the control module is also configured to use the N sets of modulation signals to open the optical path corresponding to the pixels in the optical modulator and acquire the N sets of optical signals.

[0020] In conjunction with the second aspect, in some possible implementations of the second aspect, the acquisition time of each of the N groups of optical signals is different.

[0021] In conjunction with the second aspect, in some possible implementations of the second aspect, the pixels of the optical modulation device are arranged in M ​​matrices, each matrix including N pixels, each pixel corresponding to one optical signal in each group of optical signals, and the control module is used to: use the N groups of modulation signals to simultaneously open the optical path corresponding to the pixel in the M matrices corresponding to one group of modulation signals, and sequentially open the optical path corresponding to the pixel in the M matrices corresponding to the N groups of modulation signals.

[0022] In conjunction with the second aspect, in some possible implementations of the second aspect, the ratio N of the pixels of the optical modulator to the pixels of the detector is a square number.

[0023] In conjunction with the second aspect, in some possible implementations of the second aspect, the transpose further includes: a lens for focusing the signal light of the corresponding external region to obtain each optical signal.

[0024] In conjunction with the second aspect, in some possible implementations of the second aspect, the optical modulation device is any one of the following: a liquid crystal light valve, a digital micromirror device, and a silicon-based liquid crystal.

[0025] In conjunction with the second aspect, in some possible implementations of the second aspect, the device further includes: a transmitting device for emitting an amplitude-modulated light beam for measuring depth information of a target object.

[0026] Thirdly, a lidar system is provided, which includes the apparatus described in the second aspect and any possible implementation thereof, and further includes a transmitter, a receiver, a signal processing device, and an image processing device.

[0027] Fourthly, a vehicle is provided that includes the means described in the second aspect and any possible implementation thereof, the vehicle further including: a wireless communication device, a display screen, and an image processing device.

[0028] Fifthly, an indoor three-dimensional sensing system is provided, which includes the apparatus described in the second aspect and any possible implementation thereof, and further includes an image processing device, a transmitting device, and a receiving device. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of an application scenario provided in an embodiment of this application.

[0030] Figure 2 This is a schematic flowchart of a three-dimensional imaging method according to an embodiment provided in this application.

[0031] Figure 3 This is a schematic block diagram illustrating the pixel correspondence between an optical modulation device and a detector device, as provided in an embodiment of this application.

[0032] Figure 4 This is a schematic diagram of a timing sequence provided in an embodiment of this application.

[0033] Figure 5 This is a schematic block diagram of a three-dimensional imaging device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0035] As the eyes of various smart devices, 3D sensing systems can acquire complete geometric information from 3D scenes in the real world, and based on this geometric information, or depth information, can construct a complete 3D scene of the real world.

[0036] Figure 1This is a schematic diagram illustrating an application scenario provided in an embodiment of this application. Figure 1 In the illustrated application scenario, the 3D sensing system measures the round-trip time of a light pulse between a transmitting device, a target object (e.g., a residence, office building, bus, hotel, etc.), and a receiving device. Based on the speed of light, it obtains precise distance information and constructs a 3D scene of the surrounding environment. For example, the 3D sensing system measures the light pulse traveling to and from a residence and, based on the speed of light, obtains depth information about the residence, thus constructing a 3D scene of the residence; or, the 3D sensing system measures the light pulse traveling to and from a bus and, based on the speed of light, obtains depth information about the bus, thus constructing a 3D scene of the bus, and so on.

[0037] One of the core elements of a 3D sensing system in constructing a 3D scene of the real world is acquiring a high spatial resolution 3D image and then constructing the 3D scene based on that image. Existing technologies improve the spatial resolution of the 3D image results from the 3D sensing system by fusing low spatial resolution images of SPAD arrays with high spatial resolution RGB images from two dimensions to estimate the precise depth information of the regions between SPAD pixels.

[0038] However, the accuracy of the three-dimensional images obtained by the three-dimensional sensing system based on the above-mentioned technologies is limited, and additional RGB image information is required. In addition, more powerful computing power and longer image processing time are needed to obtain three-dimensional images with high spatial resolution and high accuracy. In other words, this increases the cost of the three-dimensional sensing system to acquire high-pixel three-dimensional images.

[0039] In view of the above-mentioned technical problems, this application provides a method and apparatus for three-dimensional imaging, which can acquire high spatial resolution three-dimensional images at a lower cost under existing process conditions.

[0040] It should be noted that, Figure 1 The application scenarios shown are for illustrative purposes only and do not limit the actual application areas of the three-dimensional imaging method described in this application. For example, the three-dimensional imaging method described in this application is not limited to the measurement of amplitude-modulated beams, such as the measurement of light pulses, the measurement of sine waves, the measurement of square waves, etc., and other methods can also be used for measurement. This application does not make specific limitations.

[0041] It should be noted that the detection device in the embodiments of this application can be a SPAD array, an i-TOF image sensor, or other sensors. The embodiments of this application do not impose specific limitations.

[0042] The following will combine Figure 2 The method for three-dimensional imaging provided in this application is described.

[0043] Figure 2 This is a schematic flowchart of a three-dimensional imaging method provided in this application. Specific details are as follows: Figure 2 As shown. Method #200 includes:

[0044] S210, acquire N sets of optical signals, each set of optical signals includes M optical signals, each optical signal corresponds one-to-one with a pixel of the optical modulation device, the M optical signals in the same set of optical signals correspond to M pixels of the detection device respectively, and at least one set of optical signals in the N sets of optical signals is acquired at a different time than the other sets of optical signals.

[0045] Specifically, at least one of the N groups of optical signals is acquired at a different time than the other groups. This can be understood as the N groups of optical signals having a temporal sequence. This temporal sequence means that there is a chronological order among the N groups of optical signals. More specifically, the N groups of optical signals are not acquired simultaneously within a single time period, but rather multiple times across several time periods. That is, among the N groups of optical signals, at least one group has an acquisition time different from the other groups.

[0046] For example, when there are nine sets of optical signals, the timing sequence can be that each set of optical signals is acquired sequentially within its corresponding time period. For instance, the first set of optical signals is acquired in the first time period, the second set of optical signals is acquired in the second time period, and so on, with the ninth set of optical signals acquired in the ninth time period. This timing sequence can be understood as a sequential arrangement, or one time period corresponds to one set of optical signals.

[0047] For example, when there are nine sets of optical signals, the timing sequence can be to acquire multiple sets of optical signals simultaneously within a time period. It should be understood that acquiring multiple sets of optical signals simultaneously within a time period can mean that the number of sets of optical signals acquired simultaneously in different time periods can be the same or different. When the number of groups is the same, for example, acquiring the first, second, and third groups of light signals simultaneously in the first time period, acquiring the fourth, fifth, and sixth groups of light signals simultaneously in the second time period, and acquiring the seventh, eighth, and ninth groups of light signals simultaneously in the third time period, the pixels of each sub-3D image are uniformly consistent. In this way, the resolution of the obtained 3D image can be uniformly improved, that is, a uniform improvement of 3 times. When the number of groups is not the same, for example, acquiring the first and second groups of light signals in the first time period, acquiring the third, fourth, and fifth groups of light signals in the second time period, and acquiring the remaining sixth, seventh, eighth, and ninth groups of light signals in the third time period, the resolution of the obtained 3D image can also be improved by 3 times.

[0048] It should be understood that this timing sequence can be divided into two main categories: one category is: a time period corresponds to a set of optical signals, and the other category is: a time period corresponds to multiple sets of optical signals, and these multiple sets of optical signals are a portion of the N sets of optical signals, not all of the N sets of optical signals.

[0049] It should be noted that the duration of each time period can be the same or different. In addition, there may or may not be an interval between each time period. This application does not impose any specific limitations on this embodiment.

[0050] It should be understood that the embodiments of this application do not impose specific limitations on the timing, only requiring that the N groups of optical signals are not acquired simultaneously in the same time period.

[0051] It should be understood that the specific arrangement of the N groups of pixels in the optical modulation device can be either a clustered arrangement or a cross-arrangement. When arranged in a clustered manner, for example, all M pixels in each group are grouped together. For example, the N groups of pixels are arranged in N rows, with each row corresponding to one group of pixels; or, the N groups of pixels are divided into N arrays, each array including M pixels. When arranged in a cross-arrangement, for example, the N groups of pixels are divided into M matrices, and each matrix includes one pixel from each group of pixels, meaning that the pixels included in each matrix are composed of N groups of pixels. This application does not limit the specific arrangement of pixels within each group of N groups of pixels.

[0052] It should be understood that the embodiments of this application regulate the quantity and order of light signals captured by the detector in a timing and grouping manner. In this way, N groups of light signals with a sequential acquisition time can be acquired, thereby effectively improving the resolution of the three-dimensional image.

[0053] It should be understood that each optical signal corresponds one-to-one with a pixel of the optical modulator. This can be understood as follows: the number of pixels of the optical modulator determines the total number of optical signals in the N groups of optical signals, and each pixel corresponds to one optical signal. The corresponding optical signal can only be obtained when the optical path corresponding to each pixel is opened or turned on.

[0054] It should be understood that the M optical signals in each group of optical signals correspond to the M pixels of the detection device. This can be understood as follows: when the number of pixels of the detection device is M, then the number of optical signals included in each group of optical signals is M, and each optical signal corresponds one-to-one with each pixel.

[0055] S220, Based on N sets of optical signals, determine a three-dimensional image, which includes N sub-three-dimensional images, and each of the N sub-three-dimensional images corresponds one-to-one with one of the N sets of optical signals.

[0056] Specifically, one set of optical signals corresponds to one sub-3D image, and N sets of optical signals correspond to N sub-3D images. Since each set of optical signals includes M optical signals, the number of pixels of the detection device is M, and the resolution of a sub-3D image is M*1, the resolution of the obtained 3D image composed of N sub-3D images is generally N*M.

[0057] It should be understood that, depending on the timing method, this application can acquire three-dimensional images with different resolutions. For example, when the timing is one time period corresponding to one set of light signals, the total resolution of the three-dimensional image is N*M; or, when the timing is one time period corresponding to multiple sets of light signals, the total resolution of the three-dimensional image is greater than M*1 and less than N*M, but both can improve the total resolution of the three-dimensional image.

[0058] Through this scheme, the embodiments of this application obtain N corresponding sub-3D images based on N sets of optical signals that have a sequential acquisition time. These N sub-3D images are then combined to form a 3D image, thereby enabling the acquisition of high spatial resolution 3D images at a lower cost under existing process conditions.

[0059] More specifically, the embodiments of this application regulate the quantity and order of light signals captured by the detection device in a timing and grouping manner, so as to acquire N groups of light signals that are sequential in acquisition time, thereby effectively improving the resolution of the three-dimensional image.

[0060] As one possible implementation, N sets of optical signals are acquired, including:

[0061] N sets of modulation signals are determined, and each set of modulation signals is used to open the optical path corresponding to a set of pixels in the optical modulator.

[0062] Using N sets of modulation signals, the optical path corresponding to the pixel in the optical modulation device is opened, and N sets of optical signals are acquired.

[0063] Specifically, the timing sequence between the N sets of modulation signals is the same as the timing sequence between the N sets of optical signals, because there is a one-to-one correspondence between each set of modulation signals and each set of optical signals. For example, in this embodiment, the optical path corresponding to a corresponding pixel in the optical modulation device is controlled by modulation signals, and the corresponding optical signal is acquired. Therefore, the timing sequence between the N sets of modulation signals is the same as the timing sequence between the N sets of optical signals.

[0064] It should be understood that each of the N sets of modulation signals can control the optical path corresponding to a set of pixels in the optical modulator. In other words, each set of modulation signals can control the conduction and closure of the optical path corresponding to each pixel in that set of pixels, thereby controlling whether the corresponding set of optical signals is acquired or not.

[0065] It should be understood that the modulation signal can be a high or low level signal. For example, a high level is used to open the optical path corresponding to a pixel, and a low level is used to close the optical path corresponding to a pixel; or, a low level is used to open the optical path corresponding to a pixel, and a high level is used to close the optical path corresponding to a pixel; the modulation signal can also be a sine wave signal or a triangular wave signal, and so on.

[0066] It should be noted that the specific type of modulation signal is not limited in the embodiments of this application, as long as it can control the opening and closing of the optical path corresponding to the pixel.

[0067] It should be understood that the timing present in the N sets of modulation signals is the same as the timing present in the N sets of optical signals.

[0068] It should be noted that in this embodiment of the application, the acquisition of N sets of optical signals with a sequential acquisition time is achieved by modulating signals. However, other implementation methods that can achieve the same effect are not excluded. For example, by setting N sets of timers, and each set of timers includes M timers, and each timer is used to control the opening of the optical path corresponding to a pixel, it is also possible to acquire N sets of optical signals with a sequential acquisition time.

[0069] As one possible implementation, the acquisition time of each of the N groups of optical signals is different.

[0070] This application embodiment acquires N sets of corresponding optical signals within N time periods, determines N sub-3D images based on the N sets of optical signals, and combines the N sub-3D images based on the time sequence to obtain a complete 3D image, thereby significantly improving the resolution of the 3D image.

[0071] As one possible implementation, the pixels of the optical modulator are arranged in M ​​matrices, each matrix including N pixels. Each pixel corresponds to one optical signal in each group of optical signals. Then, using N groups of modulation signals, the optical path corresponding to the pixel in the optical modulator is opened to obtain N groups of optical signals, including:

[0072] Using these N sets of modulation signals, the optical paths corresponding to the pixels of the M matrices corresponding to the N sets of modulation signals are opened simultaneously, and the optical paths corresponding to the pixels of the M matrices corresponding to the N sets of modulation signals are opened sequentially.

[0073] Specifically, when the pixels of the optical modulation device are arranged in M ​​matrices, the optical paths corresponding to the M pixels in the M matrices corresponding to the modulation signals are opened simultaneously according to the order of the N sets of modulation signals. Then, the optical paths corresponding to the pixels in the M matrices corresponding to the N sets of modulation signals are opened in sequence, and finally, the N sets of optical signals are obtained.

[0074] For example, if N is 4, M is 4, and the order of the N groups of modulation signals is the first group of modulation signals, the second group of modulation signals, the third group of modulation signals, and the fourth group of modulation signals, then each of the four matrices includes four pixels, namely the first pixel, the second pixel, the third pixel, and the fourth pixel. The first pixel corresponds to one optical signal of the first group of optical signals, the second pixel corresponds to one optical signal of the second group of optical signals, the third pixel corresponds to one optical signal of the third group of optical signals, and the fourth pixel corresponds to one optical signal of the fourth group of optical signals. During the time period of the first group of modulation signals, the optical path corresponding to the first pixel in each of the four matrices is opened simultaneously, while the optical paths corresponding to the other pixels remain closed. Then, during the time period of the second modulation signal, the optical path corresponding to the second pixel in each of the four matrices is opened simultaneously, while the optical paths corresponding to the other pixels remain closed, and so on. The optical paths corresponding to the corresponding pixels in each of the four matrices will be opened simultaneously, but only the optical path corresponding to the pixel corresponding to the modulation signal will be opened, while the optical paths corresponding to the remaining pixels will remain closed.

[0075] In other words, by simultaneously activating the optical paths corresponding to the pixels of a set of modulation signals in M ​​matrices for each time period, a corresponding set of optical signals is obtained, resulting in a corresponding sub-3D image. Then, based on the activation order of the optical paths corresponding to each pixel in each matrix, the obtained N sub-3D images are combined according to this activation order to obtain a high-resolution 3D image. Thus, this embodiment of the application can achieve higher resolution 3D images while reducing the influence of stray light and avoiding crosstalk between different sets of optical signals, thereby further improving the resolution of the 3D image.

[0076] In the time sequence of acquiring only one set of optical signals each time, the M pixels corresponding to the modulation signal in the M matrices will be turned on simultaneously, and only the optical path corresponding to the pixel of the corresponding modulation signal in each matrix will be turned on, while the optical paths corresponding to the other pixels in each matrix will remain closed. In this way, the embodiments of this application can reduce the influence of stray light and avoid crosstalk between different sets of optical signals, thereby further improving the resolution of the three-dimensional image.

[0077] The following will combine Figure 3 and Figure 4 right Figure 2 The method for three-dimensional imaging shown will be described in further detail.

[0078] Figure 3 This is a schematic block diagram illustrating the pixel correspondence between an optical modulation device and a detector device according to an embodiment of this application. Specifically, as shown... Figure 3 As shown.

[0079] Assuming the optical modulator has 36 pixels, N is 9, and M is 4, all the pixels of the optical modulator can be divided into 4 matrices, and each matrix includes 9 pixels. Each pixel corresponds one-to-one with one of the optical signals in each of the 9 groups of optical signals.

[0080] It should be understood that Figure 3 The numbers 1-9 shown represent different groups of optical signals. For example, number 1 represents the first group of optical signals, number 2 represents the second group of optical signals, and so on. Number 9 represents the ninth group of optical signals, and each group of optical signals includes 4 optical signals.

[0081] exist Figure 3 In the schematic diagram shown, each dashed box represents a pixel of the optical modulation device, and each solid box represents a pixel of the detection device. The nine pixels of the optical modulation device correspond to the one pixel of the detection device. That is, the nine dashed boxes enclosed by the solid boxes of the optical modulation device represent the pixels represented by the one solid box of the detection device.

[0082] Figure 4 This is a schematic diagram of a timing sequence provided in an embodiment of this application. Specifically, as shown... Figure 4 As shown. Figure 4 T1 represents the first time period, T2 represents the second time period, and so on, with T9 representing the ninth time period. Only one set of optical signals can be acquired in each time period. For example, the first set of optical signals is acquired in time period T1, the second set of optical signals is acquired in time period T2, and so on, with the ninth set of optical signals acquired in time period T9.

[0083] It should be understood that this timing also applies to N modulated signals. For example, time period T1 indicates that the first modulated signal is in use, time period T2 indicates that the second modulated signal is in use, and so on, with time period T9 indicating that the ninth modulated signal is in use. Only one modulated signal can be used in each time period.

[0084] Combination Figure 3 and Figure 4 As can be seen, based on the timing sequence of one time period corresponding to one set of optical signals, or one time period corresponding to one set of modulation signals, this embodiment of the application sequentially uses each of the N sets of modulation signals, using the first set of modulation signals, the second set of modulation signals...the ninth set of modulation signals sequentially. In this case, only one pixel's optical path is opened within each matrix. For example, when the first set of modulation signals is used, the optical path corresponding to pixel number 1 in each matrix is ​​opened, while the optical paths corresponding to the remaining pixels in each matrix are closed. The detector then obtains a sub-3D image with a resolution of 4*1. Then, the optical paths corresponding to each remaining pixel in each matrix are sequentially opened, resulting in eight remaining sub-3D images, each with a resolution of 4*1. These nine sub-3D images are then combined according to the opening order of the optical paths corresponding to the pixels to obtain a 3D image with a resolution of 4*9. By sequentially opening each pixel within each matrix, this embodiment of the application can reduce the influence of stray light and avoid crosstalk between different sets of optical signals, thus further improving the resolution of the 3D image.

[0085] As one possible implementation, the ratio N between the pixels of the optical modulator and the pixels of the detector is a square number.

[0086] For example, if the optical modulation device has 36 pixels and the detection device has 4 pixels, then the pixel ratio between the two is 3. 2 :1, in other words, the pixel ratio of the optical modulation device to the detection device is 3. 2 If N is 9 and M is 4, then the pixel ratio of N to M is 4. 2 :1, in other words, the pixel ratio of the optical modulation device to the detection device is 4. 2Then the value of N is 16 and the value of M is 4.

[0087] By controlling the pixel ratio N of the optical modulation device and the detector device to be a square number, the embodiments of this application can control the aspect ratio of the final three-dimensional image to remain unchanged.

[0088] As one possible implementation, each optical signal is obtained by focusing the signal light of the corresponding external region through a lens. Thus, through the effect of lens imaging, the embodiments of this application enable a one-to-one correspondence between the object and the image. It should be understood that the corresponding external region can be understood as the corresponding imaging region of that pixel.

[0089] It should be understood that, in order to better understand the three-dimensional imaging method disclosed in the embodiments of this application, the embodiments of this application use liquid crystal light valves and digital micromirror devices (DMDs) as light modulation devices and SPAD arrays as detector devices to describe the three-dimensional imaging method of the embodiments of this application.

[0090] When the light modulation device is a liquid crystal light valve, assuming the liquid crystal light valve has 36 pixels and the SPAD array has 4 pixels, the 36 pixels of the liquid crystal light valve can be divided into 9 groups, with each group containing 4 pixels. Simultaneously, the 36 pixels of the light modulation device are arranged in 4 matrices, with each matrix containing one pixel from each group. In other words, one pixel in the SPAD array corresponds to 9 pixels in the liquid crystal light valve. For a detailed correspondence, please refer to [reference needed]. Figure 3 The schematic diagram of the pixel matrix arrangement of the optical modulation device shown is not repeated here.

[0091] After establishing the pixel correspondence between the SPAD array and the liquid crystal light valve, nine sets of modulation signals are set. Each set of modulation signals is used to control the opening and closing of the light path corresponding to a group of pixels. Specifically, the first set of modulation signals is used to control the opening and closing of the light path corresponding to the first group of pixels, and so on, with the ninth set of modulation signals used to control the opening and closing of the light path corresponding to the ninth group of pixels.

[0092] Based on Figure 3 Based on the matrix arrangement shown, in this embodiment of the application, a set of modulation signals is used to open the optical path corresponding to a set of pixels within a certain time period. However, the pixels included in each set of pixels are distributed in 4 matrices. Therefore, only the optical path corresponding to one pixel in each matrix is ​​opened. By analogy, nine sets of modulation signals are needed to open the optical path corresponding to the corresponding pixel in each matrix in sequence, thereby obtaining nine sub-3D images in sequence. After obtaining the nine sub-3D images, this embodiment of the application stitches the nine sub-3D images together according to the order of use of the modulation signals to form a complete high-resolution 3D image.

[0093] It should be understood that in the above scheme, each group of modulation signals includes 4 modulation signals, and the modulation signals are used to rotate the liquid crystal molecules corresponding to each pixel of the liquid crystal light valve, thereby opening the light path corresponding to the pixel and then obtaining the corresponding light signal.

[0094] When the optical modulation device is a DMD, the specific imaging method is as described above. The difference is that the modulation signal is used to deflect each micromirror of the DMD, thereby opening the optical path corresponding to the pixel and then acquiring the corresponding optical signal.

[0095] The following will combine Figure 5 The apparatus for three-dimensional imaging provided in the embodiments of this application will be described.

[0096] Figure 5 This is a schematic block diagram of a three-dimensional imaging device provided in an embodiment of this application. The device includes:

[0097] An optical modulation device is used to perform the aforementioned step S210, the details of which can be found in the description of the aforementioned method embodiments.

[0098] The detection device is used to perform the aforementioned step S220, and the details can be found in the description of the aforementioned method embodiments.

[0099] As one possible implementation, the device also includes:

[0100] The control module is used to execute the steps or methods involving the modulation signal in the foregoing method embodiments, and the details can also be found in the description of the foregoing method embodiments. For example, the control module may be a system-on-a-chip (SOC).

[0101] As one possible implementation, the device also includes a lens for focusing signal light on the corresponding external region to obtain each optical signal.

[0102] As one possible implementation, the device also includes a transmitting device for emitting an amplitude-modulated light beam for measuring depth information of a target object.

[0103] It should be noted that, in the embodiments of this application, each optical signal acquired by the optical modulation device is a signal carrying the depth information of the target object, which is returned by the light beam emitted by the transmitting device after it reaches the target object. The detection device constructs a three-dimensional image of the target object based on these optical signals.

[0104] As one possible implementation, the device further includes a receiving device for receiving a light beam carrying depth information of the target object. It should be understood that the light beam includes an optical signal.

[0105] It should be noted that, Figure 5 Each device and module included in the illustrated three-dimensional imaging apparatus is used to implement the corresponding methods and steps in the foregoing method embodiments.

[0106] This application embodiment also provides a lidar system, the lidar system including... Figure 5 The three-dimensional imaging device shown in the figure, the lidar system also includes: a transmitter, a receiver, a signal processing device and an image processing device.

[0107] This application embodiment also provides a vehicle, the vehicle including... Figure 5 The vehicle also includes a three-dimensional imaging device, a wireless communication device, a display screen, and an image processing device.

[0108] This application embodiment also provides an indoor three-dimensional sensing system, which includes... Figure 5 The three-dimensional imaging device shown, the indoor three-dimensional sensing system also includes: an image processing device, a transmitting device and a receiving device.

[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0114] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for three-dimensional imaging, characterized in that, include: N sets of optical signals are acquired, each set of optical signals includes M optical signals, each optical signal corresponds one-to-one with a pixel of an optical modulation device, the M optical signals in the same set of optical signals correspond to M pixels of a detection device respectively, N and M are positive integers greater than or equal to 2, and at least one set of optical signals in the N sets of optical signals is acquired at a different time than the other sets of optical signals. Based on the N sets of optical signals, a three-dimensional image is determined, the three-dimensional image comprising N sub-three-dimensional images, and the N sub-three-dimensional images corresponding one-to-one with the N sets of optical signals; The pixels of the optical modulation device are arranged in M ​​matrices, each matrix including N pixels, and each pixel corresponds to one optical signal in each group of optical signals. The pixels of each of the M matrices are arranged in M ​​matrices. The permutation is given by N, where N is a square number greater than 1.

2. The method according to claim 1, characterized in that, The acquisition of N sets of optical signals includes: N sets of modulation signals are determined, and each set of modulation signals is used to open the optical path corresponding to a set of pixels in the optical modulation device. Using the N sets of modulation signals, the optical path corresponding to the pixel in the optical modulation device is opened, and the N sets of optical signals are acquired.

3. The method according to claim 1, characterized in that, The acquisition time of each of the N groups of optical signals is different.

4. The method according to claim 2, characterized in that, The step of using N sets of modulation signals to open the optical path corresponding to the pixel in the optical modulation device and acquiring the N sets of optical signals includes: Using the N sets of modulation signals, the optical paths corresponding to the pixels of the M matrices corresponding to one set of modulation signals are opened simultaneously, and the optical paths corresponding to the pixels of the M matrices corresponding to the N sets of modulation signals are opened sequentially.

5. The method according to any one of claims 1 to 4, characterized in that, Each optical signal is obtained by focusing the signal light of the corresponding external region through a lens.

6. The method according to any one of claims 1 to 4, characterized in that, The optical modulation device is any one of the following: Liquid crystal light valves and digital micromirror devices.

7. A three-dimensional imaging device, characterized in that, include: An optical modulation device is used to acquire N sets of optical signals, each set of optical signals including M optical signals, each optical signal corresponding one-to-one with a pixel of the optical modulation device, the M optical signals in the same set of optical signals respectively corresponding to M pixels of the detection device, where N and M are positive integers greater than or equal to 2, and at least one set of optical signals in the N sets of optical signals is acquired at a different time than the other sets of optical signals. The detection device is used to determine a three-dimensional image based on the N sets of optical signals. The three-dimensional image includes N sub-three-dimensional images, and the N sub-three-dimensional images correspond one-to-one with the N sets of optical signals. The pixels of the optical modulation device are arranged in M ​​matrices, each matrix including N pixels, and each pixel corresponds to one optical signal in each group of optical signals. The pixels of each of the M matrices are arranged in M ​​matrices. The permutation is given by N, where N is a square number greater than 1.

8. The apparatus according to claim 7, characterized in that, The device further includes: The control module is used to determine N sets of modulation signals, each set of modulation signals being used to open the optical path corresponding to a set of pixels in the optical modulation device; The control module is further configured to use the N sets of modulation signals to open the optical path corresponding to the pixel in the optical modulation device and acquire the N sets of optical signals.

9. The apparatus according to claim 7, characterized in that, The acquisition time of each of the N groups of optical signals is different.

10. The apparatus according to claim 8, characterized in that, The control module is used for: Using the N sets of modulation signals, the optical paths corresponding to the pixels of the M matrices corresponding to the N sets of modulation signals are opened simultaneously, and the optical paths corresponding to the pixels of the M matrices corresponding to the N sets of modulation signals are opened sequentially.

11. The apparatus according to any one of claims 7 to 10, characterized in that, The device further includes: A lens is used to focus the signal light from the corresponding external region to obtain each optical signal.

12. The apparatus according to any one of claims 7 to 10, characterized in that, The optical modulation device is any one of the following: Liquid crystal light valves and digital micromirror devices.

13. The apparatus according to any one of claims 7 to 10, characterized in that, The device further includes: A transmitting device for emitting a light beam with modulated amplitude, the light beam being used to measure depth information of a target object.

14. A lidar system, characterized in that, The lidar system includes the device according to any one of claims 7 to 13. The lidar system also includes: Transmitter, receiver, signal processing device, and image processing device.

15. A vehicle, characterized in that, The vehicle includes the device according to any one of claims 7 to 13. The vehicle also includes: Wireless communication devices, displays, and image processing devices.

Images