Time-of-flight (ToF) based range camera and control method

Abstract

A time-of-flight (ToF) ranging camera and a control method are provided to reduce the influence of diffuse reflection on a to-be-measured point on the ranging result of the camera and improve the accuracy of the ranging of the ToF camera. In the camera, the distance between a first to-be-measured point and the camera is determined by a first light signal and a first reflected light signal, and the distance between a second to-be-measured point and the camera is determined by a second light signal and a second reflected light signal. Because different epipolar constraints are used, the first light signal and the first reflected light signal used to determine the distance between the first to-be-measured point and the camera do not interfere with the second light signal and the second reflected light signal used to determine the distance between the second to-be-measured point and the camera. That is, the different light reflection signals received by the light receiver through different epipolar lines do not interfere with each other, the influence of the diffuse reflection of the multi-channel light signal emitted by the light emitter on the to-be-measured point on the ranging result of the camera is reduced, and the accuracy of the ranging of the ToF camera is improved.

Description

Technical Field

[0001] This application relates to the field of optics, and more particularly to a time-of-flight (ToF) based ranging camera and control method. Background Technology

[0002] Time-of-flight (ToF) cameras can be used for distance measurement. The principle of distance measurement is to send light signals to an object through a light emitter and then receive the light signals returned from the object through a light receiver. The distance to the object is obtained by detecting the flight (round trip) time of the light signals.

[0003] Currently, ToF cameras typically determine the round-trip time of light signals during ranging by measuring the phase difference between the light signals emitted by the transmitter and received by the receiver. Furthermore, the transmission and reception of multiple light signals between the transmitter and receiver can optimize the ToF camera's ranging process, such as extending the ranging distance or range.

[0004] However, diffuse reflection is common on object surfaces. When the object being photographed has multiple reflected light paths from the light emitter, the light receiver will receive reflected light from many different paths and phases, which will interfere with the ranging process of the ToF camera and result in a low ranging accuracy of the ToF camera. Summary of the Invention

[0005] This application provides a time-of-flight (ToF) based ranging camera and control method to reduce the impact of diffuse reflection of multiple optical signals emitted by the optical transmitter at the measurement point on the camera ranging results, thereby improving the accuracy of ToF camera ranging.

[0006] The first aspect of this application provides a time-of-flight (ToF) based ranging camera. When capturing photos or videos using optical imaging principles, this camera can measure the distance to an object in a frame of the photo or video. The object includes one or more measurement points, such as a first measurement point, a second measurement point, etc. Specifically, the camera includes a light emitter and a light receiver. The origin of the light emitter is a first origin, and the emitting surface of the light emitter includes at least a first plane and a fourth plane. The origin of the light receiver is a second origin, and the receiving surface of the light receiver includes a second plane and a fifth plane.

[0007] During the distance measurement process, the light emitter in the camera emits a first light signal towards the first point to be measured. The first light signal is reflected by the first point to be measured to form a first reflected light signal, and the first light signal intersects with the first epipolar line, and the first reflected light signal intersects with the second epipolar line. The plane containing the first point to be measured, the first origin, and the second origin is a third plane, and the third plane intersects with the first plane at the first epipolar line, and the third plane intersects with the second plane at the second epipolar line.

[0008] Furthermore, during the ranging process, the light emitter in the camera is also used to emit a second light signal towards the second test point. The second light signal is reflected by the second test point to form a second reflected light signal, and the second light signal intersects with the third epipolar line, while the second reflected light signal intersects with the fourth epipolar line. The second test point is different from the first test point. The plane containing the first origin and the second origin of the second test point is the sixth plane, and the sixth plane intersects with the fourth plane at the third epipolar line and with the fifth plane at the fourth epipolar line. The third plane and the sixth plane are not coplanar.

[0009] Furthermore, during the distance measurement process, the light receiver in the camera is used to receive the first reflected light signal and the second reflected light signal. The first light signal and the first reflected light signal are used to determine the distance between the first measurement point and the camera, and the second light signal and the second reflected light signal are used to determine the distance between the second measurement point and the camera.

[0010] Based on the above technical solution, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point, and this first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera; and the second reflected light signal received by the optical receiver is formed by the reflection of the second light signal emitted by the optical transmitter at the second test point, and this second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Since the third plane and the sixth plane are not coplanar, i.e., the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line, the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. That is, the different light reflection signals received by the optical receiver through different epipolar lines do not interfere with each other, reducing the influence of diffuse reflection of multiple light signals emitted by the optical transmitter at the test point on the camera ranging results, and improving the accuracy of ToF camera ranging.

[0011] It should be noted that the intersection of the first optical signal and the first epipolar line can indicate that the transmission path of the first optical signal intersects the first epipolar line at one or more points, or it can indicate that the transmission path of the first optical signal passes through one or more points of the first epipolar line. Similarly, the intersection of the first reflected optical signal and the second epipolar line can indicate that the transmission path of the first reflected optical signal intersects the second epipolar line at one or more points, or it can indicate that the transmission path of the first reflected optical signal passes through one or more points of the second epipolar line. For example, when the first optical signal includes a beam of light (or multiple beams of light), the propagation path of the first optical signal passes through the first epipolar line and intersects the first epipolar line at one or more points. Subsequently, the first optical signal is reflected by the first test point to form a single beam (or multiple beams) of first reflected light signal, and the propagation path of the first reflected light signal passes through the second epipolar line and intersects the second epipolar line at one or more points. Similarly, the intersection of the second optical signal with the third pole line can indicate that the transmission path of the second optical signal intersects the third pole line at one or more points, or that the transmission path of the second optical signal passes through one or more points of the second pole line. Likewise, the intersection of the second reflected optical signal with the fourth pole line can indicate that the transmission path of the second reflected optical signal intersects the fourth pole line at one or more points, or that the transmission path of the second reflected optical signal passes through one or more points of the fourth pole line. For example, when the second optical signal includes a beam of light (or multiple beams of light), the propagation path of the second optical signal passes through the third pole line and intersects the third pole line at one or more points. Subsequently, the second optical signal is reflected by the second test point to form a single beam (or multiple beams) of second reflected optical signal, and the propagation path of the second reflected optical signal passes through the fourth pole line and intersects the fourth pole line at one or more points.

[0012] Furthermore, in this embodiment and subsequent embodiments, the emitting surface of the light emitter can specifically refer to the imaging surface of the light emitter in the camera pinhole model, that is, the plane equivalent to the imaging surface; the receiving surface of the light receiver can specifically refer to the imaging surface of the light receiver in the camera pinhole model, or, in other words, the sensor plane of the light receiver.

[0013] It should be noted that the distance between the first test point and the camera can be the distance between the first test point and the lens in the camera, the distance between the first test point and the geometric center in the camera, the distance between the first test point and the photosensitive element in the camera, or the distance between the first test point and other physical or virtual parts in the camera; no specific limitation is made here. Similarly, the distance between the second test point and the camera can be the distance between the second test point and the lens in the camera, the distance between the second test point and the geometric center in the camera, the distance between the second test point and the photosensitive element in the camera, or the distance between the second test point and other physical or virtual parts in the camera; no specific limitation is made here.

[0014] Furthermore, in addition to the first and fourth planes, the light transmitter can include other emitting surfaces, such as the seventh plane or other planes; correspondingly, in addition to the second and fifth planes, the light receiver can include other receiving planes, such as the eighth plane or other planes. Moreover, the other emitting surfaces in the light transmitter and the other receiving surfaces in the light receiver can also be connected by epipolar lines satisfying epipolar constraints, as described above, and light signals can be transmitted and received on the corresponding epipolar lines to achieve ranging of more different test points. Furthermore, since the different light signals used to measure different test points do not interfere with each other, the accuracy of ToF camera ranging can be further improved.

[0015] In one possible implementation of the first aspect of the present application, the light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane and the emitting surface of the second light source region is the fourth plane; the light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane and the receiving surface of the second pixel array region is the fifth plane.

[0016] Based on the above technical solution, the optical transmitter can be configured with multiple light source regions, and different emitting surfaces can be set on different light source regions to achieve the transmission of multiple different optical signals. Correspondingly, the optical receiver can also be configured with multiple pixel array regions, and different receiving surfaces can be set on different pixel array regions to achieve the reception of multiple different optical signals.

[0017] In one possible implementation of the first aspect of the embodiments of this application, the first optical signal and the second optical signal are orthogonal to each other.

[0018] Based on the above technical solution, the first optical signal used to measure the distance to the first test point and the second optical signal used to measure the distance to the second test point can be orthogonal to each other, that is, the coherence between the first and second optical signals is 0. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the interference between the first and second optical signals can be avoided due to the 0 coherence, further improving the accuracy of the ToF camera's ranging.

[0019] In one possible implementation of the first aspect of the embodiments of this application, both the first optical signal and the second optical signal are signals encoded by binary phase shift keying (BPSK).

[0020] Based on the above technical solution, both the first optical signal and the second optical signal can be obtained through BPSK encoding. That is, at least two original sequences are used to perform BPSK encoding to obtain the first and second optical signals respectively. Different original sequences can be used to make the first and second optical signals orthogonal. This provides a specific implementation method for the first and second optical signals, improving the feasibility of the solution.

[0021] Optionally, the first optical signal and the second optical signal can be signals obtained through other encoding methods, such as quadrature phase shift keying (QPSK), quadrature amplitude modulation (QAM), or other methods, which are not limited here.

[0022] In one possible implementation of the first aspect of the present application, the first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence, and the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0023] Based on the above technical solution, the first and second optical signals can be obtained by BPSK encoding using pseudo-random sequences. The number of sequences with a value of 0 in the first pseudo-random sequence used to generate the first optical signal is the same as the number of sequences with a value of 0 in the second pseudo-random sequence used to generate the second optical signal, ensuring zero interference between the generated first and second optical signals. That is, the first optical signal used for ranging the first test point and the second optical signal used for ranging the second test point do not interfere with each other, further improving the ranging accuracy of the ToF camera.

[0024] In one possible implementation of the first aspect of the embodiments of this application, the first optical signal is a signal obtained by BPSK encoding.

[0025] Based on the above technical solution, the first optical signal can be a signal obtained through BPSK encoding, that is, the first optical signal is obtained by BPSK encoding a certain original sequence. This provides a specific implementation method for the first optical signal, improving the feasibility of the solution.

[0026] In one possible implementation of the first aspect of the present application, the signal frequency of the first optical signal is a first frequency, the signal frequency of the second optical signal is a second frequency, and the first frequency is different from the second frequency.

[0027] Based on the above technical solution, the first and second optical signals can be emitted at different frequencies to achieve mutual orthogonality, resulting in zero coherence between them. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the different frequencies of the first and second optical signals prevent interference between them, further improving the accuracy of ToF camera ranging.

[0028] Furthermore, if the optical transmitter includes other emitting surfaces besides the first and fourth planes, such as a seventh plane or other planes, then, similar to the relationship between the first plane and the first epipolar line (or the relationship between the third epipolar line and the fourth plane), other epipolar lines also exist in these other emitting surfaces. Moreover, the optical transmitter can emit other optical signals that pass through these other epipolar lines. These other optical signals can use signal frequencies different from the first and second frequencies. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can set n different signal frequencies for the n optical signals, meaning that the signal frequencies of the n optical signals are all different. Alternatively, the other optical signals can also use either the first or second frequency. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can alternately set the n optical signals to the first and second frequencies, meaning that any adjacent optical signals among the n optical signals will have different signal frequencies, thus achieving a better anti-interference effect.

[0029] In one possible implementation of the first aspect of this application, during the distance measurement process, the light emitter in the camera is further used to emit a third light signal with a signal frequency of the second frequency to the first test point, wherein the third light signal is reflected by the first test point to form a third reflected light signal, and the third light signal intersects with the first epipolar line and the third reflected light signal intersects with the second epipolar line; the light receiver in the camera is further used to receive the third reflected light signal, wherein the first light signal, the first reflected light signal, the third light signal and the third reflected light signal are used to determine the distance between the first test point and the camera.

[0030] Based on the above technical solution, since ToF cameras generally use phase difference to determine distance, at a single frequency, the maximum ranging range of a ToF camera is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the point to be measured and the camera exceeds this maximum ranging range, the measured distance will be less accurate due to periodic aliasing. To avoid this defect, for the same point to be measured, light signals of different frequencies can be emitted separately, and the ranging distance can be calculated using the relationship between the first frequency and the second frequency, thereby extending the ranging distance of the ToF camera. Specifically, the accuracy of the ranging result for the first point to be measured can be improved by using a first light signal with a first frequency and a third light signal with a second frequency.

[0031] It should be noted that the intersection of the third optical signal with the first epipolar line can indicate that the transmission path of the third optical signal intersects the first epipolar line at one or more points, or it can indicate that the transmission path of the third optical signal passes through one or more points of the first epipolar line. Similarly, the intersection of the third reflected optical signal with the second epipolar line can indicate that the transmission path of the third reflected optical signal intersects the second epipolar line at one or more points, or it can indicate that the transmission path of the third reflected optical signal passes through one or more points of the second epipolar line. For example, when the third optical signal includes a beam of light (or multiple beams of light), the propagation path of the third optical signal passes through the first epipolar line and intersects the first epipolar line at one or more points. Subsequently, the third optical signal is reflected by the first test point to form a single beam (or multiple beams) of first reflected light signal, and the propagation path of the third reflected light signal passes through the second epipolar line and intersects the second epipolar line at one or more points.

[0032] In one possible implementation of the first aspect of this application, during the distance measurement process, the light emitter in the camera is further used to emit a fourth light signal with a signal frequency of the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal, and the fourth light signal intersects with the third polar line, and the fourth reflected light signal intersects with the fourth polar line. The light receiver in the camera is further used to receive the fourth light signal and the fourth reflected light signal at the second test point. The second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal are used to determine the distance between the second test point and the camera.

[0033] Based on the above technical solution, since ToF cameras generally use phase difference to determine distance, at a single frequency, the maximum ranging range of a ToF camera is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the point to be measured and the camera exceeds this maximum ranging range, the measured distance will be less accurate due to periodic aliasing. To avoid this defect, for the same point to be measured, light signals of different frequencies can be emitted separately, and the relationship between the first and second frequencies can be used for calculation to extend the ranging distance of the ToF camera. Specifically, the accuracy of the ranging result for the second point to be measured can be improved by using a second light signal with the second frequency and a fourth light signal with the first frequency.

[0034] It should be noted that the intersection of the fourth optical signal with the third pole line can indicate that the transmission path of the fourth optical signal intersects the third pole line at one or more points, or that the transmission path of the fourth optical signal passes through one or more points of the second pole line. Similarly, the intersection of the fourth reflected optical signal with the fourth pole line can indicate that the transmission path of the fourth reflected optical signal intersects the fourth pole line at one or more points, or that the transmission path of the fourth reflected optical signal passes through one or more points of the fourth pole line. For example, when the fourth optical signal includes a beam of light (or multiple beams of light), the propagation path of the fourth optical signal passes through the third pole line and intersects the third pole line at one or more points. Subsequently, the fourth optical signal is reflected by the second test point to form a single beam (or multiple beams) of fourth reflected optical signal, and the propagation path of the fourth reflected optical signal passes through the fourth pole line and intersects the fourth pole line at one or more points.

[0035] In one possible implementation of the first aspect of this application, the ranging function of the camera can be achieved by setting a controller within the camera. Specifically, the camera may further include a controller connected to the light transmitter and the light receiver respectively; during the ranging process, the controller is used to control the light transmitter to emit the first light signal; and the controller is also used to control the light receiver to receive the first light signal.

[0036] Based on the above technical solution, the camera can be equipped with controllers connected to the light transmitter and the light receiver respectively. The controllers control the light transmitter to emit light signals and control the light receiver to receive light signals, so that the light transmitter and the light receiver can measure the distance to the point to be measured based on the Time-of-Flight (ToF) principle under the control of the controllers.

[0037] Optionally, the controller is also used to control the light transmitter to emit the second light signal.

[0038] Optionally, the controller is also used to control the optical receiver to receive the second reflected light signal.

[0039] Optionally, the controller is also used to control the optical transmitter to emit the third optical signal.

[0040] Optionally, the controller is also used to control the optical receiver to receive the third reflected light signal.

[0041] Optionally, the controller is also used to control the optical transmitter to emit the fourth optical signal.

[0042] Optionally, the controller is also used to control the optical receiver to receive the fourth reflected light signal.

[0043] In one possible implementation of the first aspect of the present application, during the ranging process of the camera, the controller can specifically be used to sample the first reflected light signal to obtain a first sampling result; then, the controller determines a first phase difference between the first light signal and the first reflected light signal based on the first sampling result; further, the controller determines the distance between the first measurement point and the camera based on the first phase difference.

[0044] Based on the above technical solution, the controller can determine the distance between the first test point and the camera by the first light signal emitted by the light transmitter and the first reflected light signal received by the light receiver. Specifically, based on the ToF principle, the first phase difference can be obtained by solving the phase difference between the first light signal and the second reflected light signal, and then the distance between the first test point and the camera can be determined based on the first phase difference.

[0045] In one possible implementation of the first aspect of the present application, during the ranging process of the camera, the controller can specifically be used to sample the second reflected light signal to obtain a second sampling result; then, the controller determines a second phase difference between the second light signal and the second reflected light signal based on the second sampling result; further, the controller determines the distance between the second measurement point and the camera based on the second phase difference.

[0046] Based on the above technical solution, the controller can determine the distance between the second test point and the camera by the second light signal emitted by the light transmitter and the second reflected light signal received by the light receiver. Specifically, based on the ToF principle, the second phase difference can be obtained by solving the phase difference between the second light signal and the second reflected light signal, and then the distance between the second test point and the camera can be determined based on the second phase difference.

[0047] A second aspect of this application provides a time-of-flight (ToF) based ranging camera. When capturing photos or videos using optical imaging principles, this camera can measure the distance to an object in a frame of the photo or video. The object includes one or more measurement points, such as a first measurement point, a second measurement point, etc. Specifically, the camera includes a controller, and a light emitter and a light receiver connected to the controller. The origin of the light emitter is a first origin, and the emitting surface of the light emitter includes a first plane and a fourth plane. The origin of the light receiver is a second origin, and the receiving surface of the light receiver includes a second plane and a fifth plane.

[0048] During the ranging process of the camera, the light emitter is used to emit a first light signal towards the first test point under the control of the controller. The first light signal is reflected by the first test point to form a first reflected light signal, and the first light signal intersects with the first epipolar line, and the first reflected light signal intersects with the second epipolar line. The plane containing the first test point, the first origin, and the second origin is a third plane, and the third plane intersects with the first plane at the first epipolar line, and the third plane intersects with the second plane at the second epipolar line.

[0049] Furthermore, during the distance measurement process, the light emitter in the camera is also used to emit a second light signal towards the second test point under the control of the controller. The second light signal is reflected by the second test point to form a second reflected light signal, and the second light signal intersects with the third epipolar line, and the second reflected light signal intersects with the fourth epipolar line. The second test point is different from the first test point. The plane containing the first origin and the second origin of the second test point is the sixth plane, and the sixth plane intersects with the fourth plane at the third epipolar line, and the sixth plane intersects with the fifth plane at the fourth epipolar line. The third plane and the sixth plane are not coplanar.

[0050] The light receiver in the camera is used to receive the first reflected light signal and the second light signal under the control of the controller;

[0051] The controller in the camera is used to determine the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and to determine the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

[0052] Based on the above technical solution, under the control of the controller, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point, and the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Under the control of the controller, the second reflected light signal received by the optical receiver is formed by the reflection of the second light signal emitted by the optical transmitter at the second test point, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Since the third plane and the sixth plane are not coplanar, that is, the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line, the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. That is, the different light reflection signals received by the optical receiver through different epipolar lines do not interfere with each other, reducing the influence of diffuse reflection of multiple light signals emitted by the optical transmitter at the test point on the camera ranging results, and improving the accuracy of ToF camera ranging.

[0053] It should be noted that the intersection of the first optical signal and the first epipolar line can indicate that the transmission path of the first optical signal intersects the first epipolar line at one or more points, or it can indicate that the transmission path of the first optical signal passes through one or more points of the first epipolar line. Similarly, the intersection of the first reflected optical signal and the second epipolar line can indicate that the transmission path of the first reflected optical signal intersects the second epipolar line at one or more points, or it can indicate that the transmission path of the first reflected optical signal passes through one or more points of the second epipolar line. For example, when the first optical signal includes a beam of light (or multiple beams of light), the propagation path of the first optical signal passes through the first epipolar line and intersects the first epipolar line at one or more points. Subsequently, the first optical signal is reflected by the first test point to form a single beam (or multiple beams) of first reflected light signal, and the propagation path of the first reflected light signal passes through the second epipolar line and intersects the second epipolar line at one or more points. Similarly, the intersection of the second optical signal with the third pole line can indicate that the transmission path of the second optical signal intersects the third pole line at one or more points, or that the transmission path of the second optical signal passes through one or more points of the second pole line. Likewise, the intersection of the second reflected optical signal with the fourth pole line can indicate that the transmission path of the second reflected optical signal intersects the fourth pole line at one or more points, or that the transmission path of the second reflected optical signal passes through one or more points of the fourth pole line. For example, when the second optical signal includes a beam of light (or multiple beams of light), the propagation path of the second optical signal passes through the third pole line and intersects the third pole line at one or more points. Subsequently, the second optical signal is reflected by the second test point to form a single beam (or multiple beams) of second reflected optical signal, and the propagation path of the second reflected optical signal passes through the fourth pole line and intersects the fourth pole line at one or more points.

[0054] It should be noted that the distance between the first test point and the camera can be the distance between the first test point and the lens in the camera, the distance between the first test point and the geometric center in the camera, the distance between the first test point and the photosensitive element in the camera, or the distance between the first test point and other physical or virtual parts in the camera; no specific limitation is made here. Similarly, the distance between the second test point and the camera can be the distance between the second test point and the lens in the camera, the distance between the second test point and the geometric center in the camera, the distance between the second test point and the photosensitive element in the camera, or the distance between the second test point and other physical or virtual parts in the camera; no specific limitation is made here.

[0055] Furthermore, in addition to the first and fourth planes, the optical transmitter can include other emitting surfaces, such as the seventh plane or other planes; correspondingly, in addition to the second and fifth planes, the optical receiver can include other receiving planes, such as the eighth plane or other planes. Moreover, the other emitting surfaces in the optical transmitter and the other receiving surfaces in the optical receiver can be connected by epipolar lines satisfying epipolar constraints, as described above. The controller controls the transmission and reception of optical signals along the corresponding epipolar lines to achieve ranging of more different test points. Furthermore, since the different optical signals used to measure different test points do not interfere with each other, the accuracy of ToF camera ranging can be further improved.

[0056] In one possible implementation of the second aspect of the present application, the light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane and the emitting surface of the second light source region is the fourth plane; the light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane and the receiving surface of the second pixel array region is the fifth plane.

[0057] Based on the above technical solution, the optical transmitter can be configured with multiple light source regions, and different emitting surfaces can be set on different light source regions to achieve the transmission of multiple different optical signals. Correspondingly, the optical receiver can also be configured with multiple pixel array regions, and different receiving surfaces can be set on different pixel array regions to achieve the reception of multiple different optical signals.

[0058] In one possible implementation of the second aspect of the present application, the first optical signal and the second optical signal are orthogonal to each other.

[0059] Based on the above technical solution, the first optical signal used to measure the distance to the first test point and the second optical signal used to measure the distance to the second test point can be orthogonal to each other, that is, the coherence between the first and second optical signals is 0. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the interference between the first and second optical signals can be avoided due to the 0 coherence, further improving the accuracy of the ToF camera's ranging.

[0060] In one possible implementation of the second aspect of the present application, both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0061] Based on the above technical solution, both the first optical signal and the second optical signal can be obtained through BPSK encoding. That is, at least two original sequences are used to perform BPSK encoding to obtain the first and second optical signals respectively. Different original sequences can be used to make the first and second optical signals orthogonal. This provides a specific implementation method for the first and second optical signals, improving the feasibility of the solution.

[0062] Optionally, the first optical signal and the second optical signal can be signals obtained through other encoding methods, such as quadrature phase shift keying (QPSK), quadrature amplitude modulation (QAM), or other methods, which are not limited here.

[0063] In one possible implementation of the second aspect of the present application, the first optical signal is a signal obtained by encoding a first pseudo-random sequence using BPSK, the second optical signal is a signal obtained by encoding a second pseudo-random sequence using BPSK, and the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0064] Based on the above technical solution, the first and second optical signals can be obtained by BPSK encoding using pseudo-random sequences. The number of sequences with a value of 0 in the first pseudo-random sequence used to generate the first optical signal is the same as the number of sequences with a value of 0 in the second pseudo-random sequence used to generate the second optical signal, ensuring zero interference between the generated first and second optical signals. That is, the first optical signal used for ranging the first test point and the second optical signal used for ranging the second test point do not interfere with each other, further improving the ranging accuracy of the ToF camera.

[0065] In one possible implementation of the second aspect of the embodiments of this application, the first optical signal is a signal obtained by BPSK encoding.

[0066] Based on the above technical solution, the first optical signal can be a signal obtained through BPSK encoding, that is, the first optical signal is obtained by BPSK encoding a certain original sequence. This provides a specific implementation method for the first optical signal, improving the feasibility of the solution.

[0067] In one possible implementation of the second aspect of the present application, the signal frequency of the first optical signal is a first frequency, the signal frequency of the second optical signal is a second frequency, and the first frequency is different from the second frequency.

[0068] Based on the above technical solution, the first and second optical signals can be emitted at different frequencies to achieve mutual orthogonality, resulting in zero coherence between them. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the different frequencies of the first and second optical signals prevent interference between them, further improving the accuracy of ToF camera ranging.

[0069] Furthermore, if the optical transmitter includes other emitting surfaces besides the first and fourth planes, such as a seventh plane or other planes, then, similar to the relationship between the first plane and the first epipolar line (or the relationship between the third epipolar line and the fourth plane), other epipolar lines also exist in these other emitting surfaces. Moreover, the optical transmitter can emit other optical signals that pass through these other epipolar lines. These other optical signals can use signal frequencies different from the first and second frequencies. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can set n different signal frequencies for the n optical signals, meaning that the signal frequencies of the n optical signals are all different. Alternatively, the other optical signals can also use either the first or second frequency. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can alternately set the n optical signals to the first and second frequencies, meaning that any adjacent optical signals among the n optical signals will have different signal frequencies, thus achieving a better anti-interference effect.

[0070] In one possible implementation of the second aspect of this application, during the distance measurement process, the light emitter in the camera is further configured, under the control of the controller, to emit a third light signal with a signal frequency of the second frequency to the first test point, wherein the third light signal is reflected by the first test point to form a third reflected light signal, and the third light signal intersects with the first epipolar line and the third reflected light signal intersects with the second epipolar line; the light receiver in the camera is further configured, under the control of the controller, to receive the third reflected light signal; the controller in the camera is further configured to determine the distance between the first test point and the camera based on the first light signal, the first reflected light signal, the third light signal and the third reflected light signal.

[0071] Based on the above technical solution, since ToF cameras generally use phase difference to determine distance, at a single frequency, the maximum ranging range of a ToF camera is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the point to be measured and the camera exceeds this maximum ranging range, the measured distance will be less accurate due to periodic aliasing. To avoid this defect, the controller can control the transmission and reception of light signals of different frequencies for the same point to be measured, and use the relationship between the first frequency and the second frequency for calculation to extend the ranging distance of the ToF camera. Specifically, the accuracy of the ranging result for the first point to be measured can be improved by using a first light signal with the first frequency and a third light signal with the second frequency.

[0072] It should be noted that the intersection of the third optical signal with the first epipolar line can indicate that the transmission path of the third optical signal intersects the first epipolar line at one or more points, or it can indicate that the transmission path of the third optical signal passes through one or more points of the first epipolar line. Similarly, the intersection of the third reflected optical signal with the second epipolar line can indicate that the transmission path of the third reflected optical signal intersects the second epipolar line at one or more points, or it can indicate that the transmission path of the third reflected optical signal passes through one or more points of the second epipolar line. For example, when the third optical signal includes a beam of light (or multiple beams of light), the propagation path of the third optical signal passes through the first epipolar line and intersects the first epipolar line at one or more points. Subsequently, the third optical signal is reflected by the first test point to form a single beam (or multiple beams) of first reflected light signal, and the propagation path of the third reflected light signal passes through the second epipolar line and intersects the second epipolar line at one or more points.

[0073] In one possible implementation of the second aspect of this application, during the distance measurement process, the light emitter in the camera is further configured, under the control of the controller, to emit a fourth light signal with a signal frequency of the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal, and the fourth light signal intersects with the third polar line, and the fourth reflected light signal intersects with the fourth polar line. The light receiver in the camera is further configured, under the control of the controller, to receive the fourth reflected light signal of the fourth light signal at the second test point. At this time, the controller in the camera is further configured to determine the distance between the second test point and the camera based on the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal.

[0074] Based on the above technical solution, since ToF cameras generally use phase difference to determine distance, at a single frequency, the maximum ranging range of a ToF camera is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the point to be measured and the camera exceeds this maximum ranging range, the measured distance will be less accurate due to periodic aliasing. To avoid this defect, for the same point to be measured, light signals of different frequencies can be emitted separately, and the relationship between the first and second frequencies can be used for calculation to extend the ranging distance of the ToF camera. Specifically, the accuracy of the ranging result for the second point to be measured can be improved by using a second light signal with the second frequency and a fourth light signal with the first frequency.

[0075] It should be noted that the intersection of the fourth optical signal with the third pole line can indicate that the transmission path of the fourth optical signal intersects the third pole line at one or more points, or that the transmission path of the fourth optical signal passes through one or more points of the second pole line. Similarly, the intersection of the fourth reflected optical signal with the fourth pole line can indicate that the transmission path of the fourth reflected optical signal intersects the fourth pole line at one or more points, or that the transmission path of the fourth reflected optical signal passes through one or more points of the fourth pole line. For example, when the fourth optical signal includes a beam of light (or multiple beams of light), the propagation path of the fourth optical signal passes through the third pole line and intersects the third pole line at one or more points. Subsequently, the fourth optical signal is reflected by the second test point to form a single beam (or multiple beams) of fourth reflected optical signal, and the propagation path of the fourth reflected optical signal passes through the fourth pole line and intersects the fourth pole line at one or more points.

[0076] In one possible implementation of the second aspect of the present application, during the ranging process of the camera, the controller in the camera is specifically used to sample the first reflected light signal to obtain a first sampling result; then, the controller determines a first phase difference between the first light signal and the first reflected light signal based on the first sampling result; further, the controller determines the distance between the first measurement point and the camera based on the first phase difference.

[0077] Based on the above technical solution, the camera can control the light transmitter to emit light signals and the light receiver to receive light signals through the controller. This allows the light transmitter and receiver to measure the distance to the target point based on the Time-of-Flight (ToF) principle under the controller's control. Specifically, the controller can determine the distance between the first target point and the camera using the first light signal emitted by the light transmitter and the first reflected light signal received by the light receiver. This can be achieved by calculating the first phase difference between the first light signal and the second reflected light signal based on the ToF principle, and then further determining the distance between the first target point and the camera based on this first phase difference.

[0078] In one possible implementation of the second aspect of the present application, during the ranging process of the camera, the controller in the camera is specifically used to sample the second reflected light signal to obtain a second sampling result; then, the controller determines a second phase difference between the second light signal and the second reflected light signal based on the second sampling result; further, the controller determines the distance between the second measurement point and the camera based on the second phase difference.

[0079] Based on the above technical solution, the controller can determine the distance between the second test point and the camera by the second light signal emitted by the light transmitter and the second reflected light signal received by the light receiver. Specifically, based on the ToF principle, the second phase difference can be obtained by solving the phase difference between the second light signal and the second reflected light signal, and then the distance between the second test point and the camera can be determined based on the second phase difference.

[0080] A third aspect of this application provides a time-of-flight (ToF) based ranging camera. When capturing photos or videos using optical imaging principles, this camera can measure the distance to an object in a frame of the photo or video. The object includes one or more measurement points, such as a first measurement point, a second measurement point, etc. Specifically, the camera includes a controller and a light receiver connected to the controller.

[0081] During the distance measurement process, the light receiver in the camera, under the control of the controller, receives a first reflected light signal. The first reflected light signal is formed by the reflection of a first light signal emitted by the light emitter at a first point to be measured, and the first light signal intersects with a first epipolar line, while the first reflected light signal intersects with a second epipolar line. The origin of the coordinate system of the light emitter is the first origin, and the emitting surface of the light emitter includes a first plane and a fourth plane. The origin of the coordinate system of the light receiver is the second origin, and the receiving surface of the light receiver includes a second plane and a fifth plane. The plane containing the first point to be measured, the first origin, and the second origin is a third plane, and the third plane intersects with the first plane at the first epipolar line, and the third plane intersects with the second plane at the second epipolar line.

[0082] Furthermore, during the camera's ranging process, the light receiver in the camera is also used, under the control of the controller, to receive a second reflected light signal. This second reflected light signal is formed by the reflection of a second light signal emitted by the light emitter at the second test point, and the second light signal intersects the third epipolar line, while the second reflected light signal intersects the fourth epipolar line. The controller in the camera is also used to determine the distance between the second test point and the camera based on the second light signal and the second reflected light signal. Specifically, the second test point on the object being measured is different from the first test point. The plane containing the first origin and the second origin is a sixth plane, and this sixth plane intersects the fourth plane at the third epipolar line, and the sixth plane intersects the fifth plane at the fourth epipolar line; the third plane and the sixth plane are not coplanar.

[0083] In addition, the controller in the camera is used to determine the distance between the first test point and the camera based on the first light signal and the first reflected light signal.

[0084] Based on the above technical solution, under the control of the controller, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point, and the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Under the control of the controller, the second reflected light signal received by the optical receiver is formed by the reflection of the second light signal emitted by the optical transmitter at the second test point, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Since the third plane and the sixth plane are not coplanar, that is, the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line, the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. That is, the different light reflection signals received by the optical receiver through different epipolar lines do not interfere with each other, reducing the influence of diffuse reflection of multiple light signals emitted by the optical transmitter at the test point on the camera ranging results, and improving the accuracy of ToF camera ranging.

[0085] It should be noted that the intersection of the first optical signal and the first epipolar line can indicate that the transmission path of the first optical signal intersects the first epipolar line at one or more points, or it can indicate that the transmission path of the first optical signal passes through one or more points of the first epipolar line. Similarly, the intersection of the first reflected optical signal and the second epipolar line can indicate that the transmission path of the first reflected optical signal intersects the second epipolar line at one or more points, or it can indicate that the transmission path of the first reflected optical signal passes through one or more points of the second epipolar line. For example, when the first optical signal includes a beam of light (or multiple beams of light), the propagation path of the first optical signal passes through the first epipolar line and intersects the first epipolar line at one or more points. Subsequently, the first optical signal is reflected by the first test point to form a single beam (or multiple beams) of first reflected light signal, and the propagation path of the first reflected light signal passes through the second epipolar line and intersects the second epipolar line at one or more points. Similarly, the intersection of the second optical signal with the third pole line can indicate that the transmission path of the second optical signal intersects the third pole line at one or more points, or that the transmission path of the second optical signal passes through one or more points of the second pole line. Likewise, the intersection of the second reflected optical signal with the fourth pole line can indicate that the transmission path of the second reflected optical signal intersects the fourth pole line at one or more points, or that the transmission path of the second reflected optical signal passes through one or more points of the fourth pole line. For example, when the second optical signal includes a beam of light (or multiple beams of light), the propagation path of the second optical signal passes through the third pole line and intersects the third pole line at one or more points. Subsequently, the second optical signal is reflected by the second test point to form a single beam (or multiple beams) of second reflected optical signal, and the propagation path of the second reflected optical signal passes through the fourth pole line and intersects the fourth pole line at one or more points.

[0086] It should be noted that the distance between the first test point and the camera can be the distance between the first test point and the lens in the camera, the distance between the first test point and the geometric center in the camera, the distance between the first test point and the photosensitive element in the camera, or the distance between the first test point and other physical or virtual parts in the camera; no specific limitation is made here. Similarly, the distance between the second test point and the camera can be the distance between the second test point and the lens in the camera, the distance between the second test point and the geometric center in the camera, the distance between the second test point and the photosensitive element in the camera, or the distance between the second test point and other physical or virtual parts in the camera; no specific limitation is made here.

[0087] Furthermore, in addition to the first and fourth planes, the optical transmitter can include other emitting surfaces, such as the seventh plane or other planes; correspondingly, in addition to the second and fifth planes, the optical receiver can include other receiving planes, such as the eighth plane or other planes. Moreover, the other emitting surfaces in the optical transmitter and the other receiving surfaces in the optical receiver can be connected by epipolar lines satisfying epipolar constraints, as described above. The controller controls the transmission and reception of optical signals along the corresponding epipolar lines to achieve ranging of more different test points. Furthermore, since the different optical signals used to measure different test points do not interfere with each other, the accuracy of ToF camera ranging can be further improved.

[0088] In one possible implementation of the third aspect of this application, the light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane and the emitting surface of the second light source region is the fourth plane; the light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane and the receiving surface of the second pixel array region is the fifth plane.

[0089] Based on the above technical solution, the optical transmitter can be configured with multiple light source regions, and different emitting surfaces can be set on different light source regions to achieve the transmission of multiple different optical signals. Correspondingly, the optical receiver can also be configured with multiple pixel array regions, and different receiving surfaces can be set on different pixel array regions to achieve the reception of multiple different optical signals.

[0090] In one possible implementation of the third aspect of the present application, the first optical signal and the second optical signal are orthogonal to each other.

[0091] Based on the above technical solution, the first optical signal used to measure the distance to the first test point and the second optical signal used to measure the distance to the second test point can be orthogonal to each other, that is, the coherence between the first and second optical signals is 0. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the interference between the first and second optical signals can be avoided due to the 0 coherence, further improving the accuracy of the ToF camera's ranging.

[0092] In one possible implementation of the third aspect of the present application, both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0093] Based on the above technical solution, both the first optical signal and the second optical signal can be obtained through BPSK encoding. That is, at least two original sequences are used to perform BPSK encoding to obtain the first and second optical signals respectively. Different original sequences can be used to make the first and second optical signals orthogonal. This provides a specific implementation method for the first and second optical signals, improving the feasibility of the solution.

[0094] Optionally, the first optical signal and the second optical signal can be signals obtained through other encoding methods, such as quadrature phase shift keying (QPSK), quadrature amplitude modulation (QAM), or other methods, which are not limited here.

[0095] In one possible implementation of the third aspect of this application, the first optical signal is a signal obtained by encoding a first pseudo-random sequence using BPSK, the second optical signal is a signal obtained by encoding a second pseudo-random sequence using BPSK, and the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0096] Based on the above technical solution, the first and second optical signals can be obtained by BPSK encoding using pseudo-random sequences. The number of sequences with a value of 0 in the first pseudo-random sequence used to generate the first optical signal is the same as the number of sequences with a value of 0 in the second pseudo-random sequence used to generate the second optical signal, ensuring zero interference between the generated first and second optical signals. That is, the first optical signal used for ranging the first test point and the second optical signal used for ranging the second test point do not interfere with each other, further improving the ranging accuracy of the ToF camera.

[0097] In one possible implementation of the third aspect of the present application, the first optical signal is a signal obtained by BPSK encoding.

[0098] Based on the above technical solution, the first optical signal can be a signal obtained through BPSK encoding, that is, the first optical signal is obtained by BPSK encoding a certain original sequence. This provides a specific implementation method for the first optical signal, improving the feasibility of the solution.

[0099] In one possible implementation of the third aspect of the present application, the signal frequency of the first optical signal is a first frequency, the signal frequency of the second optical signal is a second frequency, and the first frequency is different from the second frequency.

[0100] Based on the above technical solution, the first and second optical signals can be emitted at different frequencies to achieve mutual orthogonality, resulting in zero coherence between them. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the different frequencies of the first and second optical signals prevent interference between them, further improving the accuracy of ToF camera ranging.

[0101] Furthermore, if the optical transmitter includes other emitting surfaces besides the first and fourth planes, such as a seventh plane or other planes, then, similar to the relationship between the first plane and the first epipolar line (or the relationship between the third epipolar line and the fourth plane), other epipolar lines also exist in these other emitting surfaces. Moreover, the optical transmitter can emit other optical signals that pass through these other epipolar lines. These other optical signals can use signal frequencies different from the first and second frequencies. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can set n different signal frequencies for the n optical signals, meaning that the signal frequencies of the n optical signals are all different. Alternatively, the other optical signals can also use either the first or second frequency. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can alternately set the n optical signals to the first and second frequencies, meaning that any adjacent optical signals among the n optical signals will have different signal frequencies, thus achieving a better anti-interference effect.

[0102] In one possible implementation of the third aspect of this application, during the distance measurement process, the light receiver in the camera is further configured to receive a third reflected light signal under the control of the controller. The third reflected light signal is formed by the reflection of the third light signal emitted by the light transmitter after passing through the first test point, and the third light signal intersects with the first polar line and the third reflected light signal intersects with the second polar line; wherein, the frequency of the third light signal is the second frequency; at this time, the controller in the camera is further configured to determine the distance between the first test point and the camera based on the first light signal, the first reflected light signal, the third light signal and the third reflected light signal.

[0103] Based on the above technical solution, since ToF cameras generally use phase difference to determine distance, at a single frequency, the maximum ranging range of a ToF camera is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the point to be measured and the camera exceeds this maximum ranging range, the measured distance will be less accurate due to periodic aliasing. To avoid this defect, the controller can control the transmission and reception of light signals of different frequencies for the same point to be measured, and use the relationship between the first frequency and the second frequency for calculation to extend the ranging distance of the ToF camera. Specifically, the accuracy of the ranging result for the first point to be measured can be improved by using a first light signal with the first frequency and a third light signal with the second frequency.

[0104] It should be noted that the intersection of the third optical signal with the first epipolar line can indicate that the transmission path of the third optical signal intersects the first epipolar line at one or more points, or it can indicate that the transmission path of the third optical signal passes through one or more points of the first epipolar line. Similarly, the intersection of the third reflected optical signal with the second epipolar line can indicate that the transmission path of the third reflected optical signal intersects the second epipolar line at one or more points, or it can indicate that the transmission path of the third reflected optical signal passes through one or more points of the second epipolar line. For example, when the third optical signal includes a beam of light (or multiple beams of light), the propagation path of the third optical signal passes through the first epipolar line and intersects the first epipolar line at one or more points. Subsequently, the third optical signal is reflected by the first test point to form a single beam (or multiple beams) of first reflected light signal, and the propagation path of the third reflected light signal passes through the second epipolar line and intersects the second epipolar line at one or more points.

[0105] In one possible implementation of the third aspect of this application, during the distance measurement process, the light receiver in the camera is further configured to receive a fourth reflected light signal under the control of the controller. The fourth reflected light signal is formed by the reflection of the fourth light signal emitted by the light emitter after the reflection at the second point to be measured, and the fourth light signal intersects with the third polar line, and the fourth reflected light signal intersects with the fourth polar line; wherein, the signal frequency of the fourth light signal is the first frequency; at this time, the controller in the camera is further configured to determine the distance between the second point to be measured and the camera based on the second light signal, the second reflected light signal, the fourth light signal and the fourth reflected light signal.

[0106] Based on the above technical solution, since ToF cameras generally use phase difference to determine distance, at a single frequency, the maximum ranging range of a ToF camera is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the point to be measured and the camera exceeds this maximum ranging range, the measured distance will be less accurate due to periodic aliasing. To avoid this defect, for the same point to be measured, light signals of different frequencies can be emitted separately, and the relationship between the first and second frequencies can be used for calculation to extend the ranging distance of the ToF camera. Specifically, the accuracy of the ranging result for the second point to be measured can be improved by using a second light signal with the second frequency and a fourth light signal with the first frequency.

[0107] It should be noted that the intersection of the fourth optical signal with the third pole line can indicate that the transmission path of the fourth optical signal intersects the third pole line at one or more points, or that the transmission path of the fourth optical signal passes through one or more points of the second pole line. Similarly, the intersection of the fourth reflected optical signal with the fourth pole line can indicate that the transmission path of the fourth reflected optical signal intersects the fourth pole line at one or more points, or that the transmission path of the fourth reflected optical signal passes through one or more points of the fourth pole line. For example, when the fourth optical signal includes a beam of light (or multiple beams of light), the propagation path of the fourth optical signal passes through the third pole line and intersects the third pole line at one or more points. Subsequently, the fourth optical signal is reflected by the second test point to form a single beam (or multiple beams) of fourth reflected optical signal, and the propagation path of the fourth reflected optical signal passes through the fourth pole line and intersects the fourth pole line at one or more points.

[0108] In one possible implementation of the third aspect of this application, during the ranging process of the camera, the controller in the camera is specifically used to sample the first reflected light signal to obtain a first sampling result; then, the controller determines a first phase difference between the first light signal and the first reflected light signal based on the first sampling result; further, the controller determines the distance between the first measurement point and the camera based on the first phase difference.

[0109] Based on the above technical solution, the camera can control the optical receiver to receive light signals through the controller, so that the optical receiver, under the control of the controller, can measure the distance to the point under test based on the Time-of-Flight (ToF) principle. Specifically, the controller can determine the distance between the first point under test and the camera by using the first light signal emitted by the light transmitter and the first reflected light signal received by the light receiver. Specifically, based on the ToF principle, the first phase difference can be obtained by solving the phase difference between the first light signal and the second reflected light signal, and then the distance between the first point under test and the camera can be further determined based on the first phase difference.

[0110] In one possible implementation of the third aspect of this application, during the ranging process of the camera, the controller in the camera is specifically used to sample the second reflected light signal to obtain a second sampling result; then, the controller determines a second phase difference between the second light signal and the second reflected light signal based on the second sampling result; further, the controller determines the distance between the second measurement point and the camera based on the second phase difference.

[0111] Based on the above technical solution, the controller can determine the distance between the second test point and the camera by the second light signal emitted by the light transmitter and the second reflected light signal received by the light receiver. Specifically, based on the ToF principle, the second phase difference can be obtained by solving the phase difference between the second light signal and the second reflected light signal, and then the distance between the second test point and the camera can be determined based on the second phase difference.

[0112] A fourth aspect of this application provides a time-of-flight (ToF) ranging method applied to a camera. When the camera captures a photograph or video using optical imaging principles, it can measure the distance to an object in a frame of the photograph or video. The object includes one or more measurement points, such as a first measurement point, a second measurement point, etc. Specifically, the camera includes a light emitter and a light receiver; the origin of the light emitter is a first origin, and the emitting surface of the light emitter includes a first plane and a fourth plane; the origin of the light receiver is a second origin, and the receiving surface of the light receiver includes a second plane and a fifth plane; the method includes:

[0113] The light emitter emits a first light signal to a first test point and a second light signal to a second test point. The first light signal is reflected by the first test point to form a first reflected light signal, which intersects with a first polar line and a second polar line. The second light signal is reflected by the second test point to form a second reflected light signal, which intersects with a third polar line and a fourth polar line. The plane containing the first test point, its first origin, and its second origin is a third plane, which intersects with the first plane at the first polar line and with the second plane at the second polar line. The second test point is different from the first test point. The plane containing the second test point, its first origin, and its second origin is a sixth plane, which intersects with the fourth plane at the third polar line and with the fifth plane at the fourth polar line. The third and sixth planes are not coplanar.

[0114] The optical receiver receives the first reflected light signal and the second reflected light signal, wherein the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera.

[0115] Based on the above technical solution, during the ranging process, the first reflected light signal received by the camera's light receiver is formed by the reflection of the first light signal emitted by the light emitter at the first test point, and this first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. The second reflected light signal received by the camera's light receiver is formed by the reflection of the second light signal emitted by the light emitter at the second test point, and this second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Since the third plane and the sixth plane are not coplanar (i.e., the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line), the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. In other words, this ensures that the different reflected light signals received by the light receiver through different epipolar lines do not interfere with each other, reducing the impact of diffuse reflection of multiple light signals emitted by the light emitter at the test point on the camera's ranging results, and improving the accuracy of ToF camera ranging.

[0116] In one possible implementation of the fourth aspect of the embodiments of this application,

[0117] The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane;

[0118] The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

[0119] In one possible implementation of the fourth aspect of the present application, the first optical signal and the second optical signal are orthogonal to each other.

[0120] In one possible implementation of the fourth aspect of the embodiments of this application,

[0121] Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0122] In one possible implementation of the fourth aspect of the embodiments of this application,

[0123] The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0124] In one possible implementation of the fourth aspect of the present application, the first optical signal is a signal obtained by BPSK encoding.

[0125] In one possible implementation of the fourth aspect of the embodiments of this application,

[0126] The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

[0127] In one possible implementation of the fourth aspect of this application, the method further includes:

[0128] The light emitter emits a third light signal with the second frequency to the first test point. The third light signal is reflected by the first test point to form a third reflected light signal. The third light signal intersects with the first polar line, and the third reflected light signal intersects with the second polar line.

[0129] The optical receiver receives the third reflected light signal, wherein the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal are used to determine the distance between the first test point and the camera.

[0130] In one possible implementation of the fourth aspect of this application, the method further includes:

[0131] The light emitter emits a fourth light signal with the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal. The fourth light signal intersects with the third pole line, and the fourth reflected light signal intersects with the fourth pole line.

[0132] The optical receiver receives the fourth reflected light signal, wherein the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal are used to determine the distance between the second test point and the camera.

[0133] In one possible implementation of the fourth aspect of this application, the camera further includes a controller connected to the light transmitter and the light receiver respectively;

[0134] The optical transmitter emits a first optical signal toward the first point to be tested, including:

[0135] The controller controls the light transmitter to emit the first light signal toward the first point under test;

[0136] The optical receiver receives the first reflected light signal, including:

[0137] The controller controls the optical receiver to receive the first reflected light signal.

[0138] In one possible implementation of the fourth aspect of this application, the method further includes:

[0139] The controller samples the first reflected light signal to obtain the first sampling result;

[0140] The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result;

[0141] The controller determines the distance between the first test point and the camera based on the first phase difference.

[0142] It should be noted that the specific implementation process and beneficial effects of the methods described in the fourth aspect and any of their possible implementations can be referred to the description of the first aspect and its possible implementations mentioned above, and will not be repeated here.

[0143] A fifth aspect of this application provides a time-of-flight (ToF) ranging method applied to a controller, wherein the controller is included in a camera. When the camera captures a photograph or video using optical imaging principles, it can measure the distance to a target object in a frame of the photograph or video. The target object includes one or more measurement points, such as a first measurement point, a second measurement point, etc. Specifically, the camera also includes a light emitter and a light receiver respectively connected to the controller; the origin of the light emitter is a first origin and the emitting surface of the light emitter includes a first plane; the origin of the light receiver is a second origin and the receiving surface of the light receiver includes a second plane; during the ranging process performed by the camera, the method executed by the controller includes:

[0144] The controller controls the light emitter to emit a first light signal and a second light signal towards a first test point. The first light signal is reflected by the first test point to form a first reflected light signal, which intersects with a first epipolar line, and the first reflected light signal intersects with a second epipolar line. The second light signal is reflected by the second test point to form a second reflected light signal, which intersects with a third epipolar line, and the second reflected light signal intersects with a fourth epipolar line. The first test point, the plane containing the first origin and the second origin is a third plane, which intersects the first plane at the first epipolar line, and the third plane intersects the second plane at the second epipolar line. The second test point is different from the first test point. The second test point, the plane containing the first origin and the second origin is a sixth plane, which intersects the fourth plane at the third epipolar line, and the sixth plane intersects the fifth plane at the fourth epipolar line. The third plane and the sixth plane are not coplanar.

[0145] Subsequently, the controller controls the light receiver to receive the first reflected light signal and the second reflected light signal, determines the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and determines the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

[0146] Based on the above technical solution, under the control of the controller, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point, and the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Under the control of the controller, the second reflected light signal received by the optical receiver is formed by the reflection of the second light signal emitted by the optical transmitter at the second test point, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Since the third plane and the sixth plane are not coplanar, that is, the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line, the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. That is, the different light reflection signals received by the optical receiver through different epipolar lines do not interfere with each other, reducing the influence of diffuse reflection of multiple light signals emitted by the optical transmitter at the test point on the camera ranging results, and improving the accuracy of ToF camera ranging.

[0147] In one possible implementation of the fifth aspect of the embodiments of this application,

[0148] The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane;

[0149] The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

[0150] In one possible implementation of the fifth aspect of this application, the first optical signal and the second optical signal are orthogonal to each other.

[0151] In one possible implementation of the fifth aspect of the embodiments of this application,

[0152] Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0153] In one possible implementation of the fifth aspect of the embodiments of this application,

[0154] The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0155] In one possible implementation of the fifth aspect of this application, the first optical signal is a signal obtained by BPSK encoding.

[0156] In one possible implementation of the fifth aspect of the embodiments of this application,

[0157] The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

[0158] In one possible implementation of the fifth aspect of this application, the method further includes:

[0159] The controller controls the light emitter to emit a third light signal with the second frequency to the first test point. The third light signal is reflected by the first test point to form a third reflected light signal, and the third light signal intersects with the first polar line and the third reflected light signal intersects with the second polar line.

[0160] The controller controls the optical receiver to receive the third reflected light signal;

[0161] The controller determines the distance between the first measurement point and the camera based on the first light signal and the first reflected light signal, including:

[0162] The controller determines the distance between the first test point and the camera based on the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal.

[0163] In one possible implementation of the fifth aspect of this application, the method further includes:

[0164] The controller controls the light emitter to emit a fourth light signal with the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal, and the fourth light signal intersects with the third pole line, and the fourth reflected light signal intersects with the fourth pole line.

[0165] The controller controls the optical receiver to receive the four reflected light signals;

[0166] The controller determines the distance between the second measurement point and the camera based on the second optical signal and the second reflected optical signal, including:

[0167] The controller determines the distance between the second test point and the camera based on the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal.

[0168] In one possible implementation of the fifth aspect of this application, the controller determines the distance between the first test point and the camera based on the first optical signal and the first reflected optical signal, including:

[0169] The controller samples the first reflected light signal to obtain the first sampling result;

[0170] The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result;

[0171] The controller determines the distance between the first test point and the camera based on the first phase difference.

[0172] In one possible implementation of the fifth aspect of this application, the controller determines the distance between the second test point and the camera based on the second optical signal and the second reflected optical signal, including:

[0173] The controller samples the second reflected light signal to obtain a second sampling result;

[0174] The controller determines the second phase difference between the second optical signal and the second reflected optical signal based on the second sampling result;

[0175] The controller determines the distance between the second test point and the camera based on the second phase difference.

[0176] It should be noted that the specific implementation process and beneficial effects of the methods described in the fifth aspect and any of their possible implementations can be found in the description of the second aspect and its possible implementations mentioned above, and will not be repeated here.

[0177] A sixth aspect of this application provides a time-of-flight (ToF) based ranging camera. The method is applied to a controller, which is included within the camera. When the camera captures a photograph or video using optical imaging principles, it can measure the distance to a target object in a frame of the photograph or video. The target object includes one or more measurement points, such as a first measurement point, a second measurement point, etc. Specifically, the camera also includes a light receiver connected to the controller. During the ranging process, the method executed by the controller includes:

[0178] The controller controls the optical receiver to receive a first reflected light signal and a second reflected light signal. The first reflected light signal is formed by the reflection of a first light signal emitted by the optical transmitter at a first test point, and this first light signal intersects with a first epipolar line, while the first reflected light signal intersects with a second epipolar line. The second reflected light signal is formed by the reflection of the second test point, and this second light signal intersects with a third epipolar line, while the second reflected light signal intersects with a fourth epipolar line. The origin of the optical transmitter's coordinates is the first origin, and the emitting surface of the optical transmitter includes a first plane. The origin of the optical receiver's coordinates is... The second origin is a second plane, and the receiving surface of the optical receiver includes a second plane; the plane containing the first origin and the second origin is a third plane, and the third plane intersects the first plane at the first epipolar line, and the third plane intersects the second plane at the second epipolar line; wherein, the second test point is different from the first test point; and the plane containing the first origin and the second origin is a sixth plane, and the sixth plane intersects the fourth plane at the third epipolar line, and the sixth plane intersects the fifth plane at the fourth epipolar line, wherein the third plane and the sixth plane are not coplanar;

[0179] Subsequently, the controller determines the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and determines the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

[0180] Based on the above technical solution, under the control of the controller, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point, and the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Under the control of the controller, the second reflected light signal received by the optical receiver is formed by the reflection of the second light signal emitted by the optical transmitter at the second test point, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Since the third plane and the sixth plane are not coplanar, that is, the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line, the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. That is, the different light reflection signals received by the optical receiver through different epipolar lines do not interfere with each other, reducing the influence of diffuse reflection of multiple light signals emitted by the optical transmitter at the test point on the camera ranging results, and improving the accuracy of ToF camera ranging.

[0181] In one possible implementation of the sixth aspect of the embodiments of this application,

[0182] The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane;

[0183] The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

[0184] In one possible implementation of the sixth aspect of the present application, the first optical signal and the second optical signal are orthogonal to each other.

[0185] In one possible implementation of the sixth aspect of the embodiments of this application,

[0186] Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0187] In one possible implementation of the sixth aspect of the embodiments of this application,

[0188] The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0189] In one possible implementation of the sixth aspect of this application, the first optical signal is a signal obtained by BPSK encoding.

[0190] In one possible implementation of the sixth aspect of the embodiments of this application,

[0191] The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

[0192] In one possible implementation of the sixth aspect of this application, the method further includes:

[0193] The controller controls the optical receiver to receive a third reflected optical signal, which is formed by the reflection of the third optical signal emitted by the optical transmitter after passing through the first test point. The third optical signal intersects with the first polar line and the third reflected optical signal intersects with the second polar line. The frequency of the third optical signal is the second frequency.

[0194] The controller determines the distance between the first measurement point and the camera based on the first light signal and the first reflected light signal, including:

[0195] The controller uses the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal to determine the distance between the first test point and the camera.

[0196] In one possible implementation of the sixth aspect of this application, the method further includes:

[0197] The controller controls the optical receiver to receive the fourth reflected optical signal; the fourth reflected optical signal is formed by the reflection of the fourth optical signal emitted by the optical transmitter after passing through the second test point, and the fourth optical signal intersects with the third pole line, and the fourth reflected optical signal intersects with the fourth pole line; wherein, the signal frequency of the fourth optical signal is the first frequency;

[0198] The controller determines the distance between the second measurement point and the camera based on the second optical signal and the second reflected optical signal, including:

[0199] The controller determines the distance between the second test point and the camera based on the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal.

[0200] In one possible implementation of the sixth aspect of this application, the controller determines the distance between the first test point and the camera based on the first optical signal and the first reflected optical signal, including:

[0201] The controller samples the first reflected light signal to obtain the first sampling result;

[0202] The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result;

[0203] The controller determines the distance between the first test point and the camera based on the first phase difference.

[0204] In one possible implementation of the sixth aspect of this application, the controller determines the distance between the second test point and the camera based on the second optical signal and the second reflected optical signal, including:

[0205] The controller samples the second reflected light signal to obtain a second sampling result;

[0206] The controller determines the second phase difference between the second optical signal and the second reflected optical signal based on the second sampling result;

[0207] The controller determines the distance between the second test point and the camera based on the second phase difference.

[0208] It should be noted that the specific implementation process and beneficial effects of the methods described in the sixth aspect and any of their possible implementations can be found in the description of the third aspect and its possible implementations mentioned above, and will not be repeated here.

[0209] A seventh aspect of this application provides a chip system including a processor for supporting a controller to implement the functions involved in the fourth aspect or any possible implementation of the fourth aspect, or supporting the controller to implement the functions involved in the fifth aspect or any possible implementation of the fifth aspect, or supporting the controller to implement the functions involved in the sixth aspect or any possible implementation of the sixth aspect.

[0210] In one possible design, the chip system may also include a memory for storing program instructions and data necessary for the controller. The chip system may consist of chips or may include chips and other discrete components.

[0211] An eighth aspect of this application provides a computer-readable storage medium storing one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method as described in the fourth aspect or any possible implementation thereof, or the processor executes the method as described in the fifth aspect or any possible implementation thereof, or the processor executes the method as described in the sixth aspect or any possible implementation thereof.

[0212] The ninth aspect of this application provides a computer program product (or computer program) that stores one or more computers. When the computer program product is run on a computer, it causes the computer to perform the fourth aspect or any possible implementation of the fourth aspect, or causes the computer to perform the fifth aspect or any possible implementation of the fifth aspect, or causes the computer to perform the sixth aspect or any possible implementation of the sixth aspect.

[0213] The technical effects of aspects seven through nine and any possible implementation can be found in aspects four through six and any possible implementation, and will not be repeated here.

[0214] This application provides a Time-of-Flight (ToF) based ranging camera, including a light emitter and a light receiver. The light receiver receives a first reflected light signal formed by the reflection of a first light signal emitted by the light emitter at a first test point, and this first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. The light receiver also receives a second reflected light signal formed by the reflection of a second light signal emitted by the light emitter at a second test point, and this second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera. Because the third plane and the sixth plane are not coplanar (i.e., the first epipolar line is different from the third epipolar line and the second epipolar line is different from the fourth epipolar line), the first light signal and the first reflected light signal used to determine the distance between the first test point and the camera do not interfere with each other, and the second light signal and the second reflected light signal used to determine the distance between the second test point and the camera do not interfere with each other. That is, it ensures that the different light reflection signals received by the light receiver through different epipolar lines do not interfere with each other, reduces the influence of diffuse reflection of the multiple light signals emitted by the light transmitter at the point to be measured on the camera ranging results, and improves the accuracy of ToF camera ranging. Attached Figure Description

[0215] Figure 1 This is a schematic diagram illustrating the ranging implementation of the ToF camera in an embodiment of this application;

[0216] Figure 2 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0217] Figure 3 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0218] Figure 4 This is a schematic diagram of a ToF camera in an embodiment of this application;

[0219] Figure 5AThis is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0220] Figure 5B This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0221] Figure 5C This is another schematic diagram of a ToF camera in an embodiment of this application;

[0222] Figure 5D This is another schematic diagram of a ToF camera in an embodiment of this application;

[0223] Figure 6 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0224] Figure 7 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0225] Figure 8 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0226] Figure 9 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0227] Figure 10A This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0228] Figure 10B This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0229] Figure 10C This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0230] Figure 11 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0231] Figure 12 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0232] Figure 13 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0233] Figure 14 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0234] Figure 15 This is another schematic diagram illustrating the ranging implementation of the ToF camera in the embodiments of this application;

[0235] Figure 16 This is another schematic diagram of a ToF camera in an embodiment of this application;

[0236] Figure 17 This is another schematic diagram of a ToF camera in an embodiment of this application;

[0237] Figure 18 This is a schematic diagram of a ToF-based ranging method in an embodiment of this application;

[0238] Figure 19 This is another schematic diagram of the ToF-based ranging method in the embodiments of this application;

[0239] Figure 20 This is another schematic diagram of the ToF-based ranging method in the embodiments of this application. Detailed Implementation

[0240] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0241] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0242] Time-of-flight (ToF) cameras can be used for distance measurement. Their principle involves sending a light signal to an object using a light emitter, and then receiving the light signal returning from the object using a light receiver. The distance to the object is determined by detecting the round-trip time of the light signal's flight. Specifically, ToF cameras determine the round-trip time of the light signal's flight by measuring the phase difference between the emitted and received light signals. The following will illustrate the ToF camera system and its distance measurement principle through a specific example.

[0243] Generally, a ToF camera has a light emitter, a light receiver, and a controller. The light emitter can also be referred to as a light source, active light source, illumination unit, etc.; the light receiver can also be referred to as a light sensor, ToF sensor, image sensor, sensor, etc.; the controller can also be referred to as a control unit, computing unit, analysis unit, etc. The controller can be integrated into the light emitter, integrated into the ToF sensor, or set independently of the light emitter and the light receiver; this is not limited here. For example, the controller can be a general-purpose processing unit implemented in software, such as a central processing unit (CPU); or it can be a dedicated circuit or chip, such as an application-specific integrated circuit (ASIC) chip 210, etc.

[0244] Please see Figure 1 This is a schematic diagram illustrating the ranging implementation of a ToF camera. In this ToF camera 100, an example is taken where the light emitter is an active light source 101 that emits light signals driven by the light source, the light receiver is a ToF sensor 102 that receives light signals through a pixel array (or pixel matrix), and the controller is integrated into the ToF sensor 102. Figure 1 In this setup, the distance between the ToF camera 100 and the subject 200 is D (or d). The controller integrated in the ToF sensor 102 drives the active light source 101 to emit light signals towards the subject 200. Simultaneously, the controller controls the pixel array in the ToF sensor 101 to receive the light signals reflected from the subject 200, enabling continuous synchronous modulation between the active light source 101 and the pixel array until the exposure ends. Specifically, the signal intensity output by the ToF sensor 102 after collecting electrons through exposure within the pixel array is represented by a set of differential signal collection structures in each pixel: Type A (TapA) and Type B (TapB). Within one cycle of modulation signal transmission and reception (0 degrees to 360 degrees), the light signal received by the ToF sensor in the first half cycle (0 degrees to 180 degrees) is TapA, and the light signal received in the second half cycle (180 degrees to 360 degrees) is TapB.

[0245] Please see Figure 2 This is a schematic diagram illustrating the ranging implementation of a ToF camera, specifically a schematic diagram of the light signal emitted by the light source 101 and the reflected light signal received by the ToF sensor 102 in the ToF camera. Figure 2As shown, within one modulation period, the ToF sensor 102 collects electrons based on TapA from 0 degrees to 180 degrees, and based on TapB from 180 degrees to 360 degrees. After one exposure, the pixel output value is the difference between TapA and TapB, i.e., TapA-TapB. The ToF camera can indirectly determine the distance using the phase difference of the reflected light. Generally, to acquire phase information, the ToF camera needs to perform multiple exposures, such as 3-phase sampling, 4-phase sampling, 6-phase sampling, etc.

[0246] Please see Figure 3 This is a schematic diagram illustrating the ranging implementation of a ToF camera, specifically a schematic diagram of the light signal emitted by the light source 101 and the reflected light signal received by the ToF sensor 102 in the ToF camera. Figure 3 As shown, taking a typical 4-phase sampling as an example, a ToF camera needs to sample the reflected light at four different phases: 0°, 90°, 180°, and 270°. This means the pixel array will repeat the exposure four times with four different phase delays of 0°, 90°, 180°, and 270°. In these four samples, the phase of the pixel array modulation signal is shifted by 0°, 90°, 180°, and 270° respectively, corresponding to outputs Q1, Q2, Q3, and Q4, with the following relationships:

[0247] Q1 = A0 - B 180 ,

[0248] Q2 = A 90 -B 270 ,

[0249] Q3 = A 180 -B0,

[0250] Q4 = A 270 -B 90 .

[0251] Where A0 is the sampled value of TapA at 0°, B 180 This is the TapB sample value at 180°, and the rest follow the same principle, which will not be elaborated here.

[0252] Therefore, the phase value The calculation method is as follows:

[0253]

[0254] Furthermore, the distance d is calculated as follows:

[0255]

[0256] Where c is the speed of light and f is the signal frequency of the light signal emitted by the light source 101.

[0257] Because diffuse reflection is common on object surfaces in general usage scenarios, Figures 1 to 3 In the Time-of-Flight (ToF) ranging process shown, when the subject has more than one reflected light path, the pixel array receives reflected light from many different paths and phases, interfering with the phase calculation. Furthermore, since ToF cameras use intensity integration for sampling, information from reflected light from different paths cannot be easily separated. In other words, in the current ToF camera ranging process, diffuse reflection from the object's surface easily interferes with the ranging process, resulting in low accuracy.

[0258] Therefore, this application provides a time-of-flight (ToF) based ranging camera and control method to reduce the influence of diffuse reflection of multiple optical signals emitted by the optical transmitter at the measurement point on the ranging results of the camera, thereby improving the accuracy of ToF camera ranging.

[0259] Please see Figure 4 This is a schematic diagram of a ToF-based ranging camera 400 (hereinafter referred to as ToF camera 400) in an embodiment of this application. The ToF camera 400 includes a controller 401, and a light transmitter 402 and a light receiver 403 respectively connected to the controller 401.

[0260] It should be noted that the controller 401 can be integrated into the optical transmitter 402, or into the optical receiver 403, or it can be set independently of both the optical transmitter 402 and the optical receiver 403. No limitation is made here.

[0261] The main technical principle applied in this application is the geometric characteristics of the epipolar line in epipolar geometry. An example will be used to illustrate epipolar geometry below. Generally, epipolar geometry describes a geometric coordinate system composed of two cameras, where the cameras can be replaced by light sources while maintaining their geometric characteristics.

[0262] Please see Figure 5A This is a schematic diagram illustrating the ranging implementation of a ToF camera, specifically a schematic diagram illustrating the principle of epipolar geometry. For example... Figure 5A As shown, the origins of the coordinate systems for the two different cameras are O and O, respectively. c With O p The imaging planes are image plane c and image plane p, respectively. The distance between the two cameras is defined as the baseline. X is a point in three-dimensional space, i.e., X is the point to be measured. X and O c With O pThey form a plane called the epipolar plane, which intersects the image plane c and the image plane p respectively. The two lines of intersection are called the epipolar lines c and p, which are an important characteristic in epipolar geometry.

[0263] like Figure 5A As shown, when the position of the point to be measured relative to the camera changes only in depth (for example, when the point to be measured moves from X to X1, X2, or X3), the coordinates of the imaging points of the two cameras will be translated in epipolar geometry. Since X, X1, X2, and X3 are all located on the epipolar plane, the translation direction of the imaging points of the two cameras follows the epipolar line direction. Specifically, the imaging points of X1, X2, and X3 on the image plane p, P1, P2, and P3 respectively, all fall on the epipolar line p, that is, the imaging points are translated on the epipolar line p.

[0264] Please see Figure 5B This is another schematic diagram illustrating the ranging implementation of a ToF camera, specifically another schematic diagram illustrating the principle of epipolar geometry. Figure 5B In, similar to Figure 5A The settings are as follows: the origins of the coordinate systems for the two different cameras are O... c With O p The imaging planes are image plane c and image plane p, respectively. The distance between the two cameras is defined as the baseline. X is a point in three-dimensional space, i.e., X is the point to be measured. X and O c With O p This forms a plane called the first epipolar plane, which intersects the image plane c and the image plane p respectively. The two lines of intersection are called the epipolar lines c and p. There is also a point Y to be measured in space. c With O p This forms a plane called the second epipolar plane, which intersects the image plane c and the image plane p respectively. The two lines of intersection are called epipolar lines n and m. Since the relative positions of the origin of the two cameras and the image plane are fixed, and Y is not located on the first epipolar plane, it is clear that epipolar line c is different from epipolar line n, and epipolar line p is different from epipolar line m.

[0265] like Figure 5B As shown, when the position of the test point relative to the camera changes, causing the moved test point to no longer be on the epipolar plane, for example, when the test point moves from X to Y, since Y is located on the second epipolar plane, the imaging point of Y on the two cameras will be translated on the epipolar lines n and m, but not on the epipolar lines c and p.

[0266] Depend on Figure 5A and Figure 5B As shown in the example, in an ideal state, due to the constraints of polar geometry, in Figure 5BIn the diagram, the reflected light signal (denoted as reflected light signal A) formed at the test point X by the light signal passing through the epipolar line c will inevitably fall on the epipolar line p. However, the reflected light signal (denoted as reflected light signal B) formed at the test point Y by the light signal passing through the epipolar line c will inevitably not fall on the epipolar line p. Therefore, when signal acquisition is performed only on the epipolar line p, only reflected light signal A can be received, and reflected light signal B cannot be received; that is, the reception of reflected light signal A is not affected by reflected light signal B. However, in practical applications, when signal acquisition is performed only on the epipolar line p, reflected light signal B may still affect reflected light signal A due to irregular diffuse reflection. But compared to a scheme that does not use epipolar geometric constraints and receives reflected light signals on the complete image plane p, the interference caused by reflected light signal B can be significantly reduced.

[0267] Similarly, in an ideal state, due to constraints on polar geometry, in Figure 5B In the diagram, the reflected light signal (denoted as reflected light signal C) formed at the test point Y by the light signal passing through the epipolar line n will inevitably fall on the epipolar line m. However, the reflected light signal (denoted as reflected light signal D) formed at the test point X by the light signal passing through the epipolar line n will inevitably not fall on the epipolar line m. Therefore, when signal acquisition is performed only on the epipolar line m, only the reflected light signal C can be received, and the reflected light signal D cannot be received; that is, the reception of the reflected light signal C is not affected by the reflected light signal D. However, in practical applications, when signal acquisition is performed only on the epipolar line m, the reflected light signal D may still affect the reflected light signal C due to irregular diffuse reflection. But compared to a scheme that does not use epipolar geometric constraints and receives the reflected light signal on the complete image plane p, the interference caused by the reflected light signal D can be significantly reduced.

[0268] The above-mentioned use in this application Figure 5A and Figure 5B The diagram illustrates an important characteristic of epipolar geometry. Due to the physical properties of the epipolar line, as the depth of the measured point changes, the received light rays and the camera's imaging point will change along the epipolar line direction. Therefore, it can be ensured that the imaging point of the measured point will fall within the same epipolar plane at various depths, without being affected by other signals.

[0269] It should be noted that the epipolar line can be an oblique line, or a horizontal or vertical line, on the image plane coordinate system, depending on whether the coordinate systems of the two cameras are parallel. For example, ... Figure 5A The coordinate axes shown in the dashed box are as follows: when the angle difference between the coordinate axes of the two cameras is not 0 (e.g., the X-axis of Oc is not parallel to the X-axis of Op), the epipolar line is a slanted line; when the angle difference between the coordinate axes of the two cameras is 0 (e.g., the X-axis of Oc is parallel to the X-axis of Op), the epipolar line is as shown in the dashed box. Figure 5AThe horizontal (or vertical) line shown is parallel to the baseline. In this embodiment and subsequent embodiments, the example is only described using the polar line as a horizontal line.

[0270] based on Figure 5A and Figure 5B The diagram illustrates the principle of epipolar realization. Figure 4 The relevant structures in the ToF camera 400 shown can be accessed via... Figure 5A and Figure 5B The polar principle is shown in the diagram. Please refer to [link / reference]. Figure 5C This is another schematic diagram of the ToF camera 400 provided in the embodiments of this application.

[0271] Specifically, in Figure 5C In this ToF camera 400, a controller 401, a light emitter 402, and a light receiver 403 are included. The origin of the light emitter 402 is a first origin 4021, and its emitting surface includes a first plane 4022. The origin of the light receiver 403 is a second origin 4031, and its receiving surface includes a second plane 4032. Specifically, the emitting surface of the light emitter 402 can be interpreted as the imaging surface of the light emitter 402 in the camera pinhole model, i.e., a plane equivalent to the imaging surface of the light emitter 402 in the camera pinhole model. Similarly, the receiving surface of the light receiver 403 can be interpreted as the imaging surface of the light receiver 403 in the camera pinhole model, i.e., a plane equivalent to the imaging surface of the light receiver 403 in the camera pinhole model, or, in other words, the sensor plane of the light receiver.

[0272] exist Figure 5C During the ranging process of the ToF camera 400 shown, the light emitter 402 in the ToF camera 400, under the control of the controller 401, emits a first light signal towards the first measurement point 100. This first light signal is reflected by the first measurement point to form a first reflected light signal, which intersects with the first epipolar line 4023, and the first reflected light signal intersects with the second epipolar line 4033. The first epipolar line 4023 and the second epipolar line 4033 are epipolar lines in epipolar geometry, meaning that the first epipolar line 4023 and the second epipolar line 4033 satisfy the epipolar constraint. The first test point 100, the plane containing the first origin 4021 and the second origin 4031 is a third plane, and the third plane intersects the first plane at the first epipolar line 4023, and the third plane intersects the second plane at the second epipolar line 4033; at this time, the light receiver 403 in the ToF camera 400 is used to receive the first reflected light signal under the control of the controller 401; the controller 401 is used to determine the distance between the first test point 100 and the camera 400 based on the first light signal and the first reflected light signal.

[0273] Based on the above technical solution, since the first epipolar line 4023 and the second epipolar line 4033 satisfy the epipolar constraint, the first light signal emitted by the light emitter 402 through the first epipolar line 4023, after being reflected by the first test point 100, can pass through the second epipolar line 4033 and be received by the light receiver 403. This reduces the interference of diffuse reflection signals received by the light receiver 403 from areas other than the second epipolar line 4033 on the first reflected light signal. Furthermore, the use of the first light signal and the first reflected light signal to determine the ranging result reduces the interference of other reflected signals generated by diffuse reflection at the first test point 100 on the ranging result, thus reducing the impact of diffuse reflection at the test point 100 on the camera ranging result and improving the accuracy of the ToF camera ranging.

[0274] also, Figure 5C In the ToF camera 400 shown, during the exposure and ranging process of the object under test, the light emitter 402 and the light receiver 403 can perform multiple exposures on the object under test through the transmission and reception of a single light signal to achieve ranging of multiple test points on the object; alternatively, they can perform fewer exposures on the object under test through the transmission and reception of multiple light signals to achieve ranging of multiple test points on the object. Furthermore, by setting different epipolar constraints for different test points, the multiple reflected light signals received by the light receiver through different epipolar lines do not interfere with each other, reducing the impact of diffuse reflection of the multiple light signals emitted by the light emitter at the test points on the camera's ranging results, and improving the accuracy of the ToF camera's ranging. The implementation scheme of transmitting and receiving multiple light signals will be described in detail below.

[0275] Please see Figure 5D This is another schematic diagram of the ToF camera 400 in an embodiment of this application. Figure 5D As shown, compared to Figure 5C The ToF camera 400 in the light transmitter 402 can be configured to transmit multiple light signals, and the light receiver 403 can be configured to receive multiple light signals.

[0276] Specifically, Figure 5DIn the ToF camera 400 shown, the emitting surface of the light emitter 402, in addition to the first plane 4022, may also include at least a fourth plane 4024; similarly, the receiving surface of the light receiver 403, in addition to the second plane 4032, may also include at least a fifth plane 4034. During the ranging process of the ToF camera 400, the light emitter 402, under the control of the controller 401, also emits a second light signal towards a second test point 200 different from the first test point 100. This second light signal is reflected by the second test point 200 to form a second reflected light signal, and this second light signal intersects with the third epipolar line 4025, while the second reflected light signal intersects with the fourth epipolar line 4035. The third epipolar line 4025 and the fourth epipolar line 4035 are epipolar lines in epipolar geometry, meaning that the third epipolar line 4025 and the fourth epipolar line 4035 satisfy epipolar constraints. The second test point 200, the plane containing the first origin 4021 and the second origin 4031 is the sixth plane, and the sixth plane intersects the fourth plane at the third epipolar line 4025, and the sixth plane intersects the fifth plane at the fourth epipolar line 4035, wherein the third plane and the sixth plane are not coplanar; at this time, the light receiver 403 is also used to receive the second reflected light signal under the control of the controller 401; correspondingly, the controller 401 is also used to determine the distance between the second test point 200 and the camera 400 based on the second light signal and the second reflected light signal.

[0277] Since the third epipolar line 4025 and the fourth epipolar line 4035 satisfy the epipolar constraint, the diffuse reflection light signal received by the light receiver 403 through areas other than the fourth epipolar line 4035 can reduce interference with the second reflected light signal. Subsequently, the controller 401 determines the ranging result based on the second light signal and the second reflected light signal, which can reduce the interference of other reflected signals generated by diffuse reflection at the second test point 200 on the ranging result, i.e., reduce the impact of diffuse reflection at the test point on the camera ranging result. Furthermore, since the third plane and the sixth plane are not coplanar, i.e., the first epipolar line 4023 is different from the third epipolar line 4025 and the second epipolar line 4033 is different from the fourth epipolar line 4035, the first light signal and the first reflected light signal used to determine the distance between the first test point 100 and the camera 400 do not interfere with each other. Therefore, the different light signals used to measure distances at different points do not interfere with each other, further improving the accuracy of ToF camera ranging.

[0278] It should be noted that the distance between the second test point 200 and the camera 400 can be the distance between the second test point 200 and the lens (not shown in the figure) in the camera 400, the distance between the second test point 200 and the geometric center (not shown in the figure) in the camera 400, the distance between the second test point 200 and the photosensitive device (e.g., the light receiver 403) in the camera 400, or the distance between the second test point 200 and other physical or virtual parts in the camera 400. No specific limitation is made here.

[0279] In addition to the first plane 4022 and the fourth plane 4024, the light transmitter 402 may include other emitting surfaces, such as the seventh plane or other planes (not shown in the figure); correspondingly, in addition to the second plane 4032 and the fifth plane 4034, the light receiver 403 may include other receiving planes (not shown in the figure), such as the eighth plane or other planes. Furthermore, the other emitting surfaces in the light transmitter 402 and the other receiving surfaces in the light receiver 403 can also be connected by epipolar lines satisfying epipolar constraints, as described above. The controller 401 controls the transmission and reception of light signals on the corresponding epipolar lines to achieve ranging of more different test points. Moreover, since the different light signals used for ranging different test points do not interfere with each other, the ranging accuracy of the ToF camera 400 can be further improved.

[0280] It should be noted that the intersection of the first optical signal and the first pole line 4023 can indicate that the transmission path of the first optical signal intersects the first pole line 4023 at one or more points, or it can indicate that the transmission path of the first optical signal passes through one or more points of the first pole line 4023; similarly, the intersection of the first reflected optical signal and the second pole line 4033 can indicate that the transmission path of the first reflected optical signal intersects the second pole line 4033 at one or more points, or it can indicate that the transmission path of the first reflected optical signal passes through one or more points of the second pole line 4033. For example, when the first optical signal includes a beam of light (or multiple beams of light), the propagation path of the first optical signal passes through the first pole line 4023 and intersects with the first pole line 4023 at a certain point (or multiple points). After that, the first optical signal is reflected by the first test point 100 to form a single beam (or multiple beams) of first reflected light signal. The propagation path of the first reflected light signal passes through the second pole line 4033 and intersects with the second pole line 4033 at a certain point (or multiple points).

[0281] It should be noted that the distance between the first test point 100 and the ToF camera 400 can be the distance between the first test point 100 and the lens (not shown in the figure) in the ToF camera 400, the distance between the first test point 100 and the geometric center (not shown in the figure) in the ToF camera 400, the distance between the first test point 100 and the photosensitive device (e.g., the light receiver 403) in the ToF camera 400, or the distance between the first test point 100 and other physical or virtual parts in the ToF camera 400. No specific limitation is made here.

[0282] In one possible implementation, the optical signal emitted by the optical transmitter 402 (including the first optical signal, the second optical signal, or other optical signals) can be a signal obtained through BPSK encoding. Optionally, the optical signal emitted by the optical transmitter 402 can also be a signal obtained through other encoding methods, such as quadrature phase shift keying (QPSK), quadrature amplitude modulation (QAM), or other methods, which are not limited here.

[0283] The following explanation will take the first optical signal obtained by BPSK encoding as an example. The principle of BPSK is to encode the signal using phase offset, defining code 0 as a signal with a phase offset of 0° and code 1 as a signal with a phase offset of 180°.

[0284] Please see Figure 6 This is a schematic diagram illustrating the ToF camera 400 ranging implementation in an embodiment of this application. Figure 6 As shown, when BPSK is applied to a ToF system, code 0 is the same as the conventional ToF control signal, and code 1 is the conventional ToF control signal with a phase shift of 180°.

[0285] Please see Figure 7 This is another schematic diagram illustrating the ToF camera 400 ranging implementation in this application embodiment. For example... Figure 7 As shown, L0 and L1 are the light source signals of coded 0 and coded 1, respectively; R0 and R1 are the reflected light signals of coded 0 and coded 1, respectively; and S0A, S0B and S1A, S1B are the signals of pixel arrays TapA and TapB of coded 0 and coded 1, respectively. Since the phase difference between the light source and the pixel array remains constant, even if the coded 1 light source and pixel array are simultaneously phase-shifted by 180°, the signals measured by the coded 1 pixel arrays TapA and TapB are still equal to those of coded 0. For example, a ToF system using BPSK random coding to compose the signal is shown in the example below. Figure 8 As shown, the 0-degree and 180-degree phase shifts can be mapped to binary sequences. Figure 8 This is a simple BPSK encoding example, with the encoding being 01011010. In a ToF system, the BPSK encoding offset of 1 / 2 period can be used to generate codes in a pseudo-random manner, and the number of 0s and 1s in the code can be restricted to be equal, thus achieving the effect of resisting interference from different ToF signal sources.

[0286] In one possible implementation, Figure 5D In the ToF camera 400 shown, the emission of multiple optical signals can be achieved at different light source regions in the light emitter 402, and the reception of multiple optical signals can also be achieved at different pixel array regions in the light receiver 403. Specifically, the light emitter 402 includes at least a first light source region for emitting a first optical signal passing through a first plane 4022, and a second light source region for emitting a second optical signal passing through a fourth plane 4024; that is, the emitting surface of the first light source region is the first plane 4022, and the emitting surface of the second light source region is the fourth plane 4025. Similarly, the light receiver 403 includes at least a first pixel array region for receiving a first reflected optical signal passing through a second plane 4032, and a second pixel array region for receiving a second reflected optical signal passing through a fifth plane 4034; wherein the receiving surface of the first pixel array region is the second plane 4032, and the receiving surface of the second pixel array region is the fifth plane 4035. The light emitter 402 can have multiple light source regions, with different emitting surfaces on different light source regions to emit multiple different light signals. Similarly, the light receiver 403 can also have multiple pixel array regions, with different receiving surfaces on different pixel array regions to receive multiple different light signals.

[0287] Please see Figure 10A This is another schematic diagram illustrating the ranging implementation of the ToF camera 400 provided in this application embodiment. Figure 10A In the example shown, in terms of spatial dimension, the light emitter 402 includes six different light source regions, and the light receiver 403 includes six pixel array regions.

[0288] like Figure 10AAs shown, controller 401 controls six light source regions in light emitter 402 to emit light signals, and controls six pixel matrix (or pixel array) regions in light receiver 403 to receive light signals. The six light source regions include a first light source region 40201, a second light source region 40202, a third light source region 40203, a fourth light source region 40204, a fifth light source region 40205, and a sixth light source region 40206; similarly, the six pixel array regions include a first pixel array region 40301, a second pixel array region 40302, a third pixel array region 40303, a fourth pixel array region 40304, a fifth pixel array region 40305, and a sixth pixel array region 40306.

[0289] Specifically, in Figure 10A In the example shown, the emitting surfaces of different light source regions in the light emitter 402 correspond one-to-one with the receiving surfaces of different pixel array regions in the light receiver 403, and the epipolar constraint is satisfied. In this case, the process by which the controller 401 controls the light emitter 402 to emit light signals and controls the light receiver 403 to receive light signals can be as follows: Figure 10B As shown.

[0290] exist Figure 10B In the diagram, the origin of the coordinate system for the light emitter 402 is 4021, and the origin of the coordinate system for the light receiver 403 is 4031. The region between the first light source region 40201 in the light emitter 402 and the first pixel array region 40301 in the light receiver 403 (or, between the second light source region 40202 in the light emitter 402 and the second pixel array region 40302 in the light receiver 403; or, between the third light source region 40203 in the light emitter 402 and the third pixel array region 40301 in the light receiver 403) is... Between column regions 40303; or, between the fourth light source region 40204 in light emitter 402 and the fourth pixel array region 40304 in light receiver 403; or, between the fifth light source region 40205 in light emitter 402 and the fifth pixel array region 40305 in light receiver 403; or, between the sixth light source region 40206 in light emitter 402 and the sixth pixel array region 40306 in light receiver 403, there exists an epipolar line that satisfies the epipolar constraint.

[0291] Specifically, in Figure 10BTaking the implementation process of the first light source region 40201 in the light emitter 402 and the first pixel array region 40301 in the light receiver 403 as an example, the light signal emitted by the first light source region 40201 passes through the first epipolar line 4023 in the first plane 4022 and is reflected by the point to be measured to form a reflected light signal. The reflected light signal passes through the second epipolar line 4033 in the second plane 4032 and is received by the first pixel array region 40301. The point to be measured and the origin 4021 of the coordinate system of the light emitter 402 and the origin 4031 of the coordinate system of the light receiver 403 form a counter-epipolar plane 1. The counter-epipolar plane 1 intersects the first plane 4022 at the first epipolar line 4023 and the second plane 4032 at the second epipolar line 4033. Similarly, other emitting surfaces in the light emitter 402 and other receiving surfaces in the light receiver 403 can also form counter-epipolar planes 2, 3...6 as shown in the figure. Subsequently, the controller 401 controls the transmission and reception of optical signals on the corresponding polar plane to achieve distance measurement of different test points. Furthermore, since the different optical signals used to measure distances of different test points do not interfere with each other, the accuracy of the ToF camera's distance measurement can be further improved.

[0292] In one possible implementation, among the multiple optical signals transmitted and received by the ToF camera 400, the first optical signal and the second optical signal are orthogonal to each other. Specifically, the first optical signal used to measure the distance to the first test point 100 and the second optical signal used to measure the distance to the second test point 200 can be orthogonal to each other, that is, the coherence between the first optical signal and the second optical signal is 0 or close to 0. In this case, even if a part of the first optical signal passes through the fourth pole line 4035 and is received by the optical receiver 403 after multiple diffuse reflections, or a part of the second optical signal passes through the second pole line 4033 and is received by the optical receiver 403 after multiple diffuse reflections, the interference between the first optical signal and the second optical signal can be avoided because the coherence between the first optical signal and the second optical signal is 0, thereby further improving the accuracy of the ToF camera's ranging.

[0293] Optionally, both the first optical signal and the second optical signal are signals obtained through Binary Phase Shift Keying (BPSK) encoding. Specifically, both the first and second optical signals can be obtained through BPSK encoding, that is, by performing BPSK encoding on at least two original sequences respectively to obtain the first and second optical signals. Different original sequences can be used to make the first and second optical signals orthogonal. This provides a specific implementation method for the first and second optical signals, improving the feasibility of the solution.

[0294] Optionally, the first optical signal and the second optical signal can be signals obtained through other encoding methods, such as quadrature phase shift keying (QPSK), quadrature amplitude modulation (QAM), or other methods, which are not limited here.

[0295] based on Figure 5D The structure of the ToF camera 400 shown is illustrated below. Several implementation examples of this ToF camera 400 are introduced below.

[0296] Please see Figure 9 This is another schematic diagram illustrating the ranging implementation of the ToF camera 400 provided in this application embodiment. Figure 9 The C1 and C2 light source regions represent different light source regions of the light emitter 402 in different ToF cameras 400. The C1 and C2 light source regions are encoded using different pseudo-random BPSK encoding methods, with the number of 0s and 1s being equal. The encoding characteristics of the pseudo-random BPSK encoding at any given time point are as follows:

[0297] If the encoded values ​​of the C2 light source region and the C1 light source region are equal, that is, (C1,C2)=(0,0) or (1,1), the C2 light source value received by the C1 pixel array during this period is TapA=i, TapB=0, TapA–TapB=i.

[0298] If the encoded values ​​of the C2 light source region and the C1 light source region are different, that is, (C1,C2)=(1,0) or (1,0), the C2 light source value received by the C1 pixel array during this period is TapA=0, TapB=i, TapA-TapB=-i.

[0299] In the entire exposure process of the ToF system, the number of cycles is extremely large, and due to the pseudo-random sequence encoding characteristics, the probability of C1 and C2 having the same encoding value and the probability of having different encoding values ​​are almost equal. Therefore, the interference of the C2 light source on the C1 pixel array is close to 0.

[0300] Therefore, the first optical signal can be a signal obtained through BPSK encoding, that is, the first optical signal is obtained by BPSK encoding a certain original sequence. This can reduce the interference between different optical signals and improve the ranging accuracy of the ToF camera in scenarios where multiple optical transmitters are working in parallel in a ToF camera (or in scenarios where multiple ToF cameras are working in parallel) by using the encoding characteristics of BPSK.

[0301] In addition, such as Figure 9In the illustrated implementation example, the first optical signal can be a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal can also be a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence. Specifically, the first and second optical signals can be obtained by BPSK encoding pseudo-random sequences. In the entire exposure process of the ToF system, the number of periods is extremely large, and due to the characteristics of pseudo-random sequence encoding, the probability of C1 and C2 having the same encoded value and the probability of their encoded values ​​being different are approximately equal. Therefore, the interference of the C2 light source on the C1 pixel array is close to 0. Thus, the number of sequences with a value of 0 in the first pseudo-random sequence used to generate the first optical signal is the same as the number of sequences with a value of 0 in the second pseudo-random sequence used to generate the second optical signal, ensuring that the interference between the generated first and second optical signals is 0. This means that the first optical signal used to measure the distance to the first test point and the second optical signal used to measure the distance to the second test point do not interfere with each other, further improving the accuracy of ToF camera ranging. In addition, using pseudo-random BPSK encoding has additional advantages, such as resisting ToF multi-camera interference when there is more than one ToF camera in the application scenario.

[0302] based on Figure 5D The ranging process of the ToF camera 400, which uses multiple optical signals, is illustrated. In terms of spatial dimension, this is exemplified by implementing a 6-channel optical signal implementation. Please refer to [link / reference]. Figure 10C This is another schematic diagram illustrating the ranging implementation of the ToF camera 400 provided in this application embodiment. Figure 10C The correspondence between the multiple different light source regions in the light emitter 402 and the multiple different pixel array regions in the light receiver 403 can be found in the reference. Figure 10A and Figure 10B The description of [the subject] will not be repeated here. For example... Figure 10C As shown, controller 401 can control the six light source regions in light emitter 402 to emit light signals through six control signals (control signal 1, controller signal 2...control signal 6), and control the six pixel matrix (or pixel array) regions in light receiver 403 to receive light signals. The control signals in different light source regions correspond one-to-one with the control signals in different pixel matrix regions, enabling distance measurement of different areas to be measured in different epipolar planes (epipole coding 1, epipolar coding 2...epipole coding 6).

[0303] In one possible implementation, among the multiple optical signals transmitted and received by the ToF camera 400, the signal frequency of the first optical signal is a first frequency, and the signal frequency of the second optical signal is a second frequency, with the first frequency being different from the second frequency. Specifically, the first and second optical signals can be transmitted at different signal frequencies to achieve mutual orthogonality between them, resulting in zero coherence between them. In this case, even if a portion of the first optical signal undergoes multiple diffuse reflections and passes through the fourth pole line to be received by the optical receiver, or a portion of the second optical signal undergoes multiple diffuse reflections and passes through the second pole line to be received by the optical receiver, the different frequencies of the first and second optical signals prevent interference between them, further improving the ranging accuracy of the ToF camera.

[0304] Furthermore, if the optical transmitter includes other emitting surfaces besides the first and fourth planes, such as a seventh plane or other planes, then, similar to the relationship between the first plane and the first epipolar line (or the relationship between the third epipolar line and the fourth plane), other epipolar lines also exist in these other emitting surfaces. Moreover, the optical transmitter can emit other optical signals that pass through these other epipolar lines. These other optical signals can use signal frequencies different from the first and second frequencies. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can set n different signal frequencies for the n optical signals, meaning that the signal frequencies of the n optical signals are all different. Alternatively, the other optical signals can also use either the first or second frequency. For example, when the optical transmitter emits n optical signals (n > 2, and the n optical signals include the first and second optical signals), it can alternately set the n optical signals to the first and second frequencies, meaning that any adjacent optical signals among the n optical signals will have different signal frequencies, thus achieving a better anti-interference effect.

[0305] Please see Figure 11 This is another schematic diagram illustrating the ranging implementation of the ToF camera 400 provided in this application embodiment. Figure 11 The correspondence between the multiple different light source regions in the light emitter 402 and the multiple different pixel array regions in the light receiver 403 can be found in the reference. Figure 10A and Figure 10B The description will not be repeated here. For example, taking the implementation of the 6 optical signals shown in Figure 10 as an example in the time dimension, as follows... Figure 11 As shown, the controller 401 can add different time-domain codes (time-domain code 1, time-domain code 2...time-domain code 6) to different pole planes of the optical transmitter 402 to ensure that different optical signals are orthogonal to each other.

[0306] Specifically, the implementation process using different time-domain encodings can be described as follows: Figure 12 and Figure 13 As shown, in Figure 12 and Figure 13 The correspondence between the multiple different light source regions in the light emitter 402 and the multiple different pixel array regions in the light receiver 403 can be found in the reference. Figure 10A and Figure 10B The description will not be repeated here. Specifically, such as Figure 12 As shown, through the multiplex control signals provided by the controller 401, different BPSK encoding sequences are used in different epipolar planes (BPSK encoding 1, BPSK encoding 2...BPSK encoding 6) in the optical transmitter 402 and optical receiver 403 to encode and transmit signals; alternatively, as shown in the diagram... Figure 13 As shown, the controller 401 controls the provided multiple control signals, and different signal frequencies (modulation frequency 1, modulation frequency 2... modulation frequency 6) are used for signal transmission and reception in different polarimetric planes in the optical transmitter 402 and optical receiver 403. Alternatively, other encoding methods can be used to distinguish optical signals in different regions in the time dimension, such as QPSK, QAM, etc., or a combination of multiple time-domain encoding methods, which will not be elaborated here. For example, when the ToF camera 400 transmits through... Figure 13 In the illustrated implementation, time-domain encoding 1 to time-domain encoding 6 are performed at different frequencies, that is, the light source and pixel matrix are encoded at different modulation frequencies to encode the epipolar regions. As an example, modulation frequency 1 = 100 MHz, modulation frequency 2 = 101 MHz, modulation frequency 3 = 102 MHz, modulation frequency 4 = 103 MHz, modulation frequency 5 = 104 MHz, and modulation frequency 6 = 105 MHz. Clearly, the values ​​of these modulation frequencies can be achieved in other ways, where the same frequency is used for the same epipolar region, and different frequencies are used for different epipolar regions. Due to the principle of the Time-of-Flight (ToF) system, different modulation frequencies cannot interfere with each other; therefore, different epipolar regions cannot interfere with each other.

[0307] In one possible implementation, through Figures 1 to 3As shown in the ranging principle of the ToF camera, since ToF cameras use phase difference to calculate distance, the maximum ranging range of a ToF camera at a single frequency f1 is c / (2f1). When the distance is greater than c / (2f1), the measured distance will exhibit periodic aliasing. To avoid this defect, the ToF camera can add a second frequency f2 for phase calculation, using the relationship between f1 and f2 to extend its usable range. There are many methods for dual-frequency dealiasing, with common combinations including dual high frequencies and high frequency plus low frequency. This technique is called phase dealiasing.

[0308] In the implementation of phase dealiasing, a relatively simple dealiasing algorithm is provided as an example. For instance, if the first frequency f1 is set to 100MHz, its maximum ranging range is c / (2×100×10). 6 = 1.5 meters. Assume there are three objects A, B, and C in the ranging scene, with actual distances of 2 meters for A, 3.5 meters for B, and 5 meters for C. The depth of objects A, B, and C measured by the ToF camera at frequency f1 is 0.5 meters due to aliasing. More precisely, the distance measured by f1 for A, B, and C should be 0.5 + n * 1.5 meters, where n is the number of aliasing cycles for objects A, B, and C. To solve for n, a second frequency f2 = 20 MHz is introduced, with a maximum ranging range of c / (2 × 20 × 10^6 MHz). 6 Given that f1 has a depth of 7.5 meters, and objects A, B, and C are all within the maximum distance range of f2, the correct depths of objects A, B, and C measured by f2 should be: A = 0.5 + 1.5 × 1 = 2 meters, B = 0.5 + 1.5 × 2 = 3.5 meters, and C = 0.5 + 1.5 × 3 = 5 meters. Object A overlaps for 1 cycle, object B for 2 cycles, and object C for 3 cycles. Because f1 has a higher frequency, the depth accuracy after de-aliasing with f1 is better than the accuracy of directly measuring the distance using f2.

[0309] Specifically, based on the aforementioned ToF camera 400 and the implementation process of multiple optical signals of different frequencies transmitted and received by the ToF camera 400, the optical transmitter 402 can also be used, under the control of the controller 401, to transmit a third optical signal with a signal frequency of the second frequency to the first test point 100. The third optical signal is reflected by the first test point 100 to form a third reflected optical signal, wherein the third optical signal intersects with the first epipolar line 4023 and the third reflected optical signal intersects with the second epipolar line 4033. Correspondingly, the optical receiver 403 is also used, under the control of the controller 401, to receive the third reflected optical signal. The controller 401 is also used to determine the distance between the first test point 100 and the camera 400 based on the first optical signal, the first reflected optical signal, the third optical signal, and the third reflected optical signal. Since ToF cameras 400 typically calculate distance using phase difference, their maximum ranging range at a single frequency is limited by the mathematical relationship between that frequency and the speed of light. When the distance between the measured point and the camera exceeds this maximum ranging range, the measured distance will suffer from poor accuracy due to periodic aliasing. To avoid this drawback, the controller 401 can control the transmission and reception of light signals at different frequencies for the same measured point, and use the relationship between the first and second frequencies for calculation to extend the ranging range of the ToF camera. Furthermore, the accuracy of the ranging result for the first measured point can be improved by using a first light signal with a first frequency and a third light signal with a second frequency.

[0310] Similarly, based on the aforementioned ToF camera 400 and the implementation process of transmitting and receiving multiple optical signals of different frequencies by the ToF camera 400, the optical transmitter 402 is further configured, under the control of the controller 401, to transmit a fourth optical signal with a signal frequency of the first frequency to the second test point 200. The fourth optical signal is reflected by the second test point to form a fourth reflected optical signal, and the fourth optical signal intersects with the third epipolar line 4025, while the fourth reflected optical signal intersects with the fourth epipolar line 4035. Correspondingly, the optical receiver 403 is further configured, under the control of the controller 401, to receive the fourth reflected optical signal of the fourth optical signal at the second test point 200. At this time, the controller 401 is also configured to determine the distance between the second test point 200 and the camera 400 based on the second optical signal, the second reflected optical signal, the fourth optical signal, and the fourth reflected signal. The distance to the second test point 200 can be calculated by transmitting optical signals of different frequencies and utilizing the relationship between the first and second frequencies, thereby extending the ranging distance of the ToF camera. In addition, the accuracy of the ranging result of the second test point can be improved by using a second optical signal with a second frequency and a fourth optical signal with a first frequency.

[0311] Please see Figure 14 This is another schematic diagram illustrating the ranging implementation of the ToF camera 400 provided in this application embodiment. Figure 14 The correspondence between the multiple different light source regions in the light emitter 402 and the multiple different pixel array regions in the light receiver 403 can be found in the reference. Figure 10A and Figure 10B The description of [the subject] will not be repeated here. For example... Figure 14 As shown, controller 401 can control the light emitter 402 and the light receiver 403 respectively, alternating between different time-domain encodings (modulation frequency 1 and modulation frequency 2) in different epipolar planes to ensure that different optical signals are orthogonal to each other. For example, modulation frequency 1 = 80MHz and modulation frequency 2 = 60MHz. Obviously, the value of this modulation frequency can be achieved in other ways, and different epipolar regions can also use the same frequency. Due to the principle of the ToF system, different modulation frequencies cannot interfere with each other, so the two different frequency epipolar regions (modulation frequency 1 and modulation frequency 2) cannot interfere with each other. In addition, using two different frequency time-domain encodings has an additional advantage: simply swapping modulation frequency 1 and modulation frequency 2 and re-exposing can simultaneously apply phase dealiasing to improve the maximum ranging distance of ToF.

[0312] In one possible implementation, based on Figures 1 to 3 The ranging principle of the ToF camera shown is as follows: In the ToF camera 400, during the process of determining the distance between the first measurement point 100 and the camera 400, the controller 401 first samples the first reflected light signal to obtain a first sampling result; then, it determines the first phase difference between the first light signal and the first reflected light signal based on the first sampling result; further, it determines the distance between the first measurement point and the camera based on the first phase difference. Specifically, the camera 400 can control the light emitter 402 to emit light signals and control the light receiver 403 to receive light signals through the controller 401, so that the light emitter 402 and the light receiver 403, under the control of the controller 401, can measure the distance to the measurement point based on the ToF principle. The controller 401 can determine the distance between the first test point 100 and the camera 400 by the first light signal emitted by the light transmitter 402 and the first reflected light signal received by the light receiver 403. Specifically, it can be based on the ToF principle, by solving the phase difference between the first light signal and the second reflected light signal, and then further determine the distance between the first test point and the camera based on the first phase difference.

[0313] Similarly, in the ToF camera 400, in determining the distance between the second test point 200 and the camera 400, the controller 401 first samples the second reflected light signal to obtain a second sampling result; then, it determines the second phase difference between the second light signal and the second reflected light signal based on the second sampling result; further, it determines the distance between the second test point 200 and the camera 400 based on the second phase difference. Specifically, the controller 401 can determine the distance between the second test point 200 and the camera 400 using the second light signal emitted by the light emitter 402 and the second reflected light signal received by the light receiver 403. Specifically, based on the ToF principle, the controller can obtain the second phase difference by solving for the phase between the second light signal and the second reflected light signal, and then further determine the distance between the second test point and the camera based on the second phase difference.

[0314] In this embodiment, since the time-coded signals are orthogonal to each other, light sources from different epipolar regions, after diffuse reflection and reception by the pixel matrix, will not affect the ranging results, thus reducing multipath interference. Figure 15 For example, in Figure 15 The correspondence between the multiple different light source regions in the light emitter 402 and the multiple different pixel array regions in the light receiver 403 can be found in the reference. Figure 10A and Figure 10B The description will not be repeated here. Specifically, in Figure 15 In this system, the first row of the pixel matrix receives the direct path (light signal) from the first row of the light source, and simultaneously receives the multipath light transmitted through diffuse reflection from the second row of the light source. Compared to conventional Time-of-Flight (ToF) systems, where all diffuse reflection and multipath light from the light source can cause multipath interference, in this case… Figure 15 In the implementation of the ToF camera shown, due to the different time encoding of different epipolar regions, the multipath light emitted by the light source in the second row of epipolar regions will not be able to interfere with the first row of epipolar regions of the pixel matrix.

[0315] based on Figures 1 to 15 In addition to the implementation process shown, this application also provides other implementation schemes for a ToF-based ranging camera, as detailed below.

[0316] like Figure 16 As shown, compared to Figures 4 to 15 The ToF camera 400 shown includes only a light transmitter 402 and a light receiver 403, and is externally connected to a controller 401 for controlling signal transmission and reception of the light transmitter 402 and the light receiver 403, as well as for distance calculation. Figure 16The process of distance measurement in the ToF camera 400 shown, including the light emitter 402, the light receiver 403, and the external controller 401, can be referred to the aforementioned... Figures 1 to 15 The implementation process shown is not described in detail here.

[0317] like Figure 17 As shown, compared to Figures 4 to 15 The ToF camera 400 shown includes only a light receiver 403 and a controller 401, and is externally connected to a light transmitter 402 for transmitting light signals. Figure 17 In the ToF camera 400 shown, the process of light receiver 403 and controller 401, as well as the distance measurement process achieved through external light transmitter 402, can be referred to the aforementioned... Figures 1 to 15 The implementation process shown is not described in detail here.

[0318] In one possible implementation, the one-to-one correspondence between different light source regions in the light emitter 402 and different pixel array regions in the light receiver 403 can be achieved through hardware structural constraints. For example, in Figure 4 and Figure 16 In the camera 400 shown, through constraints such as embedded slots and limit locks, the spatial position of each light source region in the light emitter 402 is fixed relative to the spatial position of each pixel array region in the light receiver 403, which has epipolar constraints. For example, in... Figure 17 Since the camera 400 shown does not have a light emitter 402, a slot can be reserved in the camera 400 so that after the light emitter 402 is connected, the spatial position of each light source area in the light emitter 402 remains unchanged from the spatial position of each pixel array area in the light receiver 403 with epipolar constraints.

[0319] In another possible implementation, the one-to-one positional relationship between different light source regions in the light emitter 402 and different pixel array regions in the light receiver 403 can be achieved through a manually adjusted constraint. For example, the spatial positions of the light emitter 402 and / or the light receiver 403 within the camera 400 can be adjusted, for instance, by means of pulleys or rollers. Before the camera 400 performs ranging, manual adjustment ensures that the spatial position of each light source region in the light emitter 402 remains constant relative to the spatial position of each pixel array region in the light receiver 403, which has an epipolar constraint.

[0320] In addition, the aforementioned Figures 4 to 17In any embodiment of the camera 400, different light source regions can be integrated into the light emitter 402, or each light source region can be independently disposed in the light emitter 402; this is not limited here. Similarly, different pixel array regions can be integrated into the light receiver 403, or each pixel array region can be independently disposed in the light receiver 403; this is not limited here. For example, by obtaining the camera matrix, distortion parameters, and rotation / translation matrix of both the light emitter 402 and the light receiver 403 through stereo vision correction, the epipolar regions corresponding to each of the light emitter 402 and the light receiver 403 can be determined.

[0321] Please see Figure 18 This is a schematic diagram of a ToF-based ranging method provided in an embodiment of this application, wherein the method can be applied to... Figures 4 to 17 The controller in any implementation is included in the camera; the camera further includes a light emitter and a light receiver respectively connected to the controller; the origin of the coordinates of the light emitter is a first origin and the emitting surface of the light emitter includes a first plane, and the origin of the coordinates of the light receiver is a second origin and the receiving surface of the light receiver includes a second plane.

[0322] Specifically, the ranging method includes the following steps.

[0323] S101. The controller controls the light transmitter to transmit a first light signal to the first test point, and controls the light transmitter to transmit a second light signal to the second test point.

[0324] In this embodiment, in step S101, the controller controls the light emitter to emit a first light signal towards the first test point. The first light signal is reflected by the first test point to form a first reflected light signal, which intersects with the first epipolar line and the first reflected light signal intersects with the second epipolar line. The second light signal is reflected by the second test point to form a second reflected light signal, which intersects with the third epipolar line and the second reflected light signal intersects with the fourth epipolar line. The first test point, the plane containing the first origin and the second origin is a third plane, which intersects with the first plane at the first epipolar line and with the second plane at the second epipolar line. The second test point is different from the first test point; the second test point, the plane containing the first origin and the second origin is a sixth plane, which intersects with the fourth plane at the third epipolar line and with the fifth plane at the fourth epipolar line. The third plane and the sixth plane are not coplanar.

[0325] S102. The controller controls the optical receiver to receive the first reflected light signal and the second reflected light signal.

[0326] In this embodiment, in step S102, the controller controls the optical receiver to receive the first reflected optical signal formed by the reflection of the first optical signal emitted in step S101, and to receive the second reflected optical signal formed by the reflection of the second optical signal emitted in step S101.

[0327] S103. The controller determines the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and determines the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

[0328] In this embodiment, in step S103, the controller determines the distance between the first test point and the camera based on the second light signal emitted in step S101 and the first reflected light signal received in step S102. The controller also determines the distance between the second test point and the camera based on the second light signal emitted in step S101 and the second reflected light signal received in step S102.

[0329] In one possible implementation,

[0330] The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane;

[0331] The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

[0332] In one possible implementation, the first optical signal and the second optical signal are orthogonal to each other.

[0333] In one possible implementation,

[0334] Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0335] In one possible implementation,

[0336] The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0337] In one possible implementation, the first optical signal is a signal obtained by BPSK encoding.

[0338] In one possible implementation,

[0339] The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

[0340] In one possible implementation, after step S103, the method further includes:

[0341] The controller controls the light emitter to emit a third light signal with the second frequency to the first test point. The third light signal is reflected by the first test point to form a third reflected light signal, and the third light signal intersects with the first polar line and the third reflected light signal intersects with the second polar line.

[0342] The controller controls the optical receiver to receive the third reflected light signal;

[0343] The controller determines the distance between the first measurement point and the camera based on the first light signal and the first reflected light signal, including:

[0344] The controller determines the distance between the first test point and the camera based on the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal.

[0345] In one possible implementation, after step S103, the method further includes:

[0346] The controller controls the light emitter to emit a fourth light signal with the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal, and the fourth light signal intersects with the third pole line, and the fourth reflected light signal intersects with the fourth pole line.

[0347] The controller controls the optical receiver to receive the four reflected light signals;

[0348] The controller determines the distance between the second measurement point and the camera based on the second optical signal and the second reflected optical signal, including:

[0349] The controller determines the distance between the second test point and the camera based on the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal.

[0350] In one possible implementation, step S103, where the controller determines the distance between the first measurement point and the camera based on the first light signal and the first reflected light signal, may specifically include:

[0351] The controller samples the first reflected light signal to obtain the first sampling result;

[0352] The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result;

[0353] The controller determines the distance between the first test point and the camera based on the first phase difference.

[0354] In one possible implementation, after step S103, the process by which the controller determines the distance between the second measurement point and the camera based on the second optical signal and the second reflected optical signal may specifically include:

[0355] The controller samples the second reflected light signal to obtain a second sampling result;

[0356] The controller determines the second phase difference between the second optical signal and the second reflected optical signal based on the second sampling result;

[0357] The controller determines the distance between the second test point and the camera based on the second phase difference.

[0358] It should be noted that, in Figure 18 The implementation of the corresponding ranging method can also refer to the aforementioned methods. Figures 1 to 17 The implementation process shown has been further optimized and improved, but will not be elaborated here.

[0359] In this embodiment, under the control of the controller, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point. This first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Because the first and second epipolar lines satisfy epipolar constraints, the first reflected light signal formed by the reflection of the first light signal emitted by the optical transmitter through the first epipolar line at the first test point can pass through the second epipolar line and be received by the optical receiver. This reduces interference from diffuse reflection signals received by the optical receiver from areas other than the second epipolar line. Subsequently, the controller determines the ranging result based on the first light signal and the first reflected light signal, reducing interference from other reflected signals generated by diffuse reflection at the first test point, thus reducing the impact of diffuse reflection at the test point on the camera's ranging result and improving the accuracy of the ToF camera's ranging.

[0360] Please see Figure 19 This is another schematic diagram of a ToF-based ranging method provided in an embodiment of this application, wherein the method is applied to a controller, wherein the controller is included in a camera; the camera also includes an optical receiver connected to the controller.

[0361] Specifically, the ranging method includes the following steps.

[0362] S201. The controller controls the optical receiver to receive the first reflected light signal and the second reflected light signal.

[0363] In this embodiment, in step S201, the controller controls the optical receiver to receive a first reflected light signal. The first reflected light signal is formed by the reflection of a first light signal emitted by the optical transmitter through a first test point, and the first light signal intersects with a first epipolar line and the first reflected light signal intersects with a second epipolar line. The second reflected light signal is formed by the reflection of a second light signal emitted by the optical transmitter through a second test point, and the second light signal intersects with a third epipolar line and the second reflected light signal intersects with a fourth epipolar line. The origin of the coordinates of the optical transmitter is a first origin and the emitting surface of the optical transmitter includes a first plane. The origin of the coordinates of the optical receiver is a second origin and the receiving surface of the optical receiver includes a second plane. The plane containing the first test point, the first origin, and the second origin is a third plane, and the third plane intersects with the first plane at the first epipolar line and the third plane intersects with the second plane at the second epipolar line.

[0364] In addition, the emitting surface of the light emitter also includes a fourth plane, and the receiving surface of the light receiver also includes a fifth plane; the second test point, the plane in which the first origin and the second origin are located is a sixth plane, and the sixth plane intersects the fourth plane at the third pole line, and the sixth plane intersects the fifth plane at the fourth pole line; wherein, the third plane and the sixth plane are not coplanar.

[0365] S202. The controller determines the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and determines the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

[0366] In this embodiment, in step S202, the controller determines the distance between the first test point and the camera based on the first light signal and the first reflected light signal in step S101.

[0367] In one possible implementation,

[0368] The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane;

[0369] The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

[0370] In one possible implementation, the first optical signal and the second optical signal are orthogonal to each other.

[0371] In one possible implementation,

[0372] Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0373] In one possible implementation,

[0374] The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0375] In one possible implementation, the first optical signal is a signal obtained by BPSK encoding.

[0376] In one possible implementation,

[0377] The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

[0378] In one possible implementation, after step S202, the method further includes:

[0379] The controller controls the optical receiver to receive a third reflected optical signal, which is formed by the reflection of the third optical signal emitted by the optical transmitter after passing through the first test point. The third optical signal intersects with the first polar line and the third reflected optical signal intersects with the second polar line. The frequency of the third optical signal is the second frequency.

[0380] The controller determines the distance between the first measurement point and the camera based on the first light signal and the first reflected light signal, including:

[0381] The controller uses the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal to determine the distance between the first test point and the camera.

[0382] In one possible implementation, after step S202, the method further includes:

[0383] The controller controls the optical receiver to receive the fourth reflected optical signal; the fourth reflected optical signal is formed by the reflection of the fourth optical signal emitted by the optical transmitter after passing through the second test point, and the fourth optical signal intersects with the third pole line, and the fourth reflected optical signal intersects with the fourth pole line; wherein, the signal frequency of the fourth optical signal is the first frequency;

[0384] The controller determines the distance between the second measurement point and the camera based on the second optical signal and the second reflected optical signal, including:

[0385] The controller determines the distance between the second test point and the camera based on the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal.

[0386] In one possible implementation, in step S202, the controller determines the distance between the first measurement point and the camera based on the first light signal and the first reflected light signal, including:

[0387] The controller samples the first reflected light signal to obtain the first sampling result;

[0388] The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result;

[0389] The controller determines the distance between the first test point and the camera based on the first phase difference.

[0390] In one possible implementation, after step S202, the controller determines the distance between the second test point and the camera based on the second light signal and the second reflected light signal by including:

[0391] The controller samples the second reflected light signal to obtain a second sampling result;

[0392] The controller determines the second phase difference between the second optical signal and the second reflected optical signal based on the second sampling result;

[0393] The controller determines the distance between the second test point and the camera based on the second phase difference.

[0394] Based on the above technical solution, under the control of the controller, the first reflected light signal received by the optical receiver is formed by the reflection of the first light signal emitted by the optical transmitter at the first test point. This first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Since the first and second epipolar lines satisfy epipolar constraints, the first reflected light signal formed by the reflection of the first light signal emitted by the optical transmitter through the first epipolar line at the first test point can pass through the second epipolar line and be received by the optical receiver. This reduces interference from diffuse reflection signals received by the optical receiver from areas other than the second epipolar line. Subsequently, the controller determines the ranging result based on the first light signal and the first reflected light signal, reducing interference from other reflected signals generated by diffuse reflection at the first test point, thus reducing the impact of diffuse reflection at the test point on the camera's ranging result and improving the accuracy of the ToF camera's ranging.

[0395] It should be noted that, in Figure 19 The implementation of the corresponding ranging method can also refer to the aforementioned methods. Figures 1 to 17 The implementation process shown has been further optimized and improved, but will not be elaborated here.

[0396] Please see Figure 20 This is another schematic diagram of a ToF-based ranging method provided in the embodiments of this application. The method is applied to a camera, which includes a light emitter and a light receiver. The origin of the coordinates of the light emitter is a first origin and the emitting surface of the light emitter includes a first plane. The origin of the coordinates of the light receiver is a second origin and the receiving surface of the light receiver includes a second plane.

[0397] Specifically, the ranging method includes the following steps.

[0398] S301. The optical transmitter transmits a first optical signal to the first test point and a second optical signal to the second test point.

[0399] In this embodiment, in step S301, during the ranging process, the camera's light emitter emits a first light signal towards the first test point. This first light signal is reflected by the first test point to form a first reflected light signal. This first light signal intersects with a first epipolar line, and the first reflected light signal intersects with a second epipolar line. The plane containing the first test point, the first origin, and the second origin is a third plane, which intersects with the first plane at the first epipolar line, and with the second plane at the second epipolar line.

[0400] Furthermore, the emitting surface of the light emitter also includes a fourth plane, and the receiving surface of the light receiver also includes a fifth plane; in step S101, the light emitter emits a second light signal to the second test point, the second light signal is reflected by the second test point to form a second reflected light signal, the second light signal intersects with the third pole line, and the second reflected light signal intersects with the fourth pole line; wherein, the second test point is different from the first test point; the plane containing the first origin and the second origin of the second test point is a sixth plane, and the sixth plane intersects with the fourth plane at the third pole line, and the sixth plane intersects with the fifth plane at the fourth pole line, wherein the third plane and the sixth plane are not coplanar;

[0401] S302. The optical receiver receives the first reflected light signal and the second reflected light signal.

[0402] In this embodiment, in step S302, the light receiver in the camera receives a first reflected light signal and a second reflected light signal. The first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera.

[0403] In one possible implementation,

[0404] The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane;

[0405] The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

[0406] In one possible implementation, the first optical signal and the second optical signal are orthogonal to each other.

[0407] In one possible implementation,

[0408] Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

[0409] In one possible implementation,

[0410] The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence. The number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

[0411] In one possible implementation, the first optical signal is a signal obtained by BPSK encoding.

[0412] In one possible implementation,

[0413] The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

[0414] In one possible implementation, in addition to steps S301 and S302, the method further includes:

[0415] The light emitter emits a third light signal with the second frequency to the first test point. The third light signal is reflected by the first test point to form a third reflected light signal. The third light signal intersects with the first polar line, and the third reflected light signal intersects with the second polar line.

[0416] The optical receiver receives the third reflected light signal, wherein the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal are used to determine the distance between the first test point and the camera.

[0417] In one possible implementation, in addition to steps S301 and S302, the method further includes:

[0418] The light emitter emits a fourth light signal with the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal. The fourth light signal intersects with the third pole line, and the fourth reflected light signal intersects with the fourth pole line.

[0419] The optical receiver receives the fourth reflected light signal, wherein the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal are used to determine the distance between the second test point and the camera.

[0420] In one possible implementation, the camera also includes a controller connected to the light transmitter and the light receiver respectively;

[0421] In step S301, the process of the optical transmitter transmitting a first optical signal to the first test point may specifically include:

[0422] The controller controls the light transmitter to emit the first light signal toward the first point under test;

[0423] In step S302, the process of the optical receiver receiving the first reflected light signal may specifically include:

[0424] The controller controls the optical receiver to receive the first reflected light signal.

[0425] In one possible implementation, after step S302, the method may further include:

[0426] The controller samples the first reflected light signal to obtain the first sampling result;

[0427] The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result;

[0428] The controller determines the distance between the first test point and the camera based on the first phase difference.

[0429] Based on the above technical solution, during the ranging process, the first reflected light signal received by the camera's light receiver is formed by the reflection of the first light signal emitted by the light emitter at the first test point. This first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera. Since the first and second epipolar lines satisfy epipolar constraints, the first reflected light signal formed by the reflection of the first light signal emitted by the light emitter through the first epipolar line at the first test point can pass through the second epipolar line and be received by the light receiver. This reduces the interference of diffuse reflection signals received by the light receiver from areas outside the second epipolar line on the first reflected light signal, thus reducing the interference of other reflected signals generated by diffuse reflection at the test point on the ranging process of the ToF camera. This reduces the impact of diffuse reflection at the test point on the camera's ranging results and improves the accuracy of the ToF camera's ranging.

[0430] It should be noted that, in Figure 20 The implementation of the corresponding ranging method can also refer to the aforementioned methods. Figures 1 to 17 The implementation process shown has been further optimized and improved, but will not be elaborated here.

[0431] This application also provides a chip system, which includes a processor for supporting the controller to implement the above-described features. Figure 18 or Figure 19 or Figure 20 The functions involved in the method shown.

[0432] In one possible design, the chip system may also include a memory for storing necessary program instructions and data for the controller. This chip system can be composed of chips or may include chips and other discrete components. The technical advantages of the chip system can be found in [reference needed]. Figure 18 or Figure 19 or Figure 20 The technical effects of the method shown will not be elaborated here.

[0433] This application also provides a computer-readable storage medium storing one or more computer-executable instructions, which, when executed by a processor, perform the above-described actions. Figure 18 or Figure 19 or Figure 20 The method shown is described in detail above and will not be repeated here.

[0434] This application also provides a computer program product (or computer program) that stores one or more computers. When the computer program product is run on a computer, it causes the computer to perform the above-described... Figure 18 or Figure 19 or Figure 20The details shown are as described above and will not be repeated here.

[0435] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be included within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A ranging camera based on Time-of-Flight (ToF), characterized in that, It includes an optical transmitter and an optical receiver; the origin of the optical transmitter is a first origin and the emitting surface of the optical transmitter includes a first plane and a fourth plane; the origin of the optical receiver is a second origin and the receiving surface of the optical receiver includes a second plane and a fifth plane. The light emitter is used to emit a first light signal to a first test point and a second light signal to a second test point. The first light signal is reflected by the first test point to form a first reflected light signal, which intersects with a first epipolar line and a second epipolar line. The second light signal is reflected by the second test point to form a second reflected light signal, which intersects with a third epipolar line and a fourth epipolar line. Specifically, for the first test point, the plane containing the first origin and the second origin is a third plane, which intersects the first plane at the first epipolar line and the second plane at the second epipolar line. For the second test point, the plane containing the first origin and the second origin is a sixth plane, which intersects the fourth plane at the third epipolar line and the fifth plane at the fourth epipolar line. The third plane and the sixth plane are not coplanar. The light receiver is used to receive the first reflected light signal and the second reflected light signal, wherein the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera, and the second light signal and the second reflected light signal are used to determine the distance between the second test point and the camera.

2. The camera according to claim 1, characterized in that, The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane; The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

3. The camera according to claim 1 or 2, characterized in that, The first optical signal and the second optical signal are orthogonal to each other.

4. The camera according to any one of claims 1 to 3, characterized in that, Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

5. The camera according to claim 4, characterized in that, The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence, wherein the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

6. The camera according to any one of claims 1 to 5, characterized in that, The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

7. The camera according to claim 6, characterized in that, The light emitter is also used to emit a third light signal with a signal frequency of the second frequency to the first test point. The third light signal is reflected by the first test point to form a third reflected light signal. The third light signal intersects with the first polar line, and the third reflected light signal intersects with the second polar line. The light receiver is further configured to receive the third reflected light signal, wherein the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal are used to determine the distance between the first test point and the camera.

8. The camera according to claim 6 or 7, characterized in that, The light emitter is also used to emit a fourth light signal with a signal frequency of the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal. The fourth light signal intersects with the third pole line, and the fourth reflected light signal intersects with the fourth pole line. The light receiver is further configured to receive the fourth reflected light signal, wherein the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal are used to determine the distance between the second test point and the camera.

9. The camera according to any one of claims 1 to 8, characterized in that, The camera also includes a controller connected to the light transmitter and the light receiver respectively; The controller is used to control the light transmitter to transmit the first light signal to the first test point; The controller is also used to control the optical receiver to receive the first reflected light signal.

10. The camera according to claim 9, characterized in that, The controller is specifically used for: The first reflected light signal is sampled to obtain a first sampling result; The first phase difference between the first optical signal and the first reflected optical signal is determined based on the first sampling result; The distance between the first test point and the camera is determined based on the first phase difference.

11. A ranging camera based on Time-of-Flight (ToF), characterized in that, Includes a controller and an optical receiver connected to the controller; The optical receiver, under the control of the controller, receives a first reflected optical signal and a second reflected optical signal. The first optical signal emitted by the optical transmitter is reflected by a first test point to form a first reflected optical signal, which intersects a first epipolar line and a second epipolar line. The second optical signal emitted by the optical transmitter is reflected by a second test point to form a second reflected optical signal, which intersects a third epipolar line and a fourth epipolar line. The origin of the optical transmitter's coordinate system is a first origin, and the emission surface of the optical transmitter includes a first plane and a fourth plane. The origin of the optical receiver is a second origin, and the receiving surface of the optical receiver includes a second plane and a fifth plane; for the first test point, the plane containing the first origin and the second origin is a third plane, and the third plane intersects the first plane at the first epipolar line, and the third plane intersects the second plane at the second epipolar line; for the second test point, the plane containing the first origin and the second origin is a sixth plane, and the sixth plane intersects the fourth plane at the third epipolar line, and the sixth plane intersects the fifth plane at the fourth epipolar line; wherein, the third plane and the sixth plane are not coplanar; The controller is configured to determine the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and to determine the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

12. The camera according to claim 11, characterized in that, The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane; The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

13. The camera according to claim 11 or 12, characterized in that, The first optical signal and the second optical signal are orthogonal to each other.

14. The camera according to any one of claims 11 to 13, characterized in that, Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

15. The camera according to claim 14, characterized in that, The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence, wherein the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

16. The camera according to any one of claims 11 to 15, characterized in that, The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

17. The camera according to claim 16, characterized in that, The optical receiver is further configured to receive a third reflected optical signal under the control of the controller. The third reflected optical signal is formed by the reflection of a third optical signal through the first test point. The third optical signal intersects with the first epipolar line and the third reflected optical signal intersects with the second epipolar line. The frequency of the third optical signal is the second frequency. The controller is further configured to determine the distance between the first test point and the camera based on the first optical signal, the first reflected optical signal, the third optical signal, and the third reflected optical signal.

18. The camera according to claim 16 or 17, characterized in that, The optical receiver is further configured to receive a fourth reflected optical signal under the control of the controller. The fourth reflected optical signal is formed by the reflection of the fourth optical signal through the second test point. The fourth optical signal intersects with the third pole line. The fourth reflected optical signal intersects with the fourth pole line. The signal frequency of the fourth optical signal is the first frequency. The controller is further configured to determine the distance between the second test point and the camera based on the second optical signal, the second reflected optical signal, the fourth optical signal, and the fourth reflected optical signal.

19. The camera according to any one of claims 18, characterized in that, The controller is specifically used for: The first reflected light signal is sampled to obtain a first sampling result; The first phase difference between the first optical signal and the first reflected optical signal is determined based on the first sampling result; The distance between the first test point and the camera is determined based on the first phase difference.

20. The camera according to any one of claims 11 to 19, characterized in that, The controller is specifically used for: The second reflected light signal is sampled to obtain a second sampling result; The second phase difference between the second optical signal and the second reflected optical signal is determined based on the second sampling result; The distance between the second test point and the camera is determined based on the second phase difference.

21. A ranging method based on Time-of-Flight (ToF), characterized in that, The method is applied to a camera, the camera including a light emitter and a light receiver; the origin of the coordinate system of the light emitter is a first origin and the emitting surface of the light emitter includes a first plane and a fourth plane, the origin of the coordinate system of the light receiver is a second origin and the receiving surface of the light receiver includes a second plane and a fifth plane; the method includes: The light emitter emits a first light signal to a first test point and a second light signal to a second test point. The first light signal is reflected by the first test point to form a first reflected light signal, which intersects with a first epipolar line and a second epipolar line. The second light signal is reflected by the second test point to form a second reflected light signal, which intersects with a third epipolar line and a fourth epipolar line. Specifically, the plane containing the first origin and the second origin at the first test point is a third plane, which intersects with the first plane at the first epipolar line and with the second plane at the second epipolar line. The plane containing the first origin and the second origin at the second test point is a sixth plane, which intersects with the fourth plane at the third epipolar line and with the fifth plane at the fourth epipolar line. The third plane and the sixth plane are not coplanar. The light receiver receives the first reflected light signal and the second reflected light signal, wherein the first light signal and the first reflected light signal are used to determine the distance between the first test point and the camera.

22. The method according to claim 21, characterized in that, The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane; The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

23. The method according to claim 21 or 22, characterized in that, The first optical signal and the second optical signal are orthogonal to each other.

24. The method according to any one of claims 21 to 23, characterized in that, Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

25. The method according to claim 24, characterized in that, The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence, wherein the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

26. The method according to any one of claims 21 to 25, characterized in that, The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

27. The method according to claim 26, characterized in that, The method further includes: The light emitter emits a third light signal with a frequency of the second frequency to the first test point. The third light signal is reflected by the first test point to form a third reflected light signal. The third light signal intersects with the first polar line, and the third reflected light signal intersects with the second polar line. The optical receiver receives the third reflected light signal, wherein the first light signal, the first reflected light signal, the third light signal, and the third reflected light signal are used to determine the distance between the first test point and the camera.

28. The method according to claim 26 or 27, characterized in that, The method further includes: The light emitter emits a fourth light signal with the first frequency to the second test point. The fourth light signal is reflected by the second test point to form a fourth reflected light signal. The fourth light signal intersects with the third pole line, and the fourth reflected light signal intersects with the fourth pole line. The optical receiver receives the fourth reflected light signal, wherein the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal are used to determine the distance between the second test point and the camera.

29. The method according to any one of claims 21 to 28, characterized in that, The camera also includes a controller connected to the light transmitter and the light receiver respectively; The optical transmitter transmits a first optical signal to the first point to be tested, including: The controller controls the light transmitter to transmit the first light signal toward the first point to be tested; The optical receiver receives the first reflected light signal including: The controller controls the optical receiver to receive the first reflected light signal.

30. The method according to claim 29, characterized in that, The method further includes: The controller samples the first reflected light signal to obtain a first sampling result; The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result; The controller determines the distance between the first test point and the camera based on the first phase difference.

31. A ranging method based on Time-of-Flight (ToF), characterized in that, The method is applied to a controller, wherein the controller is included in a camera; the camera further includes a light receiver connected to the controller; the method includes: The controller controls the optical receiver to receive a first reflected light signal and a second reflected light signal. The first light signal emitted by the optical transmitter is reflected by a first test point to form a first reflected light signal, which intersects a first epipolar line and a second epipolar line. The second light signal emitted by the optical transmitter is reflected by a second test point to form a second reflected light signal, which intersects a third epipolar line and a fourth epipolar line. The origin of the optical transmitter's coordinates is a first origin, and the emission surface of the optical transmitter includes a first plane and a fourth plane. The origin of the optical receiver's coordinates is a second epipolar line. The optical receiver has two origins, and its receiving surface includes a second plane and a fifth plane; the first test point has a third plane where the first origin and the second origin are located, and the third plane intersects the first plane at the first epipolar line, and the third plane intersects the second plane at the second epipolar line; the second test point has a sixth plane where the first origin and the second origin are located, and the sixth plane intersects the fourth plane at the third epipolar line, and the sixth plane intersects the fifth plane at the fourth epipolar line; the second test point is different from the first test point; wherein, the third plane and the sixth plane are not coplanar; The controller determines the distance between the first test point and the camera based on the first light signal and the first reflected light signal, and determines the distance between the second test point and the camera based on the second light signal and the second reflected light signal.

32. The method according to claim 31, characterized in that, The light emitter includes a first light source region and a second light source region, wherein the emitting surface of the first light source region is the first plane, and the emitting surface of the second light source region is the fourth plane; The light receiver includes a first pixel array region and a second pixel array region, wherein the receiving surface of the first pixel array region is the second plane, and the receiving surface of the second pixel array region is the fifth plane.

33. The method according to claim 31 or 32, characterized in that, The first optical signal and the second optical signal are orthogonal to each other.

34. The method according to any one of claims 31 to 33, characterized in that, Both the first optical signal and the second optical signal are signals obtained by binary phase shift keying (BPSK) encoding.

35. The method according to claim 34, characterized in that, The first optical signal is a signal obtained by BPSK encoding a first pseudo-random sequence, and the second optical signal is a signal obtained by BPSK encoding a second pseudo-random sequence, wherein the number of sequences with a value of 0 in the first pseudo-random sequence is equal to the number of sequences with a value of 0 in the second pseudo-random sequence.

36. The method according to any one of claims 31 to 35, characterized in that, The first optical signal has a first frequency, the second optical signal has a second frequency, and the first frequency is different from the second frequency.

37. The method according to claim 36, characterized in that, The method further includes: The controller controls the optical receiver to receive a third reflected light signal, which is formed by the reflection of a third light signal through the first test point. The third light signal intersects with the first epipolar line and the third reflected light signal intersects with the second epipolar line. The frequency of the third light signal is the second frequency. The controller determines the distance between the first test point and the camera based on the first optical signal and the first reflected optical signal, including: The controller uses the first optical signal, the first reflected optical signal, the third optical signal, and the third reflected optical signal to determine the distance between the first test point and the camera.

38. The method according to claim 36 or 37, characterized in that, The method further includes: The controller controls the optical receiver to receive a fourth reflected optical signal; the fourth reflected optical signal is formed by the reflection of the fourth optical signal through the second test point, the fourth optical signal intersects with the third pole line, and the fourth reflected optical signal intersects with the fourth pole line; wherein, the signal frequency of the fourth optical signal is the first frequency; The controller determines the distance between the second point to be measured and the camera based on the second optical signal and the second reflected optical signal, including: The controller determines the distance between the second test point and the camera based on the second light signal, the second reflected light signal, the fourth light signal, and the fourth reflected light signal.

39. The method according to any one of claims 38, characterized in that, The controller determines the distance between the first test point and the camera based on the first optical signal and the first reflected optical signal, including: The controller samples the first reflected light signal to obtain a first sampling result; The controller determines the first phase difference between the first optical signal and the first reflected optical signal based on the first sampling result; The controller determines the distance between the first test point and the camera based on the first phase difference.

40. The method according to any one of claims 31 to 39, characterized in that, The controller determines the distance between the second point to be measured and the camera based on the second optical signal and the second reflected optical signal, including: The controller samples the second reflected light signal to obtain a second sampling result; The controller determines a second phase difference between the second optical signal and the second reflected optical signal based on the second sampling result; The controller determines the distance between the second test point and the camera based on the second phase difference.

41. A computer-readable storage medium, characterized in that, The medium stores instructions that, when executed by a computer, implement the method of any one of claims 21 to 40.

42. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 21 to 40.

43. A chip, characterized in that, The chip includes a processor and a communication interface; wherein the communication interface is coupled to the processor, and the processor is used to run computer programs or instructions to implement the method as described in any one of claims 21 to 40.

Images