Distance image sensor

Abstract

To provide a distance image sensor which can self-diagnose synchronization gaps in an optical scan and make a self-recovery.SOLUTION: A distance image sensor 100 forms a distance image based on reception data indicating the time difference between light emission and light reception acquired through optical scanning. A pixel number obtained by associating one light emission in the optical scanning with one pixel position in the distance image is set in advance, and a pixel-number assigning unit PG assigns the pixel number to the acquired reception data. The sensor further includes a pixel-number matching unit PV for performing matching of pixel numbers for each frame of reception data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a range image sensor that generates a range image that measures the distance to a monitoring target based on the reflected component of light emitted toward the monitoring target. [Background technology]

[0002] For example, a known distance measuring device that measures distance by optical scanning uses a switching mechanism that switches between transmission and reflection to perform self-diagnostic distance measurements at a specified location within the device, and performs fault diagnosis based on changes in the distance measurement state (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-91221 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned Patent Document 1, a dedicated switching mechanism is required, and there is also a possibility that an abnormality may be overlooked depending on how synchronization is lost in the optical scanning.

[0005] The present invention has been made in view of the above-mentioned points, and an object of the present invention is to provide a range image sensor that can self-diagnose a deviation in synchronization in optical scanning and automatically recover from the deviation. [Means for solving the problem]

[0006] The distance image sensor for achieving the above-mentioned object is a distance image sensor that forms a distance measurement image based on received data indicating the time difference between light emission and light reception obtained by optical scanning, and is equipped with a pixel number assignment unit that sets a pixel number that corresponds one light emission in optical scanning to one pixel position in the distance measurement image and assigns a pixel number to the acquired received data, and a pixel number matching unit that matches the pixel number for each frame of received data.

[0007] In the above-mentioned range image sensor, pixel numbers corresponding to pixel positions are set, and the pixel numbers are assigned to the received data. By comparing the pixel numbers for each frame of received data, any deviation in synchronization timing during optical scanning can be easily and reliably detected, enabling detection and repair of abnormalities and improving the reliability of the device.

[0008] In a specific aspect of the present invention, the pixel number comparison unit determines that there is a distance measurement abnormality when there is a mismatch in pixel numbers. In this case, it is possible to quickly and accurately determine the abnormality based on the comparison of pixel numbers.

[0009] In another aspect of the present invention, if the pixel number comparison unit determines that there is a distance measurement abnormality, the received data acquired before the start of optical scanning for a new frame is erased. In this case, it is possible to avoid using the received data that is in an abnormal state and to start acquiring received data that is updated frame by frame.

[0010] In yet another aspect of the present invention, a received data storage unit is provided that sequentially stores the received data to which pixel numbers have been assigned by the pixel number assignment unit, and the received data storage unit outputs the received data to the pixel number verification unit on a first-in, first-out basis. In this case, it is possible to avoid using received data that is in an abnormal state and to start obtaining received data that has been updated frame by frame.

[0011] In yet another aspect of the present invention, two-dimensional scanning is performed when forming a distance measurement image by optical scanning, which makes it possible to capture the target range planarly and maintain accurate synchronous operation.

[0012] In yet another aspect of the present invention, a scanning control unit performs scanning control based on a timing table that associates the timing of light emission from the light source with the scanning direction due to mirror reflection from the start to end of optical scanning for one frame, the scanning control unit assigning pixel numbers to data on the timing of light emission in the timing table, and the pixel number assigning unit assigning pixel numbers to received data based on the timing table. In this case, it can be quickly and accurately determined whether scanning control is being maintained appropriately based on the timing table. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an overview of an example of the configuration of a range image sensor according to an embodiment; [Figure 2] FIG. 2 is a block diagram for explaining the details of the functions of each part in one configuration example of a range image sensor. [Figure 3] (A) is a data table showing an example of a time table among various data handled in the operation of each part, and (B) and (C) are data tables showing examples of received data. [Figure 4] (A) to (D) are data tables showing examples of various data that have been simplified for the sake of simplicity. [Figure 5] 5 is a conceptual diagram for explaining how data is handled in the determination process in the example shown in FIG. 4. FIG. [Figure 6] 10A and 10B are conceptual diagrams for explaining an example of normal operation. [Figure 7] 10A and 10B are conceptual diagrams for explaining an example of the operation during an abnormality. [Figure 8] FIG. 10 is a conceptual diagram for explaining an example of how operation is restored. [Figure 9] 10 is a flowchart illustrating an operation of the range image sensor for performing a process of determining whether an abnormality has occurred. [Figure 10] FIG. 1 is a conceptual diagram outlining the characteristics of a range image sensor. [Figure 11] 10A is a data table for explaining the operation mode of one modified example, and FIG. 10B is a block diagram. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of a range image sensor according to an embodiment will be described below with reference to Fig. 1 etc. As shown in the block diagram of Fig. 1, the range image sensor 100 of this embodiment includes an optical unit 20, a control unit 30, a data management unit 40, and a range image (distance image) formation unit 50, and measures distance by projecting laser light, which is an electromagnetic wave, over a predetermined scanning range and receiving the returned light component, and generates a range image based on the acquired range data.

[0015] The optical unit 20 includes a light projecting unit 21, a light scanning unit 22, and a light receiving unit 23 to perform optical scanning in accordance with instructions from the control unit 30. Here, the range image sensor 100 performs two-dimensional scanning when forming a distance measurement image (range image) by optical scanning.

[0016] In the optical unit 20, the light projecting unit 21 is configured to include, for example, a laser light source, a lens, etc., and is a laser light projecting unit that emits and projects laser light (pulsed light) PL in accordance with instructions from the control unit 30.

[0017] The optical scanning unit 22 is a device for scanning the laser light PL from the light projecting unit 21 in a predetermined scanning range including an object OB to be used as a target in monitoring, for example, in synchronization with the light projecting unit 21, in accordance with a command from the control unit 30. The optical scanning unit 22 can be configured with, for example, a two-dimensional scanning mirror (scanner), and scans light toward the object OB. In this embodiment, a Lissajous scanning type is used as an example.

[0018] The light receiving unit 23 is a laser light receiving unit that receives reflected light RL as a reflected component of the laser light PL emitted from the light scanning unit 22, which becomes scanned light by the light scanning unit 22. The light receiving unit 23 converts the reflected light RL into, for example, a detectable pulse wave, and outputs the converted received data to the control unit 30. Here, the light receiving unit 23 converts the received data as a result of receiving the reflected light RL into data equivalent to time difference data indicating the time difference between light emission and light reception obtained by light scanning, and outputs the converted data to the control unit 30.

[0019] The control unit 30 is a scanning control unit OC that controls various operations of each part of the optical system that constitutes the optical unit 20. Specifically, it outputs a command signal to the light projecting unit 21 to control the projection timing of the laser light PL, and outputs various command signals to the optical scanning unit 22 to cause it to perform two-dimensional scanning (Lissajous scanning) synchronized with the projection timing. The control unit 30 outputs command signals in accordance with a time table (timing table) TT to synchronize the projection timing with the Lissajous scanning.

[0020] The control unit 30 also receives the above-mentioned received data (time difference data) as information on the light reception result from the light receiving unit 23. In particular, in this embodiment, pixel numbers are set in advance to correspond to the positions of each pixel that constitutes the distance measurement image to be formed. Since each pixel position is determined in relation to the light projection timing, pixel numbers are assigned in relation to the light projection timing, and the pixel number assigning unit PG assigns the corresponding pixel number to the received data acquired during the above-mentioned data conversion.

[0021] In the above cases, it can be said that the control unit 30 is capable of acquiring the time (time difference) from the projection of the laser light PL to the reception of the reflected light RL as its reflected component, or the distance value calculated based on this and data (ranging data) regarding the projection direction.

[0022] The control unit 30 outputs the received data information, which is the light reception result from the light receiving unit 23 acquired as described above, to the data management unit 40.

[0023] The data management unit 40 manages data from the control unit 30. The data management unit 40 accepts received data sequentially generated by the control unit 30 and outputs the received data for each ranging image, i.e., each frame, in units of the number of pixels to the ranging image (distance image) formation unit 50 on a first-in, first-out basis. The data management unit 40 functions as a received data storage unit ST that sequentially stores the received data to which pixel numbers have been assigned.

[0024] The distance image (distance image) forming unit 50 forms a corresponding distance image (distance image) based on the data for each frame output from the data management unit 40. In particular, in this embodiment, the distance image forming unit 50 stores information about the pixel numbers described above, and is provided with a pixel number checking unit PV that checks the received data for one frame that has been output when forming the image, making it possible to determine whether or not an abnormality has occurred during the data generation process.

[0025] Here, when generating a distance image (distance measurement image) using the above-described distance image sensor 100, if it operates continuously, for example, 24 hours a day, 365 days a year, various factors can cause errors in synchronization timing, resulting in incorrect data being acquired. In contrast, in this embodiment, pixel numbers that are previously set in correspondence with pixel positions are assigned when received data is acquired, and these numbers are compared on a frame-by-frame basis when forming a distance measurement image (distance image), so that any errors in synchronization timing during optical scanning can be easily and reliably detected.

[0026] Hereinafter, with reference to FIG. 2, a series of operations from optical scanning in the range image sensor 100 to data generation and acquisition will be described together with details of the configuration of each part.

[0027] As shown in the figure, in the optical unit 20, first, the light projection unit 21 includes a light emitting unit 21e for emitting (projecting) laser light, and a driver 21d for operating the light emitting unit 21e in accordance with a time table TT. In addition, the light projection unit 21 includes a timing detection unit 21t for detecting the timing of laser emission in the light emitting unit 21e.

[0028] The optical scanning unit 22 includes a MEMS mirror MM that reflects the light from the light projecting unit 21 in the direction of emission, various drivers 22d that send drive signals to the MEMS mirror MM to cause it to move in accordance with two-dimensional scanning and receive detection signals for operation confirmation (posture confirmation), and various converters 22c that perform signal conversion.

[0029] As described above, the light receiving unit 23 includes a light receiving element (photodiode) 23r including a transformer, a light amount detection unit 23q that detects the amount of light from the detection result of the light receiving element 23r, a first receiving circuit 221d (receiving system 1), a second receiving circuit 222d (receiving system 2), etc., in order to generate time difference data as reception data from the reception result of the reflected light RL, which is the reflected component. Each of the receiving circuits 221d, 222d (receiving systems 1, 2) includes an amplifier, comparator, converter, etc., and performs reception confirmation according to the magnitude (gain) of the reflected component as the detection result of the light receiving element 23r, and compares it with the detection result of the emission timing by the timing detection unit 21t to calculate the time difference from the emission (transmission) time to the light reception (reception) time, and outputs reception data (time difference data) including the calculation result, etc. to the control unit 30. In the illustrated example, if the value of the reflected component is large, it is detected in the first receiving circuit 221d (receiving system 1), and if the value of the reflected component is small, it is detected in the second receiving circuit 222d (receiving system 2).

[0030] Next, the control unit 30 is a scanning control unit OC that issues various command signals during a series of operations from optical scanning to data generation and acquisition, and controls the series of scans. In addition to the time table TT described above, the control unit 30 includes, for example, an optical scanning control unit 31 that controls optical scanning (the direction of light emission), a timing management unit 32 that manages the timing of optical scanning and light emission, and a time difference data collection unit 33 that acquires various data for collecting time difference data. Note that the time table TT includes information on synchronization timing (number of clocks) to synchronize, for example, the attitude control of the MEMS mirror MM to change the emission direction by optical scanning with the irradiation (emission) of laser light PL by the light-emitting unit 21e of the light-projecting unit 21. In other words, the information on synchronization timing (number of clocks) links the operation of the MEMS mirror MM with the operation of the light-emitting unit 21e.

[0031] The optical scanning control unit 31 outputs various command signals to the optical scanning unit 22 for driving the MEMS mirror MM based on the time table TT, and also receives detection signals relating to the operating status of the MEMS mirror MM.

[0032] The timing management unit 32 manages the timing (synchronization) of the attitude control by the optical scanning control unit 31 based on the time table TT and the light emission in the light projecting unit 21 corresponding to the light emission trigger based on the time table TT.

[0033] The time difference data collection unit 33 acquires time difference data as received data generated from the reception results in the light receiving unit 23. The time difference data collection unit 33 also acquires information regarding the timing of optical scanning and light emission set in the time table TT and the status of the actual laser emission timing detected by the timing detection unit 21t of the light projecting unit 21. The time difference data collection unit 33 also has a pixel number assignment unit PG, which assigns pixel numbers to the acquired data based on this information, i.e., the assumed synchronization data in the time table TT, the actual emission timing data, and the received data (time difference data) related to the reception timing. The pixel numbers are associated with each pixel position in a range image (distance image) and also correspond to one light emission from the light projecting unit 21 during optical scanning by the optical unit 20. The pixel number assignment unit PG assigns the set pixel number to the corresponding received data. In other words, in this embodiment, the pixel number assigning unit PG assigns pixel number information to the received data (time difference data).

[0034] In addition to the above, data management unit 40 is provided with, for example, a FIFO data management unit 41 and a FIFO controller 42. FIFO data management unit 41 sequentially receives received data (time difference data) in pixel units to which pixel numbers have been assigned in time difference data collection unit 33, and uses the FIFO method to group the data in frame units and output it to ranging image (distance image) formation unit 50. At this time, FIFO controller 42 counts the number of data items to be output, and when it confirms based on this count that one frame's worth of received data (time difference data) has accumulated in FIFO data management unit 41, it outputs this data as a group to ranging image (distance image) formation unit 50.

[0035] Distance image (distance image) forming unit 50 includes distance image generation determination processing unit 51 for forming a distance image (distance image) based on the data output from data management unit 40. Distance image generation determination processing unit 51 accepts data output frame by frame from data management unit 40 and performs image formation processing based on this data. However, here, as a prerequisite for this image formation, a determination process is performed to determine whether or not an abnormality has occurred in the accepted data. For this reason, distance image generation determination processing unit 51 is provided with the pixel number matching unit PV described above.

[0036] In distance image generation determination processing section 51, the reception section (receiving section) that receives received data sent frame by frame from data management section 40 has a data intake port to which preset pixel numbers for one frame are assigned, and pixel number comparison section PV compares the pixel numbers assigned to this data intake port with the pixel numbers assigned to the received data taken into the data intake port. In other words, pixel number comparison section PV compares the pixel numbers for each frame of received data.

[0037] Hereinafter, with reference to FIG. 3 etc., an example of a method for determining whether or not there is an abnormality (misalignment) in the synchronization of operations using pixel numbers in the series of operations described above will be described.

[0038] Fig. 3(A) is a data table showing an example of a time table TT among various data handled in the operation of each section, and Fig. 3(B) and Fig. 3(C) are data tables showing examples of received data. The data table RD shown in Fig. 3(B) shows a list of received data that is extracted sequentially, and the data table RDn shown in Fig. 3(C) shows the contents of data included in individual received data. Note that here, the received data n received nth by the light receiving section 23 is represented as received DTn.

[0039] 3A, the time table TT is linked by a clock number that indicates a numerical value of time as a reference for synchronizing the timing of irradiation of the laser light PL by the light projector 21 with the posture of the MEMS mirror MM in the optical scanning unit 22, i.e., the irradiation direction. That is, the posture of the MEMS mirror MM in the optical scanning unit 22 and the irradiation of the laser light PL by the light projector 21 operate in synchronization with each other so that the laser light PL is irradiated in a direction corresponding to the image position indicated by the X and Y coordinates at the synchronization timing (clock number).

[0040] As a result of light being emitted at the synchronized timing as described above and the corresponding reflected components being received (received) by the light receiving unit 23, received data such as that shown in FIG. 3B is sequentially acquired. In particular, in this embodiment, as described above, pixel numbers are assigned to the received data. The illustrated example shows how pixel numbers are assigned to each piece of generated received data. As shown in FIG. 3C, each piece of received data may include, for example, information indicating the amount of light, the reception results of each receiving system 1, 2, and their status in the light receiving unit 23. For example, as described above, data including calculation results regarding the time difference is generated as received data (time difference data). The illustrated example shows a case where one distance image is composed of 1,000 pixels or more.

[0041] 4(A) to 4(D) are simplified to simplify the explanation of the various data described above. Specifically, FIG. 4(A) shows the distance image GDs formed based on the scan with 4 × 4 = 16 pixels, which is fewer than the number of pixels in the actual distance image, and FIG. 4(B) shows the time table TTs corresponding to the 16 pixels in FIG. 4(A). As mentioned above, since Lissajous-type optical scanning is performed here, as shown in FIG. 4(A), the pixel positions corresponding to the nth received received data n (received DTn) are not in numerical order of the coordinates but are in order according to the movement of the Lissajous scan. Note that in FIG. 4(B), for ease of understanding, the clock count indicating the synchronization timing is shown to be synchronized with the received data n (received DTn), and the X and Y coordinates are shown in order.

[0042] Figures 4(C) and 4(D) correspond to Figures 3(B) and 3(C), and illustrate the state in which pixel numbers are assigned to each piece of received data when a distance image is formed using 4 × 4 = 16 pixels. In this case, for example, 16 pieces of received data 1 through 16, i.e., 16 pieces of time difference data, form one distance image (one frame of image) arranged as shown in Figure 4(A). At this time, pixel numbers 1 through 16 are assigned to received data 1 through 16, respectively. By repeating the same process, distance images are sequentially formed. That is, the next distance image is formed using data from the 17th received data 17 onward. At this time, pixel numbers 1 through 16 are reassigned. For example, pixel number 1 is assigned to the 17th received data 17. This means that the 17th received data 17 is treated as time difference data (distance data) corresponding to pixel number 1, i.e., the coordinate position (X=0, Y=0) corresponding to clock count 1. If we look at the above from another perspective, we can say that 16 pixel numbers are preset in association with 16 pixel positions.

[0043] In this embodiment, the pixel numbers assigned to the received data as described above are used to verify (determine) whether or not there is an abnormality in the generated data.

[0044] An example of how received data is handled in data determination processing will be described below with reference to Fig. 5. That is, an example of processing performed by distance image generation determination processing unit 51 and the pixel number matching unit PV (see Fig. 2) included therein will be described. Note that Fig. 5 will also be described based on the simplified example of various data shown in Fig. 4.

[0045] Fig. 5(A) conceptually illustrates a data input port DA for inputting received data grouped in frame units into the distance image generation determination processor 51, and Fig. 5(B) illustrates the state in which a received data group RG1 for one frame has been input into the data input port DA. As shown in Fig. 5(A), pixel numbers 1 to 16 (determination-side pixel numbers 1 to 16) corresponding to the pixel positions of each pixel constituting the distance image GDs illustrated in Fig. 4(A) are preset in the data input port DA, and the received data transmitted in frame units, i.e., in groups of 16, are accepted. More specifically, as shown in Fig. 5(A), 16 data insertion sections DS1 to DS16 are fixedly provided in the data input port DA corresponding to pixel numbers 1 to 16 (determination-side pixel numbers 1 to 16), and received data group RG1 is input into these sections for each received data, as shown in Fig. 5(B). Once the data is captured, the pixel number comparison unit PV (see Figure 2) compares the pixel numbers on the side generating the received data with the pixel numbers on the side determining the data. That is, if all pixel numbers 1 to 16 match, the data is treated as normal.

[0046] An example of how the range image sensor 100 operates normally will be described below with reference to the conceptual diagram shown in FIG. 6, and an example of how the range image sensor 100 operates abnormally will be described with reference to the conceptual diagram shown in FIG. 7.

[0047] Fig. 6(A) is a chart showing the operation of each unit in chronological order, and Fig. 6(B) is a conceptual diagram showing how received data is handled.

[0048] First, as shown in Figure 6(A), a Lissajous reference signal is used as the starting reference for forming one frame image, and the mirror (MEMS mirror MM) of the optical scanning unit 22 is driven in two directions (X and Y directions) to perform Lissajous scanning, while the light projection unit 21 emits laser light in accordance with timing data. Note that since the Lissajous scanning operates in a sinusoidal manner in the two directions, the timing data values, i.e., the time intervals between each light irradiation, are different so that light irradiation is performed at the appropriate pixel position, as shown in the figure. The above operations are performed at predetermined timings according to a time table TT.

[0049] Next, the light receiving unit 23 receives reflected light corresponding to each light irradiation described above. The figure shows a situation in which the reflected light is received a certain amount of time after the laser irradiation timing. When the reflected light reception is confirmed (including the case where the reflected light is not detected within a predetermined time and is treated as being at infinity), the time difference between the laser irradiation timing and the reflected light reception is calculated, and time difference data (received data or a part thereof) is generated. The generated time difference data is collected by the control unit 30 (time difference data collecting unit 33) with pixel numbers assigned, and output to the FIFO data management unit 41 of the data management unit 40. At this time, the data management unit 40 also counts the number of data items in the FIFO controller 42 to group the received data for each frame (as a group of received data for one frame) on a first-in, first-out basis. When one frame's worth of data (e.g., 16 pixels' worth) has been accumulated, data management unit 40 (FIFO data management unit 41) outputs the data (one frame's worth of received data group) to distance image (distance image) formation unit 50. Distance image (distance image) formation unit 50 generates a distance image based on the data in distance image generation determination processing unit 51, which also functions as a distance image generation unit. As a prerequisite for this, the data verification process described above is performed to check for any abnormalities in the received data.

[0050] The handling of received data as described above will be confirmed with reference to Fig. 6(B). In the figure, the upper part shows how received data is handled in FIFO data management unit 41 of data management unit 40, and the lower part shows how received data is handled in distance image generation determination processing unit 51 or pixel number comparison unit PV.

[0051] First, as shown in the upper part, received data is sequentially accumulated in FIFO data management section 41 of data management section 40. When one frame's worth of data (e.g., 16 pixels) has been accumulated, that is, when the data for the set pixel number has been accumulated, the received data is output together to distance image generation determination processing section 51 (in the lower part of the figure), and the data is then deleted from FIFO data management section 41 (FIFO method).

[0052] Meanwhile, as shown in the lower part, in the distance image generation determination processing unit 51 (pixel number matching unit PV) on the distance image generation side, the data insertion unit DSn (1≦n≦16) of the data input port DA is empty before the output from the data management unit 40 is received, but when the output from the data management unit 40 is received, the pixel number matching unit PV checks whether the pixel numbers match. If the series of processes related to obtaining the received data as described above are performed normally, the pixel numbers on the received data generation side and the pixel numbers on the determination side will match perfectly, the data will be determined to be normal, and the distance image will be generated based on that data.

[0053] On the other hand, as described above, various factors may cause errors in the synchronization timing etc. in the range image sensor 100. A typical example is as shown in FIG.

[0054] An example of the operation during an abnormality will be described below with reference to Fig. 7. Fig. 7(A) corresponds to Fig. 6(A) and is a chart showing the operation of each part in chronological order. Fig. 7(B) corresponds to Fig. 6(B) and is a conceptual diagram showing how received data is handled.

[0055] The illustrated example shows a situation in which a problem occurs when data is stored in data management unit 40 (FIFO data management unit 41). More specifically, data management unit 40 does not receive received data DT3, which should be the third received data, resulting in data loss. That is, in this case, on the received data generation side, an object to be assigned pixel number 3 does not exist, and is therefore lost. Therefore, data for one frame (e.g., 16 pixels) is output to distance image generation determination processing unit 51 in a state in which up to the 17th received data, assigned pixel number 1, has been received due to the shift caused by the loss of pixel number 3.

[0056] On the other hand, as mentioned above, the distance image generation determination processor 51 on the determination side has a fixed pixel number setting in advance, and therefore has a predetermined number (for example, 16) of data insertion units DS1 to DS16, including an acceptance unit for pixel number 3. Therefore, when a received data group with data loss as described above is accepted, the pixel numbers on the received data generation side and the pixel numbers on the determination side will not match. In other words, the distance image generation determination processor 51 will detect the occurrence of a data abnormality. In other words, if the pixel number comparison unit PV detects a mismatch in pixel numbers during the above processing, it will determine that there is a ranging abnormality in this case.

[0057] An example of how the range image sensor 100 of this embodiment returns to operation will be described below with reference to the conceptual diagram shown in FIG.

[0058] FIG. 8, corresponding to FIG. 7A, is a chart showing the operation of each component in chronological order during processing after data loss occurs. When data loss occurs, the data captured frame by frame becomes out of sync, i.e., a mismatch occurs. This mismatch persists in subsequent data (data groups). Therefore, here, data generated after the data loss occurs is also temporarily rendered unusable. Specifically, if an abnormality (distance measurement abnormality) is detected, the received data acquired before the start of a new frame of optical scanning is deleted. In the example shown in FIG. 8, if a data abnormality is detected as a result of comparison (image data comparison) by the pixel number comparison unit PV of the distance image generation determination processor 51, the distance image generation determination processor 51 clears the time difference data corresponding to the data and outputs information indicating that the comparison result is NG to the data management unit 40, disabling writing to the FIFO data management unit 41 from the control unit 30. At this time, for example, the number of data counted in the FIFO controller 42 is also cleared, and a series of processes relating to the received data is resumed upon receipt of a Lissajous reference signal indicating the starting reference for forming a new frame image.

[0059] An example of the operation of the range image sensor 100 to perform the determination process for the occurrence of the above-mentioned abnormality will be described below with reference to the flowchart shown in FIG.

[0060] First, when various operations for Lissajous scanning are started in each section, in the distance image (distance image) forming section 50, the distance image generation determination processing section 51 starts reading the time table TT (step S101) and records the pixel number for distance image generation in memory according to the recorded contents of the time table TT (step S102), i.e., the determination side pixel number is set and the data input port DA is configured.

[0061] Once the distance image generation determination processor 51 is ready to accept received data, it waits for an interrupt from the FIFO controller 42 of the data management unit 40 (step S103) and checks whether an interrupt has occurred (step S104). Specifically, the distance image generation determination processor 51 checks whether received data (time difference data) equivalent to one frame of pixels has been input. If there is no interrupt (step S104: No), it waits until an interrupt is detected (step S103). If there is an interrupt (step S104: Yes), it acquires the data, i.e., the time difference data (step S105), imports it into the data input port DA, and compares the pixel numbers on the generated side (pixel numbers included in the data from the FIFO data management unit 41) with the pixel numbers on the determination side (pixel numbers for generating the distance image) (step S106) to check whether they match (step S107). If they match in step S107 (step S107: Yes), the determination process ends. In this case, a distance image is generated based on the data that is the subject of the determination.

[0062] On the other hand, if there is no match in step S107 (step S107: No), the distance image generation determination processor 51 performs a process to delete the received data and any subsequent received data. Specifically, first, the distance image generation determination processor 51 clears the determination-side pixel number (pixel number for distance image generation), which is the current data (step S108). The distance image generation determination processor 51 also outputs an instruction to the data management unit 40 to disable writing from the control unit 30 to the FIFO data management unit 41 (step S109), and outputs an instruction to clear the current received data in the FIFO data management unit 41 (step S110), thereby completing the series of processes. In this case, distance image generation based on the data that was the subject of the determination is not performed. Furthermore, as described with reference to FIG. 8, after clearing the various data, the process resumes after waiting for the start criterion for forming one frame image to be met.

[0063] Below, an overview of the characteristics of the range image sensor 100 of this embodiment will be described with reference to the conceptual diagram shown in FIG.

[0064] As shown in the figure and as described above, the range image sensor 100 of this embodiment forms a range image (range image) GD based on received data (time difference data) DD indicating the time difference between light emission and light reception obtained by optical scanning in the optical unit 20. Here, a pixel number PN corresponding to each pixel PX constituting the range image GD is assigned to each position on the range image GD. That is, a pixel number PN is assigned to each pixel position on the range image GD. This pixel number PN also corresponds to each light emission in the optical scanning. In addition, in the range image sensor 100, the pixel number assigning unit PG assigns the pixel number assigned as described above to the corresponding received data. Furthermore, the pixel number comparing unit PV compares the pixel number PN for each frame of received data, i.e., for each piece of range data corresponding to one range image GD. Regarding the pixel number PN matching process by the pixel number matching unit PV, it is possible to pre-set a fixed pixel number PN corresponding to each position on the ranging image GD (say, pixel number PNα) on the judgment side of the pixel number matching unit PV.

[0065] If there is a mismatch between the pixel number PN and the pixel number PNα, for example, the pixel number matching unit PV determines that there is an abnormality in the received data obtained by distance measurement (determines that there is a distance measurement abnormality), and if it determines that there is a distance measurement abnormality, it is possible to avoid using the abnormal received data, for example, by erasing the received data obtained before the start of a new frame of optical scanning.

[0066] 〔others〕 The present invention is not limited to the above-described embodiment, and can be embodied in various forms without departing from the spirit and scope of the present invention.

[0067] First, the method described above is merely an example, and in addition to the above embodiment, pixel numbers may be set in a time table (timing table) TTα as shown in Fig. 11(A) and assigned to received data based on this. That is, as shown in Fig. 11(B), when the control unit 30 serving as the scanning control unit OC performs scanning control and other various operational controls based on a time table TTα that associates the timing of light emission from the light source with the scanning direction due to mirror reflection from the start to end of optical scanning of one frame, pixel numbers may be assigned to data on the timing of light emission in the time table TTα, and the pixel number assigning unit PG may assign pixel numbers to received data from the light receiving unit 23 based on the time table TTα.

[0068] Furthermore, while the above description has been given of an example of a cause of an abnormality, such as data loss during time difference data generation, the abnormality can be confirmed in various ways based on pixel number matching. For example, in FIG. 6A, which shows an example of normal operation, the timing data for the emission timing according to the scanning control shown in the fourth row from the top is 400, 700, 200, etc. If this data becomes out of sync and becomes 400, 1400, 200, etc., the emission timing will be delayed. If such an abnormality occurs, it will ultimately lead to a situation where the received data cannot be acquired. In this case, the received data (time difference data) and the pixel numbers assigned to it will also be impossible to acquire, and as a result, the matching process will not be performed properly, and the abnormality will be detected.

[0069] Furthermore, various types of light sources and light receiving elements for optical scanning can be used, and for example, an APD (avalanche photodiode) or a SPAD (single photon avalanche diode) can be used.

[0070] Furthermore, various types of optical scanning are also envisioned, for example, raster-type two-dimensional scanning is also envisioned, and in this case, the present application can also be applied. It is also conceivable that the present application can be applied to one-dimensional scanning. [Explanation of symbols]

[0071] 20...optical unit, 21...light-emitting unit, 21d...driver, 21e...light-emitting unit, 21t...timing detection unit, 22...light scanning unit, 22c...converter, 22d...driver, 23...light-receiving unit, 23q...light amount detection unit, 23r...light-receiving element (photodiode), 30...control unit, 31...light scanning control unit, 32...timing management unit, 33...time difference data collection unit, 40...data management unit, 41...FIFO data management unit, 42...FIFO controller, 50...range measurement image (distance image) formation unit, 51...distance image generation determination processing unit, 100...distance image Sensor, 221d, 222d...receiving circuit, DA...intake port, DD...received data (time difference data), DS1 to DS16, DSn...data insertion unit, GD...range image (distance image), GDs...distance image, MM...MEMS mirror, OB...object, OC...scanning control unit, PG...pixel number assignment unit, PL...laser light, PN, PNα...pixel number, PV...pixel number matching unit, PX...pixel, RD...data table, RDn...data table, RG1...received data group, RL...reflected light, ST...received data storage unit, TT, TTs, TTα...time table

Claims

1. A distance image sensor that forms a distance measurement image based on reception data indicating a time difference between light emission and light reception obtained by optical scanning, a pixel number assigning unit that assigns a pixel number to the acquired received data, the pixel number being associated with one light emission in the optical scanning and one pixel position in the distance measurement image; a pixel number collating unit that collates the pixel numbers for each frame of the received data; A distance image sensor comprising:

2. The range image sensor according to claim 1 , wherein the pixel number comparison unit determines that there is an abnormality in distance measurement when there is a mismatch in the pixel numbers.

3. 3. The range image sensor according to claim 2, wherein, when the pixel number collation unit determines that there is an abnormality in the distance measurement, the received data acquired before the start of optical scanning for a new frame is deleted.

4. a received data storage unit that sequentially stores the received data to which the pixel numbers have been assigned by the pixel number assigning unit, 2. The range image sensor according to claim 1, wherein the received data storage section outputs the received data to the pixel number comparison section for each frame on a first-in, first-out basis.

5. 2. The range image sensor according to claim 1, wherein two-dimensional scanning is performed when forming the range image by optical scanning.

6. a scanning control unit that performs scanning control based on a timing table that associates the timing of light emission from the light source with the scanning direction by mirror reflection from the start to the end of optical scanning of one frame, In the scanning control unit, the pixel number is assigned to data of the timing of light emission in the timing table, The range image sensor according to claim 1 , wherein the pixel number assigning unit assigns the pixel number to the received data based on the timing table.

Images