Information processing device, information processing method, and information processing program

Abstract

An information processing device according to the present disclosure comprises: a setting unit for setting an expansion region which is a pixel region where the intensity of reflected laser light can be observed around an activation region which is a pixel region where the intensity of the reflected light can be observed in a light-receiving element; and a determination unit that uses the intensity of the reflected light observed from the pixels of the set expansion region as a basis to determine whether reflected light from a highly reflective object is included in the reflected light.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] Distance measurement technologies such as LiDAR (Light Detection and Ranging) are known that measure the distance from a position to an object based on the time of flight (ToF), which is the time it takes for laser light irradiated on the object to be reflected by the object and return to a position corresponding to the light source of the laser light. In these distance measurement technologies, to measure the ToF, a light receiving element such as a single photon avalanche diode (SPAD) that receives reflected light is provided at a position corresponding to the light source.

[0003] In the above-described distance measurement technology, scattering of reflected light from a highly reflective object causes flare, a phenomenon in which the reflected light leaks into pixels surrounding the pixel of the light receiving element on which the reflected light hits. When flare occurs, the scattered light from the highly reflective object may be determined to be reflected light from a false object. In this case, accurate distance measurement cannot be performed even if output data output from the pixel where flare occurs is used as distance measurement data, resulting in reduced distance measurement accuracy. For this reason, techniques for removing flare are known.

[0004] International Publication No. 2022 / 190770

[0005] In conventional technology, if the laser light could be irradiated so that the reflected light would return only to the effective area, which is the area of ​​the pixel where the intensity of the reflected laser light can be observed, flare could be eliminated, preventing a decrease in distance measurement accuracy. However, it is difficult to irradiate the laser light so that the reflected light returns only to the effective area. For this reason, there is room for improvement in the conventional technology.

[0006] Therefore, the present disclosure proposes an information processing device and the like that can further prevent a decrease in distance measurement accuracy.

[0007] In order to solve the above problem, the information processing device according to the present disclosure includes a setting unit that sets an extended area, which is an area of ​​pixels in which the intensity of reflected light of laser light can be observed, around an enabled area, which is an area of ​​pixels in the light receiving element in which the intensity of the reflected light can be observed, and a determination unit that determines whether the reflected light includes reflected light from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended area.

[0008] 1 is a diagram illustrating an example of a distance measurement technology. FIG. 1 is a diagram illustrating the background of an information processing system according to an embodiment. FIG. 2 is a diagram illustrating the purpose of an information processing system according to an embodiment. FIG. 3 is a diagram illustrating an overview of an information processing system according to an embodiment. FIG. 4 is a block diagram illustrating an example configuration of an information processing system according to a first embodiment. FIG. 5 is a flowchart illustrating an example procedure of information processing by an information processing device according to a first embodiment. FIG. 6 is a block diagram illustrating an example configuration of an information processing system according to a second embodiment. FIG. 7 is a diagram illustrating a first setting example of an extended region. FIG. 8 is a diagram illustrating a second setting example of an extended region. FIG. 9 is a diagram illustrating a third setting example of an extended region. FIG. 10 is a block diagram illustrating an example configuration of an information processing system according to a third embodiment. FIG. 11 is a diagram illustrating a determination of whether reflected light observed from pixels in a valid region and reflected light observed from pixels in the extended region includes reflected light from a white object. FIG. 12 is a diagram illustrating a comparison result between a case where reflected light from only a highly reflective object hits a pixel region and a case where reflected light from a highly reflective object and a white object hits a pixel region. FIG. 13 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of an information processing device according to the present disclosure.

[0009] Hereinafter, embodiments will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0010] The present disclosure will be described in the following order: 1. Overview of information processing system 1-1. Example of distance measurement technology 1-2. Background of information processing system 1-3. Purpose of information processing system 1-4. Overview of information processing system 2. First embodiment 2-1. Configuration of information processing system 2-2. Information processing procedure 3. Second embodiment 4. Third embodiment 5. Other embodiments 6. Effects of information processing device according to the present disclosure 7. Hardware configuration 8. Supplementary information

[0011] (1. Overview of Information Processing System) (1-1. Example of Ranging Technology) The information processing system according to the embodiment is applied to solve problems with ranging technology. Therefore, as a premise for explaining the information processing system according to the embodiment, an example of ranging technology will be explained using FIG. 1. FIG. 1 is a diagram showing an example of ranging technology.

[0012] 1 uses a D (Direct) ToF method in which a pulsed laser beam 100 is irradiated onto an object 200, and the distance from a light receiving element 300 to the object 200 is measured based on the ToF, which is the time it takes for the reflected light from the object 200 to return to the light receiving element 300. In this method, for example, the time at which the intensity of the reflected light reaches its peak is converted into the distance.

[0013] The above-mentioned distance measurement technology uses LiDAR (Light Detection and Ranging). Specifically, this distance measurement technology measures distance by irradiating the target with near-infrared laser light as laser light 100 in a short period of time in pulses at high intensity multiple times to prevent a decrease in the signal-to-noise (SNR) ratio due to the influence of external light. However, high-power lasers that can output high intensity light are expensive and consume a lot of power.

[0014] Therefore, the above-described distance measurement technique measures distance by scanning the laser light 100 rather than irradiating the target object 200 with a high-intensity pulsed laser light 100 all at once. Furthermore, this distance measurement technique measures distance by collectively receiving reflected light using multiple pixels of the light receiving element 300 in order to suppress peaks in the intensity of the laser light 100 while realizing a field of view (FOV) and high resolution.

[0015] For example, in the above-described distance measurement technology, laser light 100 irradiated from a laser diode (LD) 400 including a light emitting diode (LED) is scanned horizontally and in a strip shape across an object 200 by a micro electro mechanical systems (MEMS) mirror 500. Then, a light receiving element 300 such as a SPAD that receives light reflected from the irradiated area of ​​the object 200 also activates an area that corresponds to the scan, thereby enabling measurement of the ToF of the reflected light.

[0016] "LD 400" is a light source of laser light 100. "MEMS mirror 500" is a mirror that reflects laser light 100. "Activation" means setting the pixels of the light receiving element 300 to pixels that can observe the intensity of reflected laser light. Ranging data can be generated based on the intensity of reflected light observed in the "activated" pixel area (activated area).

[0017] In the above-described distance measurement technology, if the target object 200 includes a highly reflective object such as a traffic sign, the highly reflective object reflects the laser light 100 with high intensity. This causes flare, a phenomenon in which reflected light leaks into pixels surrounding a pixel of the light receiving element 300 that is hit by the highly reflective object. Furthermore, because the flare spreads concentrically, saturation occurs, in which the intensity of the reflected light observed from the pixel of the light receiving element 300 exceeds a threshold.

[0018] Furthermore, when a flare occurs, scattered light reflected from a highly reflective object may be determined to be reflected light from a false object. In this case, even if the output data output from the pixel of the light receiving element 300 where the flare occurred is used as distance measurement data, accurate distance measurement cannot be performed, resulting in a decrease in distance measurement accuracy.

[0019] On the other hand, if the laser beam 100 can be irradiated so that the reflected light returns only to the valid area, which is the pixel area of ​​the light receiving element 300 where the intensity of the reflected light of the laser beam can be observed, flare can be eliminated, and a decrease in distance measurement accuracy can be prevented. However, in reality, it is difficult to irradiate the laser beam 100 so that the reflected light returns only to the valid area. This point will be explained using FIG. 2.

[0020] (1-2. Background of Information Processing System) Fig. 2 is a diagram for explaining the background of the information processing system according to the embodiment. As described above, in order to prevent a decrease in distance measurement accuracy, it is ideal to irradiate laser light so that the SPAD block 301 and the VCSEL (Vertical Cavity Surface Emitting Laser) irradiation area 302 are the same size, as shown in the "ideal state" in Fig. 2, for example.

[0021] The "SPAD block 301" is the enabled area of ​​the pixel of the light receiving element including the SPAD. The "VCSEL irradiation area 302" is the irradiation area of ​​the laser light from the VCSEL, which is a laser that irradiates the light receiving element with laser light in the vertical direction. Note that the "laser light irradiation area" includes the area where reflected light hits.

[0022] However, in reality, due to optical issues such as the precision of focusing the laser light, misalignment of the laser light irradiation position and the optical axis of the light receiving element, and cost issues, it is difficult to irradiate the laser light so that the reflected light returns only to the SPAD block 301. For this reason, the reflected light may strike the periphery of the SPAD block 301. For example, as shown in the "Actual" section of Figure 2, the VCSEL irradiation area 302 may be larger than the SPAD block 301.

[0023] In this way, when light reflected from a highly reflective object strikes a pixel outside the validity area but in the vicinity of the area, the reflected light leaks into pixels surrounding the pixel in question, causing flare. Furthermore, since it is not possible to determine from the output data output from the surrounding pixels whether the flare is caused by light reflected from a highly reflective object located outside the validity area, it is difficult to remove the flare. This point will be described in detail with reference to FIG. 3 .

[0024] (1-3. Purpose of Information Processing System) Fig. 3 is a diagram for explaining the purpose of the information processing system according to the embodiment. The following description will be given on the assumption that, among the positional relationships shown in the first pattern 304 to the third pattern 306 shown in the table 303 in Fig. 3, the positional relationship between the SPAD block 301 and the VCSEL irradiation area 302 is predetermined.

[0025] In the first pattern 304, reflected light 307 from a low-reflectivity object is present in the SPAD block 301. Furthermore, as a result of observing the intensity of the reflected light in the SPAD block 301, it is estimated that the reflected light observed from the SPAD block 301 is reflected light 307X from a low-reflectivity object, including reflected light 307 from the low-reflectivity object. Furthermore, as a result of signal processing for flare correction, which is signal processing for removing flare, a result similar to this estimation is obtained.

[0026] The signal processing for flare correction is, for example, a process of restoring an image based on a restoration characteristic opposite to the degradation characteristic of image quality caused by flare. This process is well known, and therefore a detailed description thereof will be omitted.

[0027] In this way, in the first pattern 304, it is not necessary to remove the flare, and as a result of the signal processing for flare correction, it is determined that the influence of the flare is small.

[0028] In the second pattern 305, reflected light 308 from a highly reflective object is present within the SPAD block 301. In this case, as a result of observing the intensity of the reflected light within the SPAD block 301, it is estimated that the reflected light observed from the SPAD block 301 is reflected light 308X from the highly reflective object because the flare is diffused when the laser light is irradiated onto the highly reflective object, which is the source of the flare.

[0029] On the other hand, as a result of signal processing for flare correction, it is determined that reflected light 308 from a highly reflective object is included in reflected light 307X from a low-reflectivity object observed by the SPAD block 301, based on information such as output data from pixels of the SPAD block 301. This makes it possible to predict that reflected light around reflected light 308 from a highly reflective object is scattered light due to flare, and therefore, as a result of signal processing for flare correction, it is possible to remove flare caused by reflected light from a highly reflective object. For this reason, it is determined that the influence of flare is small in the second pattern 305.

[0030] In the third pattern 306, reflected light 308 from a highly reflective object is outside the SPAD block 301 and exists within the VCSEL irradiation area 302. In this case, as a result of observing the intensity of reflected light within the SPAD block 301, it is estimated that the reflected light observed from the SPAD block 301 is reflected light 308X from the highly reflective object because flare is diffused when the laser light is irradiated onto the highly reflective object.

[0031] Furthermore, signal processing for flare correction does not operate normally because there is no reflected light 308 from a highly reflective object within the SPAD block 301. In other words, even if this signal processing is performed, it is not possible to determine whether the flare is caused by the reflected light 308 from a highly reflective object present in the SPAD block 301, and therefore it is not possible to remove the flare caused by the highly reflective object. As a result, it is determined that the third pattern 306 is significantly affected by the flare.

[0032] Therefore, an object of the information processing system according to the embodiment is to further prevent a decrease in distance measurement accuracy.

[0033] (1-4. Overview of Information Processing System) Next, an overview of the information processing system according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the overview of the information processing system according to the embodiment.

[0034] The information processing system sets a SPAD extension region 309, which is a region of pixels where the intensity of reflected light can be observed, around the SPAD block 301 so as to change from the "conventional" pattern to the "new pattern" in Fig. 4. In Fig. 4, the information processing system expands the region so that the region of pixels where the intensity of reflected light can be observed is larger than the VCSEL irradiation region 302. Furthermore, based on the intensity of reflected light observed from the pixels in the SPAD extension region 309, the information processing system determines whether the reflected light includes light reflected from a highly reflective object.

[0035] In this way, by setting an extended region such as the SPAD extended region 309 around an enabled region such as the SPAD block 301, the information processing system can detect reflected light from a highly reflective object outside the enabled region, thereby making it possible to predict flare that may occur from the reflected light. As a result, the information processing system can further prevent a decrease in distance measurement accuracy.

[0036] (2. First Embodiment) (2-1. Configuration of Information Processing System) Next, an example configuration of an information processing system 1 according to a first embodiment will be described with reference to FIG. 5. FIG. 5 is a block diagram showing an example configuration of an information processing system according to the first embodiment. In FIG. 5, the information processing system 1 includes an information processing device 10 and a SPAD array 300A.

[0037] The information processing device 10 is a sensor or the like that sets an extension area around the valid area and determines whether the reflected light includes light reflected from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extension area. The SPAD array 300A is an array of light receiving elements including SPADs.

[0038] 5, the information processing device 10 includes a SPAD control unit 11, a highly reflective object determination unit 12, a flare removal unit 13, and a post-processing unit 14. Each of these units included in the information processing device 10 functions as a control unit. The control unit is a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) that executes a program stored in a storage device such as a RAM (Random Access Memory). The control unit may also be an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0039] (SPAD Control Unit) The SPAD control unit 11 performs various controls of the SPAD array 300 A, including setting an extension area around the enabled area included in the SPAD array 300 A. For example, the SPAD control unit 11 sets an extension area of ​​SPAD pixels, such as a SPAD extension area 309, around the SPAD block 301 included in the SPAD array 300 A, based on SPAD information related to the SPAD block 301 that has been set in advance.

[0040] The SPAD control unit 11 also sets the valid area so that the intensity of reflected light can be observed and output data output from the pixels in the valid area is used as distance measurement data. When setting the extended area, the SPAD control unit 11 also sets the extended area so that the intensity of reflected light can be observed and output data output from the pixels in the extended area is not used as distance measurement data.

[0041] For example, the SPAD control unit 11 sets the detection of reflected light from a highly reflective object that is a flare source to use all pixels of the SPAD block 301 and the SPAD extended region 309. However ... to use only output data from the SPAD block 301 for distance measurement data, and not to use output data from the SPAD extended region 309, which has a low intensity of reflected light and a low sensitivity (degree of sensitivity to reflected light).

[0042] (Highly Reflective Object Determination Unit) The highly reflective object determination unit 12 is a signal processing unit or the like that determines whether or not the reflected light observed from the pixels in the set extended region includes reflected light from a highly reflective object, based on the intensity of the reflected light. When determining whether or not the reflected light observed from the pixels in the extended region includes reflected light from a highly reflective object, the highly reflective object determination unit 12 refers to the intensity of the reflected light from the VCSEL, etc., as the intensity of the reflected light observed from the pixels in the extended region.

[0043] For example, the high-reflectivity object determination unit 12 refers to the output for determining surrounding high-reflectivity objects from the SPAD array 300A and the ranging block output, and determines whether the reflected light of the VCSEL observed from the pixels of the SPAD extension area 309 includes reflected light from a highly reflective object.

[0044] The "output for determining surrounding highly reflective objects" is output data for determining reflected light from highly reflective objects present around the validity area. The output for determining surrounding highly reflective objects includes, for example, output data relating to the intensity of reflected light observed from pixels in the SPAD extension area 309. The "distance measurement block output" is output data corresponding to the original data for distance measurement data.

[0045] Specifically, when the intensity of reflected light observed from pixels in the SPAD extension region 309 included in the output for determining surrounding highly reflective objects is equal to or greater than a threshold value for a predetermined period or longer, the highly reflective object determination unit 12 determines that the reflected light includes light reflected from a highly reflective object. Furthermore, the highly reflective object determination unit 12 determines that reflected light from a highly reflective object exists in an area that is equal to or greater than the threshold value for a predetermined period or longer.

[0046] In addition, the highly reflective object determination unit 12 can also determine whether the reflected light observed from the extended area includes reflected light from a highly reflective object, based on image data detected by a sensor such as LiDAR.

[0047] For example, the highly reflective object determination unit 12 refers to the image data when determining whether the reason why the intensity of reflected light observed from pixels in the extended region is above a threshold for a predetermined period or more is due to reflected light from a highly reflective object. If the image data includes a highly reflective object, the highly reflective object determination unit 12 determines that the reflected light observed from the extended region includes reflected light from a highly reflective object.

[0048] Furthermore, when the highly reflective object determination unit 12 cannot determine whether the flare is caused by reflected light from a highly reflective object hitting the validity area, it determines whether reflected light from a highly reflective object is included in the reflected light observed from the pixels in the extended area. That is, the highly reflective object determination unit 12 determines whether reflected light from a highly reflective object hits the validity area based on the intensity of reflected light observed in the validity area, and when the highly reflective object determination unit 12 cannot make this determination, it can determine whether reflected light from a highly reflective object is included in the reflected light observed from the pixels in the extended area.

[0049] For example, in the third pattern 306 in Fig. 3, it is only determined that the reflected light observed from the pixels of the SPAD block 301 is reflected light 308X from a highly reflective object. In this case, it is not determined whether the flare is caused by reflected light from a highly reflective object hitting the SPAD block 301. Therefore, the highly reflective object determination unit 12 determines whether the reflected light 308 from the highly reflective object in Fig. 3 is included in the reflected light observed from the pixels of the SPAD extension region 309 in Fig. 4.

[0050] Furthermore, when the highly reflective object determination unit 12 determines that the reflected light observed from the pixels of the set extended region includes the reflected light from a highly reflective object, it determines that the flare is caused by the reflected light from the highly reflective object hitting the extended region. For example, when the highly reflective object determination unit 12 determines that the reflected light 308 from the highly reflective object of Fig. 3 is included in the reflected light observed from the pixels of the SPAD extended region 309 of Fig. 4, it determines that the flare is caused by the reflected light 308 from the highly reflective object hitting the SPAD extended region 309.

[0051] (Flare Removal Unit) Returning to the explanation of Fig. 5, the flare removal unit 13 is a signal processing unit or the like that removes flare when it is determined that flare, which is caused by light reflected from a highly reflective object leaking into pixels in the validated region, is caused by light reflected from the highly reflective object hitting the extended region.

[0052] For example, if the flare removal unit 13 determines that the reflected light observed from the pixels in the SPAD extension region 309 includes reflected light from a highly reflective object, it determines that the flare is caused by reflected light from a highly reflective object hitting the SPAD extension region 309.

[0053] In this case, the flare removal unit 13 performs the above-described flare correction signal processing to remove flare caused by light reflected from a highly reflective object leaking into pixels around the SPAD extended region 309. Specifically, the flare removal unit 13 performs flare correction signal processing to perform restoration based on restoration characteristics opposite to the deterioration characteristics to remove flare from the ranging block output from the SPAD array 300A.

[0054] The flare removal unit 13 outputs the distance measurement data, etc., from which the flare has been removed, to the post-processing unit 14 .

[0055] (Post-Processing Unit) The post-processing unit 14 is a signal processing unit that processes ranging data, etc. For example, the post-processing unit 14 performs ranging by using the ranging data for ToF ranging or LiDAR.

[0056] (2-2. Information Processing Procedure) Next, an example of the information processing procedure by the information processing device 10 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the information processing procedure by the information processing device according to the first embodiment. In this flowchart, before step S1, the SPAD control unit 11 sets a SPAD extension area 309 around the SPAD block 301.

[0057] In step S1, the SPAD control unit 11 selects a SPAD block. For example, the SPAD control unit 11 selects a valid area such as the SPAD block 301 or an extension area such as the SPAD extension area 309.

[0058] In step S2, the highly reflective object determining unit 12 reads out SPAD information, which is information about pixels in the valid area or the extended area of ​​the light receiving element for which SPAD is used.

[0059] For example, the highly reflective object determination unit 12 reads out, as the SPAD information, information on an effective region such as the selected SPAD block 301, or information on an extended region such as the selected SPAD extended region 309. Specifically, the highly reflective object determination unit 12 reads out, as the SPAD information, the intensity of reflected light observed from pixels in the selected SPAD block 301, or the intensity of reflected light observed from pixels in the selected SPAD extended region 309.

[0060] In step S3, the highly reflective object determining unit 12 determines whether the read SPAD information is information relating to an extension area such as the selected SPAD extension area 309 or not.

[0061] If the read SPAD information is information about the selected SPAD extended region 309 (step S3; Yes), the highly reflective object determination unit 12 performs highly reflective object estimation for the extended region. For example, as the highly reflective object estimation for the extended region, the highly reflective object determination unit 12 determines whether the reflected light observed from the pixels of the extended region such as the SPAD extended region 309 includes reflected light from a highly reflective object, based on the intensity of the reflected light.

[0062] In this case, the highly reflective object determination unit 12 estimates highly reflective objects for the extended area based on the intensity of reflected light observed from pixels in an enabled area such as the SPAD block 301, in the same way as when determining whether the reflected light includes reflected light from a highly reflective object.

[0063] However, when estimating a highly reflective object for the extended area, the highly reflective object determination unit 12 makes an estimation that takes into account the property that in the extended area, the intensity of reflected light observed from pixels in the extended area, such as the SPAD extended area 309, is lower than the intensity of reflected light observed from pixels in the enabled area.

[0064] Furthermore, the highly reflective object determination unit 12 performs estimation of highly reflective objects for the extended area, taking into consideration the positional relationship between the enabled area, such as the SPAD block 301, and the extended area. For example, the highly reflective object determination unit 12 performs estimation taking into consideration the property that the farther the peripheral extended area, such as the SPAD extended area 309, is from a predetermined side of the SPAD block 301, the lower the intensity of reflected light observed from the extended area.

[0065] In step S3, if the read SPAD information is information about a valid area such as the selected SPAD block 301 (step S3; No), in step S5, the highly reflective object determination unit 12 performs highly reflective object estimation for the valid area. In this case, the highly reflective object determination unit 12 determines whether reflected light observed from pixels in the valid area includes reflected light from a highly reflective object.

[0066] For example, if the intensity of reflected light observed from a pixel of the SPAD block 301 is equal to or greater than a threshold for a predetermined period of time, the highly reflective object determination unit 12 determines that the reflected light includes light reflected from a highly reflective object.

[0067] In step S6, the highly reflective object determination unit 12 integrates the results of the two types of highly reflective object estimations, the highly reflective object estimation for the extended region in step S4 and the highly reflective object estimation in step S5.

[0068] For example, if reflected light from a highly reflective object is detected in the highly reflective object estimation in step S5, the highly reflective object determination unit 12 uses the estimation result of the highly reflective object estimation in step S5 as the highly reflective object estimation result. If reflected light from a highly reflective object is not detected in the highly reflective object estimation in step S5, but reflected light from a highly reflective object is detected in the highly reflective object estimation for the extended area in step S4, the highly reflective object determination unit 12 uses the estimation result of the highly reflective object estimation for the extended area as the highly reflective object estimation result.

[0069] When reflected light from a highly reflective object is not detected in the highly reflective object estimation in step S5, this refers to a case where, even if the intensity of reflected light observed from pixels in the validation area is above a threshold for a predetermined period of time or more, it is not possible to determine whether the reflected light is due to reflected light from a highly reflective object present within the validation area.

[0070] In step S7, the high-reflecting object determining unit 12 determines whether a flare has occurred in the output block.

[0071] First, the highly reflective object determination unit 12 determines whether a flare has occurred in the output block, which is the validated area, based on whether the reflected light (echo) observed from the pixels in the validated area includes an echo from a highly reflective object. When the estimation result of the highly reflective object estimation in step S5 is used as the highly reflective object estimation result, the highly reflective object determination unit 12 determines that a flare has occurred in the output block due to an echo from a highly reflective object present in the validated area.

[0072] When it is not possible to determine whether the flare is caused by an echo from a highly reflective object hitting the validity area, the highly reflective object determination unit 12 determines whether the echo observed from the pixels in the extension area includes an echo from a highly reflective object.

[0073] When the high-reflecting object determination unit 12 determines that the echoes observed from the pixels in the set extended region include echoes from a highly reflective object, it determines that the flare is caused by the echo from the highly reflective object hitting the extended region. In other words, when the estimation result of the high-reflecting object estimation for the extended region is used as the high-reflecting object estimation result, the high-reflecting object determination unit 12 determines that the flare is caused by the echo from the highly reflective object hitting the extended region.

[0074] If the high-reflecting object determination unit 12 determines that a flare has occurred in the output block (step S7; Yes), the flare removal unit 13 removes the flare caused by the echo from the high-reflecting object in step S8. For example, if the flare removal unit 13 determines that a flare caused by the echo from a high-reflecting object present in the SPAD extension region 309 has occurred in the SPAD block 301, the flare removal unit 13 removes the flare by signal processing that restores the signal based on restoration characteristics that are opposite to the degradation characteristics.

[0075] In step S9, if the above-mentioned processes for all blocks of the SPAD array 300A are completed (step S9; Yes), the information processing device 10 completes the process. If the above-mentioned processes for all blocks are not completed (step S9; No), each unit of the information processing device 10 repeats the above-mentioned processes from step S1.

[0076] (3. Second Embodiment) The information processing device 10 can also set an area including a part of the laser light irradiation area as the extended area. An example configuration of an information processing device 10A included in an information processing system 1A according to the second embodiment will be described below with reference to FIG. 7. FIG. 7 is a block diagram showing an example configuration of the information processing system according to the second embodiment. The information processing device 10A includes a SPAD control unit 11A instead of the SPAD control unit 11.

[0077] (SPAD control unit) The SPAD control unit 11A sets an area including a part of the area irradiated with laser light as the extended area. Examples of setting the extended area by the SPAD control unit 11A will be described below with reference to Figs. 8 to 10. Fig. 8 is a diagram showing a first setting example of the extended area. Fig. 9 is a diagram showing a second setting example of the extended area. Fig. 10 is a diagram showing a third setting example of the extended area.

[0078] As shown in FIG. 8, when the SPAD block 301 is a polygonal area, the SPAD control unit 11A sets a plurality of SPAD extension areas 309A including extension areas adjacent to part of each side of the SPAD block 301 as extension areas.

[0079] For example, the SPAD control unit 11A sets one pixel of each extension area on each side of a polygon as the plurality of extension areas. Specifically, if the SPAD block 301 is a square area of ​​5 × 5 pixels, the SPAD control unit 11A sets one pixel of each SPAD extension area 309A on each side of the square.

[0080] Furthermore, when setting the SPAD extension region 309A, the SPAD control unit 11A sets the SPAD extension region 309A so that it is larger than the extent of the VCSEL irradiation region 302, that is, so that the SPAD extension region 309A extends outside the VCSEL irradiation region 302.

[0081] 9, the laser beam irradiation means is a block scan that scans the laser beam in the vertical and horizontal directions. In this case, when setting the multiple extended areas, the SPAD control unit 11A sets the multiple SPAD extended areas 309B that are hit by the reflected light in the vertical scan 601 of the laser beam and the horizontal scan 602 so that they do not overlap.

[0082] Furthermore, when setting the multiple SPAD extended regions 309B, the SPAD control unit 11A sets the regions so that reflected light hits two or more extended regions among the multiple SPAD extended regions 309B during each of one vertical scan and one horizontal scan of the laser light. Specifically, if the size of the SPAD block 301 is 5 × 5 pixels, the SPAD control unit 11A sets the regions so that reflected light hits two SPAD extended regions 309B, each consisting of one pixel, during each of one vertical scan and one horizontal scan.

[0083] 10, the SPAD control unit 11A sets a plurality of SPAD extension regions 309C as extension regions at positions where the horizontal length of the SPAD block 301C, which is the enabled region, is divided at approximately equal intervals and vertically adjacent to the SPAD block 301C. The intervals are not particularly limited, but may be, for example, 1 / 16 of the horizontal length of the SPAD block 301C. Specifically, if the horizontal length of the SPAD block 301C is 256 pixels, the intervals are 16 pixels.

[0084] In addition, when the laser light irradiation means is a line scan in which the laser light is shifted vertically each time it scans one line horizontally, the SPAD control unit 11A sets a SPAD extension area 309C for line scan as a plurality of extension areas.

[0085] Furthermore, when setting a plurality of SPAD expansion areas 309C, the SPAD control unit 11A sets the SPAD expansion areas 309C so that the horizontal positions of the SPAD expansion areas 309C on vertically adjacent lines are different among the plurality of SPAD expansion areas 309C. In this case, the SPAD control unit 11A sets the SPAD expansion areas 309C so that the vertical positions of horizontally adjacent SPAD expansion areas 309C are different among the plurality of SPAD expansion areas 309C.

[0086] (4. Third Embodiment) The information processing device 10 can further determine whether the reflected light observed from the pixels in the valid area and the reflected light observed from the pixels in the extended area include reflected light from a white object. Hereinafter, an example configuration of an information processing device 10B included in an information processing system 1B according to the third embodiment will be described with reference to FIG. 11 . FIG. 11 is a block diagram showing an example configuration of an information processing system according to the third embodiment. The information processing device 10B includes a highly reflective object determination unit 12B instead of the highly reflective object determination unit 12.

[0087] (Highly Reflective Object Determination Unit) The highly reflective object determination unit 12B further determines whether the reflected light observed from the pixels in the validity area includes reflected light from a white object, based on the intensity of reflected light observed from the pixels in the validity area and the intensity of reflected light observed from the pixels in the extended area. This point will be described in detail with reference to Fig. 12. Fig. 12 is a diagram for explaining the determination of whether the reflected light observed from the pixels in the validity area and the reflected light observed from the pixels in the extended area includes reflected light from a white object.

[0088] 12 indicates a region where the intensity of reflected light from the SPAD block 301 is higher than a predetermined intensity. Because the SPAD block 301 is highly sensitive, the SPAD block 301 may be completely saturated not only by light reflected from a highly reflective object but also by light reflected from a white object. In such cases, it is often impossible to distinguish whether the saturation is due to light reflected from a highly reflective object or light reflected from a white object.

[0089] On the other hand, since the intensity of the reflected light hitting the SPAD extension region 309D is weaker than the intensity of the reflected light hitting the SPAD block 301, the SPAD extension region 309D may not be saturated even if it is hit by light reflected by a white object. This makes it easier to distinguish between the reflected light from a highly reflective object and the reflected light from a white object.

[0090] In this way, the highly reflective object determination unit 12B utilizes the above-described properties of the extended region when determining whether the reflected light observed from the pixels in the valid region and the reflected light observed from the pixels in the extended region includes reflected light from a white object. This point will be explained using Fig. 13. Fig. 13 is a diagram showing the results of a comparison between a case where reflected light from only highly reflective objects hits the pixel region and a case where reflected light from both highly reflective objects and white objects hits the pixel region.

[0091] "Only highly reflective objects" in Figure 13 means that the reflected light observed from the validity area includes only reflected light from highly reflective objects. "Another object exists at the same distance as the highly reflective object" means that a white object exists at the same distance (depth) as the highly reflective object when the SPAD array 300A is used as the reference. It also means that the reflected light observed from the validity area includes not only reflected light from highly reflective objects but also reflected light from white objects.

[0092] 13 conceptually indicates the position from, for example, the top edge and near the center in the horizontal direction of the SPAD block 301 in FIG. 12 to the left edge of the SPAD extension area 309D adjacent to the SPAD block 301 at a block boundary 800. The "block boundary 800" is the boundary between the SPAD block 301 and the SPAD extension area 309D.

[0093] As shown in Figure 13, the laser irradiation intensity, which is the intensity of the laser light irradiated onto the target object, is the same for "highly reflective object only" and "another object exists at the same distance as the highly reflective object." On the other hand, the intensity of the reflected light differs between "highly reflective object only" and "another object exists at the same distance as the highly reflective object."

[0094] Therefore, when determining whether reflected light from a white object is included, the highly reflective object determination unit 12B refers to the continuity of change from the intensity of reflected light observed from pixels in the enabled area to the intensity of reflected light observed from pixels in the extended area.

[0095] For example, in the case of "highly reflective objects only," the intensity of reflected light continuously (smoothly) attenuates from the upper limit intensity at the highly reflective object position 900, where reflected light from the highly reflective object is present, through the block boundary 800 to the intensity at the left end of the SPAD extension area 309D. "Continuously" means that the degree of change in intensity with respect to position is less than a predetermined degree, such as when the differential value of the intensity with respect to position is less than a predetermined value.

[0096] On the other hand, in the case of "another object exists at the same distance as the highly reflective object," the reflected light intensity of the SPAD extension region 309D is weaker than that of the SPAD block 301, and therefore is reduced by the amount of the reflected light intensity from the white object that was added to the overall reflected light intensity. As a result, in the case of "another object exists at the same distance as the highly reflective object," the change in reflected light intensity becomes discontinuous at the block boundary 800.

[0097] "Discontinuous" refers to a situation in which the degree of intensity relative to a position undergoes a sudden change and then returns to its original degree, such as when the differential value of the intensity relative to the position changes by more than a predetermined value and then returns to less than the predetermined value.

[0098] In this way, if the change from the intensity of reflected light observed from pixels in the enabled region to the intensity of reflected light observed from pixels in the extended region is discontinuous, the highly reflective object determination unit 12B determines that the reflected light observed from pixels in the enabled region includes reflected light from a white object.

[0099] (5. Other Embodiments) The processes according to the embodiments can be implemented in various different forms other than the above-described embodiments.

[0100] Of the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. The various information shown in each drawing is not limited to the information shown in the drawings.

[0101] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of the configuration can be functionally or physically distributed or integrated in any unit depending on various loads and usage conditions.

[0102] For example, at least some of the SPAD control unit 11, the high-reflecting object determination unit 12, the flare removal unit 13, and the post-processing unit 14 included in the information processing device 10 can be distributed. Specifically, the post-processing unit 14 of the information processing device 10 can be distributed to another device. Furthermore, the information processing device 10 can be a device other than a sensor. In this case, the high-reflecting object determination unit 12 and the flare removal unit 13 perform processing outside the sensor. Furthermore, the SPAD array 300A is configured to output a peripheral high-reflecting object determination output and a ranging block output to the outside of the sensor.

[0103] The above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0104] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0105] (6. Effects of Information Processing Device According to the Present Disclosure) As described above, the information processing device according to the present disclosure (information processing device 10 in the embodiment) includes a setting unit (SPAD control unit 11 in the embodiment) and a determination unit (highly reflective object determination unit 12 in the embodiment). The setting unit sets an extended region, which is a region of pixels in which the intensity of reflected light of laser light can be observed, around an enabled region, which is a region of pixels in the light receiving element in which the intensity of reflected light of laser light can be observed. The determination unit determines whether the reflected light includes reflected light from a highly reflective object, based on the intensity of reflected light observed from the pixels in the set extended region.

[0106] In this way, by setting an extended area around the valid area, the information processing device can detect reflected light from highly reflective objects outside the valid area, making it possible to predict flare that may occur from the reflected light. As a result, the information processing device can further prevent a decrease in distance measurement accuracy.

[0107] The setting unit sets the valid area so that the intensity of reflected light can be observed and output data output from the pixels in the valid area is used as distance measurement data, and when setting the extended area, sets it so that the intensity of reflected light can be observed and output data output from the pixels in the extended area is not used as distance measurement data.

[0108] This allows the information processing device to use only output data with high reflected light intensity and high sensitivity output from the pixels in the validation area as distance measurement data, further preventing a decrease in distance measurement accuracy. Furthermore, the information processing device can reduce the amount of pixel-related information read compared to when the validation area is previously made larger than the irradiation area, thereby reducing memory costs. For example, the information processing device can reduce the amount of information read by 400 pixels per laser beam irradiation in the alternative method.

[0109] When it is not possible to determine whether reflected light from a highly reflective object has hit the validation area based on the intensity of reflected light observed in the validation area, the determination unit determines whether reflected light from a highly reflective object is included in reflected light observed from pixels in the extension area.Therefore, the information processing device determines whether reflected light from a highly reflective object is included in reflected light observed from pixels in the extension area only when it is not possible to determine whether flare is caused by reflected light from a highly reflective object hitting the validation area, thereby reducing the amount of processing.

[0110] When the determination unit determines that the reflected light observed from the pixels in the set extended region includes reflected light from a highly reflective object, it determines that the flare caused by the reflected light from the highly reflective object leaking into the pixels in the valid region is caused by the reflected light from the highly reflective object hitting the extended region. This allows the information processing device to determine the cause of the flare with high accuracy, making it easier to appropriately remove the flare.

[0111] The information processing device further includes an elimination unit (in the embodiment, a flare elimination unit 13) that eliminates flare when it is determined that the flare is caused by light reflected from a highly reflective object hitting the extended area. This allows the information processing device to appropriately eliminate flare caused by light reflected from a highly reflective object hitting the extended area by signal processing for flare correction, thereby further preventing a decrease in distance measurement accuracy.

[0112] The setting unit sets, as the extended region, a region including a part of the irradiation region of the laser light. As a result, the information processing device sets, as the extended region, a region including a part of the irradiation region in which reflected light from a highly reflective object may exist, and therefore, it is possible to easily detect reflected light from a highly reflective object while suppressing an increase in the number of pixels in the extended region to be processed due to the setting of the extended region.

[0113] When the validation area is a polygonal area, the setting unit sets a plurality of extension areas, including extension areas adjacent to a part of each side of the polygon, as the extension area. As a result, the information processing device sets an extension area smaller than the validation area and adjacent to the validation area into which reflected light from highly reflective objects is likely to leak, thereby making it easier to detect reflected light from highly reflective objects while further suppressing an increase in the number of pixels in the extension area to be processed due to the setting of the extension area.

[0114] The setting unit sets the multiple extended regions, one pixel per side of the polygon, so that the information processing device sets smaller extended regions relative to the validity region, thereby further suppressing an increase in the number of pixels in the extended regions to be processed due to the setting of the extended regions.

[0115] When the laser light irradiation means is a block scan that scans the laser light in the vertical and horizontal directions, the setting unit, when setting the multiple extension regions, sets them so that the multiple extension regions hit by the reflected light do not overlap during the vertical scanning and the horizontal scanning of the laser light. This allows the information processing device to distribute the multiple extension regions while reducing the size of the extension regions, and therefore allows the pixels of the multiple extension regions to function as search pixels that can efficiently search for reflected light from a reflecting object while reducing memory costs.

[0116] For example, if the information processing device holds output results (output data) from pixels in other enabled areas such as other SPAD blocks, it can refer to the output data from the pixels into which the reflected light has leaked, even without a search pixel. However, in this case, the amount of output data read from the pixels in the enabled areas into which the reflected light has leaked increases, requiring a large amount of memory.

[0117] In particular, when the laser light irradiation means is block scanning, the amount of memory required becomes enormous, such as 520 pixels per line, so the information processing device can achieve cost advantages by setting multiple extension areas that include pixels that function as search pixels.

[0118] When setting the multiple extended regions, the setting unit sets the multiple extended regions so that reflected light hits two or more of the multiple extended regions in each of one vertical scan and one horizontal scan of the laser light. This allows the information processing device to observe the intensity of reflected light from pixels in two or more extended regions in one scan, making it easier to detect reflected light from highly reflective objects.

[0119] The setting unit sets a plurality of extension regions at positions where the horizontal length of the activation area is divided at approximately equal intervals and where the extension regions are adjacent to the activation area in the vertical direction. This allows the information processing device to distribute the plurality of extension regions while suppressing the size of the extension regions, and allows pixels of the plurality of extension regions to function as search pixels.

[0120] When the laser beam irradiation means is a line scan means that shifts the laser beam vertically each time it scans one line in the horizontal direction, the setting unit sets the extension regions for the line scan as the plurality of extension regions. In this way, the information processing device can apply the plurality of extension regions described above as the extension regions for the line scan.

[0121] When setting the multiple extension regions, the setting unit sets the multiple extension regions so that the horizontal positions of the extension regions on vertically adjacent lines are different. In this way, the information processing device can disperse the multiple extension regions more by shifting the horizontal positions of the extension regions that function as regions including search pixels on vertically adjacent lines above and below, thereby enabling efficient search of reflected light from a reflecting object from a wider range of search pixels.

[0122] The determination unit further determines whether the reflected light observed from the pixels in the valid area and the reflected light observed from the extended area includes reflected light from a white object, based on the intensity of the reflected light observed from the pixels in the valid area and the intensity of the reflected light observed from the pixels in the extended area.

[0123] This allows the information processing device to detect reflected light from a white object, and therefore distinguish between reflected light from a highly reflective object and reflected light from a white object even when the reflected light overlaps with the highly reflective object. As a result, the information processing device can improve the accuracy of determining whether the reflected light includes reflected light from a highly reflective object. Furthermore, the information processing device can prevent erroneous removal of not only flare caused by reflected light from a highly reflective object but also reflected light from a white object.

[0124] When determining whether reflected light from a white object is included, the determination unit refers to the continuity of change from the intensity of reflected light observed from pixels in the valid area to the intensity of reflected light observed from pixels in the extended area.

[0125] As a result, when making this determination, the information processing device can refer to the property that the continuity of change in the intensity of reflected light differs between when only reflected light from a highly reflective object is present in the validation area and the extension area and when reflected light from a white object is present in these areas in addition to the highly reflective object. This makes it easier for the information processing device to detect reflected light from a white object even when the reflected light from a highly reflective object and the reflected light from a white object overlap.

[0126] When determining whether reflected light from a white object is included, the determination unit determines that reflected light from a white object is included if there is a discontinuous change from the intensity of reflected light observed from pixels in the enabled area to the intensity of reflected light observed from pixels in the extended area.

[0127] As a result, when making this determination, the information processing device can refer to the property that, if reflected light from a white object is present, the reflected light intensity in the extended region is weaker than that in the enabled region, and therefore the intensity of the reflected light from the white object, which had previously been added to the intensity of the overall reflected light, is reduced by the amount of the intensity of the reflected light from the white object. This makes it easier for the information processing device to detect reflected light from a white object even when the reflected light from a highly reflective object and the reflected light from the white object overlap.

[0128] The setting unit sets the region of pixels of the SPAD as the extended region. Because the SPAD has high sensitivity, the information processing device sets the region of pixels of the SPAD as the extended region, thereby making it easier to detect reflected light from a highly reflective object from reflected light observed from the extended region.

[0129] When determining whether the reflected light observed from the pixels of the extended region includes light reflected from a highly reflective object, the determination unit refers to the intensity of the reflected light from the VCSEL as the intensity of the reflected light observed from the pixels of the extended region. Because the VCSEL irradiates laser light perpendicularly to the light receiving element, optical problems are less likely to occur and reflected light is more likely to return to the extended region, making it easier for the information processing device to detect reflected light from a highly reflective object from the reflected light of the VCSEL.

[0130] (7. Hardware Configuration) Information devices such as the information processing device 10 according to each of the above-described embodiments are realized by a computer 1000 configured as shown in FIG. 14. FIG. 14 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device according to the embodiments. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0131] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0132] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, and programs that depend on the hardware of the computer 1000 .

[0133] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0134] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (such as the Internet). The CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0135] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. The CPU 1100 receives data from input devices such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 transmits data to output devices such as a display, a speaker, and a printer via the input / output interface 1600. The input / output interface 1600 can also function as a media interface for reading a program recorded on a predetermined recording medium.

[0136] The media may be optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Discs), magneto-optical recording media such as MOs (Magneto-Optical disks), tape media, magnetic recording media, or semiconductor memories.

[0137] When the computer 1000 functions as the information processing device 10 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize the functions of the control units such as the SPAD control unit 11 to the post-processing unit 14 in Fig. 5. The information processing program according to the present disclosure and data in the storage device are stored in the HDD 1400.

[0138] The CPU 1100 reads and executes the program data 1450 from the HDD 1400. However, as another example, the CPU 1100 can also obtain these programs from other devices via an external network 1550.

[0139] (8. Supplementary Information) The present technology can also be configured as follows. (1) An information processing device comprising: a setting unit that sets an extended region, which is a region of pixels where the intensity of reflected light of laser light can be observed, around an enabled region that is a region of pixels where the intensity of the reflected light of the light receiving element can be observed, and a determination unit that determines whether the reflected light includes light reflected from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended region. (2) The information processing device described in (1), wherein the setting unit sets the enabled region so that the intensity of the reflected light can be observed and output data output from the pixels in the enabled region is used as distance measurement data, and when setting the extended region, sets the extended region so that the intensity of the reflected light can be observed and output data output from the pixels in the extended region is not used as the distance measurement data. (3) The information processing device according to (1) or (2), wherein, when it is not possible to determine whether reflected light from the highly reflective object has hit the validation region based on the intensity of the reflected light observed in the validation region, the determination unit determines whether reflected light observed from pixels in the extended region includes reflected light from the highly reflective object. (4) The information processing device according to (3), wherein, when it is determined that reflected light observed from pixels in the set extended region includes reflected light from the highly reflective object, the determination unit determines that flare caused by reflected light from the highly reflective object leaking into pixels in the validation region is caused by reflected light from the highly reflective object hitting the extended region. (5) The information processing device according to (4), further comprising a removal unit that removes the flare when it is determined that the flare is caused by reflected light from the highly reflective object hitting the extended region. (6) The information processing device according to any one of (1) to (5), wherein the setting unit sets, as the extended region, an area including a part of the irradiation region of the laser light. (7) The information processing device according to (6), wherein, when the validation area is a polygonal area, the setting unit sets, as the extension area, a plurality of extension areas including extension areas adjacent to parts of each side of the polygon.(8) The information processing device according to (7), wherein the setting unit sets the extended regions by one pixel on each side of the polygon as the extended regions. (9) The information processing device according to (7) or (8), wherein, when the laser light irradiation means is a block scan that scans the laser light in vertical and horizontal directions, the setting unit sets the extended regions so that the extended regions hit by the reflected light do not overlap in the vertical scanning and the horizontal scanning of the laser light when setting the extended regions. (10) The information processing device according to (9), wherein, when setting the extended regions, the setting unit sets the extended regions so that the reflected light hits two or more of the extended regions in each of the vertical scanning and the horizontal scanning of the laser light. (11) The information processing device according to any one of (1) to (10), wherein the setting unit sets, as the extended region, a plurality of extended regions at positions where the horizontal length of the enabled region is divided at approximately equal intervals and at positions vertically adjacent to the enabled region. (12) The information processing device according to (11), wherein, when the laser light irradiation means is a line scan that shifts the laser light vertically each time it scans one line in the horizontal direction, the setting unit sets extended regions for line scanning as the plurality of extended regions. (13) The information processing device according to (12), wherein, when setting the plurality of extended regions, the setting unit sets the positions of the horizontal extended regions in lines adjacent in the vertical direction among the plurality of extended regions to be different. (14) The information processing device according to any one of (1) to (13), wherein the determination unit further determines whether the reflected light observed from the pixels of the enabled region and the reflected light observed from the pixels of the extended region includes reflected light from a white object, based on the intensity of the reflected light observed from the pixels of the enabled region and the intensity of the reflected light observed from the pixels of the extended region.(15) The information processing device according to (14), wherein the determination unit, when determining whether reflected light from the white object is included, refers to continuity of change from the intensity of reflected light observed from pixels in the enabled region to the intensity of reflected light observed from pixels in the extended region. (16) The information processing device according to (15), wherein, when determining whether reflected light from the white object is included, the determination unit determines that reflected light from the white object is included if there is discontinuity in the change from the intensity of reflected light observed from pixels in the enabled region to the intensity of reflected light observed from pixels in the extended region. (17) The information processing device according to any one of (1) to (16), wherein the setting unit sets an area of ​​pixels of a SPAD as the extended region. (18) The information processing device according to any one of (1) to (17), wherein the determination unit, when determining whether reflected light observed from pixels in the extended region includes reflected light from the highly reflective object, refers to intensity of reflected light from a VCSEL as the intensity of reflected light observed from pixels in the extended region. (19) An information processing method comprising: a computer setting an extended region, which is a region of pixels where the intensity of reflected light of laser light can be observed, around an enabled region, which is a region of pixels in a light receiving element where the intensity of the reflected light can be observed, and determining whether or not the reflected light includes light reflected from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended region. (20) An information processing program for causing a computer to function as an information processing device comprising: a setting unit that sets an extended region, which is a region of pixels where the intensity of reflected light of laser light can be observed, around an enabled region, which is a region of pixels in a light receiving element where the intensity of the reflected light can be observed, and a determining unit that determines whether or not the reflected light includes light reflected from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended region.

[0140] REFERENCE SIGNS LIST 1 Information processing system 10 Information processing device 11 SPAD control unit 12 Highly reflective object determination unit 13 Flare removal unit 14 Post-processing unit 300A SPAD array

Claims

a setting unit that sets an extended region, which is a region of pixels in which the intensity of reflected laser light can be observed, around an enabled region, which is a region of pixels in the light receiving element in which the intensity of reflected laser light can be observed; a determination unit that determines whether the reflected light includes reflected light from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended region; An information processing device comprising: 　 The setting unit setting the valid area so that the intensity of the reflected light can be observed and output data output from pixels in the valid area is used as distance measurement data; When setting the extension area, the intensity of the reflected light can be observed and output data output from pixels in the extension area is set so as not to be used as the distance measurement data. The information processing device according to claim 1 . 　 When it is not possible to determine whether the reflected light from the highly reflective object has hit the validated area based on the intensity of the reflected light observed in the validated area, the determination unit determines whether the reflected light from the highly reflective object is included in the reflected light observed from the pixels in the extended area. The information processing device according to claim 1 . 　 When the determination unit determines that the reflected light observed from the pixels of the set extended region includes the reflected light from the highly reflective object, the determination unit determines that flare caused by the reflected light from the highly reflective object leaking into the pixels of the valid region is caused by the reflected light from the highly reflective object hitting the extended region. The information processing device according to claim 3 . 　 a removal unit that removes the flare when it is determined that the flare is caused by light reflected from the highly reflective object hitting the extended region, The information processing device according to claim 4 . 　 the setting unit sets, as the extended region, a region including a part of the irradiation region of the laser light. The information processing device according to claim 1 . 　 When the validity area is a polygonal area, the setting unit sets a plurality of extended areas as the extended area, including extended areas adjacent to parts of each side of the polygon. The information processing device according to claim 6 . 　 the setting unit sets one pixel of the extended regions on each side of the polygon as the plurality of extended regions; The information processing device according to claim 7 . 　 When the laser light irradiation means is a block scan that scans the laser light in a vertical direction and a horizontal direction, the setting unit sets the plurality of extended areas so that the plurality of extended areas that are hit by the reflected light do not overlap during the vertical direction scanning and the horizontal direction scanning of the laser light. The information processing device according to claim 7 . 　 When setting the plurality of extended regions, the setting unit sets the extended regions so that the reflected light hits two or more of the plurality of extended regions in each of one scanning of the laser light in the vertical direction and one scanning of the laser light in the horizontal direction. The information processing device according to claim 9 . 　 the setting unit sets a plurality of extended regions at positions obtained by dividing the horizontal length of the enabled region at approximately equal intervals and adjacent to the enabled region in a vertical direction. The information processing device according to claim 1 . 　 When the laser light irradiation means is a line scan means that shifts the laser light in the vertical direction every time it scans one line in the horizontal direction, the setting unit sets the plurality of extension areas as extension areas for line scanning. The information processing device according to claim 11. 　 When setting the plurality of extension regions, the setting unit sets the plurality of extension regions so that positions of the extension regions in the horizontal direction on lines adjacent to each other in the vertical direction are different. The information processing device according to claim 12. 　 the determination unit further determines whether the reflected light observed from the pixels of the valid area and the reflected light observed from the pixels of the extended area includes reflected light from a white object, based on the intensity of the reflected light observed from the pixels of the valid area and the intensity of the reflected light observed from the pixels of the extended area. The information processing device according to claim 1 . 　 When determining whether reflected light from the white object is included, the determination unit refers to a continuity of change from the intensity of reflected light observed from the pixels of the valid area to the intensity of reflected light observed from the pixels of the extended area. The information processing device according to claim 14. 　 When determining whether reflected light from the white object is included, the determination unit determines that reflected light from the white object is included if there is a discontinuous change from the intensity of reflected light observed from the pixels in the enabled region to the intensity of reflected light observed from the pixels in the extended region. The information processing device according to claim 15. 　 the setting unit sets a region of pixels of a SPAD (Single Photon Avalanche Diode) as the extended region; The information processing device according to claim 1 . 　 When determining whether the reflected light observed from the pixels in the extended region includes reflected light from the highly reflective object, the determination unit refers to the intensity of reflected light from a Vertical Cavity Surface Emitting Laser (VCSEL) as the intensity of the reflected light observed from the pixels in the extended region. The information processing device according to claim 1 . 　 The computer an extended region, which is a pixel region where the intensity of reflected laser light can be observed, is set around an enabled region, which is a pixel region where the intensity of reflected laser light can be observed, among the light receiving elements; determining whether the reflected light includes light reflected from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended region; An information processing method including: 　 Computer, a setting unit that sets an extended region, which is a region of pixels in which the intensity of reflected laser light can be observed, around an enabled region, which is a region of pixels in the light receiving element in which the intensity of reflected laser light can be observed; a determination unit that determines whether the reflected light includes reflected light from a highly reflective object based on the intensity of the reflected light observed from the pixels in the set extended region; An information processing program for causing an information processing device to function as an information processing device comprising:

Images