Information processing device and information processing method

Abstract

Even if the amount of noise contained in the light received by the photodetector fluctuates, it can accurately detect reflected light from an object. [Solution] The information processing device includes a distance detection unit that generates first point cloud data including distance information of objects included in the origin point cloud data, a difference extraction unit that extracts third point cloud data from the difference between the first point cloud data and second point cloud data acquired in the past, and a difference adjustment unit that generates an index for adjusting the amount of difference extracted, wherein the difference extraction unit extracts the third point cloud data from the difference based on the index.

Description

Technical Field

[0004] , , , , , , , , , , , , , , , , , , , ,

[0003] , ,

[0005]

[0001] One embodiment of the present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] Distance detection devices can accurately detect the distance of an object due to the evolution of LiDAR (Light Detection & Ranging) technology, and are thus applied in a wide range of fields such as automatic driving devices and object recognition devices.

[0003] However, in LiDAR, since the distance of an object is detected based on the result of repeatedly receiving the reflected light from the object, the amount of noise contained in the light received by LiDAR varies depending on the distance of the object, the reflectivity of the object, and the presence or absence of ambient light such as sunlight. The output level of the light receiving element of LiDAR changes depending on the presence or absence of light reception. However, if the threshold for determining the output level is set to a constant level, the reflected light from the object cannot be accurately detected due to fluctuations in the amount of noise, and there is a risk that the distance detection accuracy of the object will decrease.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

[0006] To solve the above problems, according to one embodiment of the present invention, a distance detection unit generates first point cloud data including distance information of objects included in the origin point cloud data, A difference extraction unit extracts a third point cloud data from the difference between the first point cloud data and the second point cloud data acquired in the past. The system includes a difference adjustment unit that generates an index for adjusting the amount of difference extracted, The difference extraction unit extracts the third point cloud data from the difference based on the index. An information processing device is provided. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing the schematic configuration of the information processing device according to the first embodiment. [Figure 2] A block diagram showing the schematic configuration of the information processing device according to the second embodiment. [Figure 3] A diagram showing the correspondence between the distance of an object and a threshold value in the second embodiment. [Figure 4] Figures 4A, 4B, and 4C illustrate the processing of the difference extraction unit 3. [Figure 5] A diagram illustrating the processing operation of the difference extraction unit. [Figure 6A] This figure shows the difference image when the surrounding pixel area of ​​the background image is 1x1 pixels. [Figure 6B] This figure shows the difference image when the surrounding pixel area is 3x3 pixels. [Figure 6C] This figure shows the difference image when the surrounding pixel area is 5x5 pixels. [Figure 7] A block diagram showing the schematic configuration of an information processing device according to the third embodiment. [Figure 8] A diagram showing the correspondence between the brightness of an object and the threshold value in the third embodiment. [Figure 9] A block diagram showing the schematic configuration of the information processing device according to the fourth embodiment. [Figure 10]A diagram showing the correspondence relationship among the distance, luminance, and threshold value of an object in the fourth embodiment. [Figure 11] A block diagram showing the schematic configuration of an information processing apparatus according to the fifth embodiment. [Figure 12] A diagram showing the correspondence relationship between the distance of an object and the voxel size in the fifth embodiment. [Figure 13] A block diagram showing the schematic configuration of an information processing apparatus according to the sixth embodiment. [Figure 14] A diagram showing the correspondence relationship between the luminance of an object and the voxel size in the sixth embodiment. [Figure 15] A diagram showing the correspondence relationship between the luminance of an object and the voxel size in a modified example of the sixth embodiment. [Figure 16] A block diagram showing the schematic configuration of an information processing apparatus according to the seventh embodiment. [Figure 17] A graph representing Equation (1). [Figure 18] A block diagram showing the schematic configuration of an information processing apparatus that generates an adjustment signal α inside the information processing apparatus. [Figure 19] A flowchart showing the processing operation of the information processing apparatus in FIG. 18.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of an information processing apparatus and an information processing method will be described with reference to the drawings. In the following description, the main components of the information processing apparatus will be mainly described. However, the information processing apparatus may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0009] (First Embodiment) FIG. 1 is a block diagram showing the schematic configuration of an information processing apparatus 1 according to the first embodiment. As shown in FIG. 1, the information processing apparatus 1 according to the first embodiment includes a distance detection unit 2, a difference extraction unit 3, and a difference adjustment unit 4.

[0010] The distance detection unit 2 receives the origin cloud data (RAW data) output from the light detection unit 5. The light detection unit 5 projects light onto an object and receives the light reflected by the object. For example, the light detection unit 5 scans the direction of light projection in one or two dimensions to receive reflected light from objects existing in three-dimensional space. The light detection unit 5 may also have a function to variably control the scanning range of the light. The light detection unit 5 is, for example, a LiDAR device provided separately from the information processing device 1. Alternatively, the light detection unit 5 may be built into the information processing device 1.

[0011] The distance detection unit 2 generates first point cloud data that includes distance information of objects included in the origin point cloud data. The greater the distance, the sparser the density of the first point cloud data becomes, and the closer the distance, the denser the density of the first point cloud data becomes. In this way, the distance information of an object can be detected by the density of the first point cloud data. The first point cloud data is data in which distance information is added to the data of each point. More precisely, each point has orientation information (polar coordinates φ and θ components) corresponding to the direction of irradiation of the laser beam of the LiDAR device, and distance information d to an object located in the orientation of each point. The distance image is a two-dimensional image in which the distance information d of each point is arranged. The first point cloud data may include the distance image, or it may include data obtained by converting the polar coordinates φ, θ and distance information d of each point to XYZ coordinates.

[0012] The difference extraction unit 3 extracts the third point cloud data from the difference between the first point cloud data generated by the distance detection unit 2 and the second point cloud data acquired in the past. The second point cloud data acquired in the past may be the point cloud data acquired in the past itself, or it may be point cloud data obtained by performing an averaging process on the point cloud data acquired in the past. In this specification, the difference between the first point cloud data and the second point cloud data is sometimes referred to as difference point cloud data. The first point cloud data is generated based on the origin point cloud data output from the light detection unit 5.

[0013] The second point cloud data is a three-dimensional background image for one frame, and the point cloud does not change over time. The information processing device 1 according to the first embodiment may include a storage unit (not shown) for storing the second point cloud data.

[0014] The first point cloud data is a three-dimensional depth image that includes a background image composed of the second point cloud data. More specifically, the first point cloud data is a depth image that adds object distance information to the background image. The difference point cloud data between the first and second point cloud data shows the pixel-by-pixel distance difference between the depth image and the background image.

[0015] The third point cloud data extracted from the difference point cloud data is point cloud data obtained by extracting at least a portion of the difference (difference point cloud data) between the first point cloud data and the second point cloud data. In other words, the third point cloud data is the difference image between the depth image and the background image. As will be described later, the difference extraction unit 3 extracts the third point cloud data from the difference point cloud data by removing noise contained in the difference point cloud data, for example.

[0016] The difference adjustment unit 4 generates an index for adjusting the amount of difference (difference point cloud data) extracted. For example, the difference adjustment unit 4 generates the above-mentioned index based on the first point cloud data or two-dimensional image data. Alternatively, the difference adjustment unit 4 may generate the above-mentioned index based on the object recognition result.

[0017] The index is, for example, a threshold for comparison with the difference (difference point cloud data). Alternatively, the index is the size of the voxel (hereinafter referred to as voxel size). A voxel refers to a three-dimensional unit region obtained by dividing the first point cloud data and the second point cloud data in three-dimensional space into sizes corresponding to the distance of an object. Inside each voxel is the divided first point cloud data or the second point cloud data. A voxel has a size corresponding to the distance of the object corresponding to the point cloud data contained inside the voxel.

[0018] If the index is voxel size, the difference extraction unit 3 takes the difference between the multiple voxels that divide the first point cloud data and the multiple voxels that divide the second point cloud data. For example, if the first point cloud data includes point cloud data of an object not present in the background image, it will have new voxels that do not exist in the multiple voxels that divide the second point cloud data. By taking the difference, voxels that do not exist in the multiple voxels that divide the second point cloud data are extracted. If the index is voxel size, the difference extraction unit 3 extracts the point cloud data within the voxels obtained by taking the difference as the third point cloud data.

[0019] As shown in Figure 1, the information processing device 1 according to the first embodiment may include an object recognition unit 6. The object recognition unit 6 recognizes an object based on the third point cloud data extracted by the difference extraction unit 3. As described above, the third point cloud data is point cloud data extracted from the difference (difference point cloud data) between the first point cloud data constituting the depth image and the second point cloud data constituting the background image, and the third point cloud data contains object information. Furthermore, since the difference extraction unit 3 extracts third point cloud data that exceeds a threshold from the difference point cloud data, the third point cloud data is data with noise components suppressed. Therefore, the object recognition unit 6 can accurately recognize an object from the third point cloud data.

[0020] The information processing device 1 according to the first embodiment may detect only the presence or absence of an object based on the third point cloud data, without performing object recognition.

[0021] Thus, in the first embodiment, an index is generated to adjust the amount of difference (difference point cloud data) extracted, and the generated index is used to extract third point cloud data from the difference (difference point cloud data). This allows the index to be adjusted, for example, by the amount of noise contained in the first point cloud data, making the third point cloud data less susceptible to noise. Therefore, objects can be recognized with high accuracy from the third point cloud data.

[0022] (Second embodiment) Figure 2 is a block diagram illustrating the schematic configuration of the information processing device 1 according to the second embodiment. The information processing device 1 according to the second embodiment shown in Figure 2 has the same block configuration as in Figure 1, but the difference adjustment unit 4 receives the first point cloud data generated by the distance detection unit 2 as input. In Figure 1, the difference adjustment unit 4 does not necessarily receive the first point cloud data generated by the distance detection unit 2 as input, so the arrow lines indicating the input path of the difference adjustment unit 4 are omitted in Figure 1.

[0023] The difference adjustment unit 4 in Figure 2 generates a threshold as an indicator for extracting third point cloud data from the difference point cloud data between the first point cloud data and the second point cloud data, based on the distance information detected by the distance detection unit 2. The difference extraction unit 3 extracts third point cloud data based on the difference point cloud data that exceeds the threshold. The difference point cloud data is data that shows the distance difference for each pixel between the distance image and the background image, and the third point cloud data that constitutes the difference image is the point cloud data of pixels that have a distance difference of more than or equal to the threshold.

[0024] Figure 3 shows the relationship between the distance of an object and the threshold in the second embodiment. In Figure 3, the horizontal axis represents the distance of the object, and the vertical axis represents the threshold. As shown in Figure 3, the threshold increases as the distance increases, and decreases as the distance decreases. As the distance of the object increases, the density of the first point cloud data becomes coarser and the amount of noise increases, so the threshold is increased. Figure 3 shows an example where the relationship between distance and threshold is linear, but it may also be nonlinear.

[0025] Figure 4 illustrates the processing of the difference extraction unit 3. Figure 4A shows an example of a distance image IG1 consisting of first point cloud data generated by the distance detection unit. Figure 4B shows an example of a background image IG2 consisting of second point cloud data. Figure 4C shows a difference image IG3 between the distance image IG1 in Figure 4A and the background image IG2 in Figure 4B. Although the distance image IG1, background image IG2, and difference image IG3 are all point cloud data, the notation for point cloud is omitted in Figure 4 for simplicity.

[0026] The difference extraction unit 3 extracts the third point cloud data (Figure 4C) that constitutes the difference image IG3, which exceeds a threshold, from the difference point cloud data (IG1 consisting of the first point cloud data in Figure 4A and the background image IG2 consisting of the second point cloud data in Figure 4B).

[0027] Figure 5 illustrates the processing operation of the difference extraction unit 3. The difference extraction unit 3 sequentially selects each of the multiple pixels constituting the depth image IG1, which consists of the first point cloud data, as the target pixel SPX. If there is even one pixel in the surrounding pixel region PPA centered on the pixel of the background image IG2, which consists of the second point cloud data corresponding to the selected target pixel SPX, that pixel SPX is removed from the candidate for the difference image. This efficiently removes noise components contained in the depth image IG1 and reduces the amount of data in the third point cloud data that constitutes the difference image.

[0028] Figure 6A shows the difference image when the surrounding pixel area of ​​the background image is 1x1 pixels, Figure 6B shows the difference image when the surrounding pixel area is 3x3 pixels, and Figure 6C shows the difference image when the surrounding pixel area is 5x5 pixels.

[0029] As can be seen from Figures 6A to 6C, the larger the area of ​​the surrounding pixel region of the background image compared to the target pixel in the depth image, the more the amount of data in the third point group included in the difference image can be reduced.

[0030] Note that the processing operation of the difference extraction unit 3 does not necessarily have to be performed using the method shown in Figure 5. For example, the decision of whether or not to include the pixel of interest in the difference image may be made based on whether or not the distance difference based on the difference between the average value of pixels in the surrounding pixel region and the pixel of interest exceeds a threshold.

[0031] Thus, in the second embodiment, the threshold for extracting third point cloud data from difference point cloud data is adjusted according to the distance of the object. For example, when the distance of the object is large, the threshold is increased compared to when it is small, making the third point cloud data less susceptible to noise. Therefore, according to the second embodiment, even if the distance of the object changes, fluctuations in the extraction accuracy of the third point cloud data can be suppressed.

[0032] (Third embodiment) Figure 7 is a block diagram showing the schematic configuration of the information processing device 1 according to the third embodiment. As shown in Figure 7, the information processing device 1 according to the third embodiment includes a brightness detection unit 7 in addition to the configuration shown in Figure 2.

[0033] The light detection unit 5 outputs both origin group data (RAW data) and two-dimensional image data. The two-dimensional image data contains brightness information for each pixel.

[0034] The luminance detection unit 7 detects luminance information for each pixel based on two-dimensional image data. The luminance detection unit 7 may determine the luminance value of a target pixel by taking the average of the luminance values ​​of multiple pixels surrounding the target pixel.

[0035] The difference adjustment unit 4 generates a threshold value as an indicator for extracting third point cloud data from the difference point cloud data, based on the brightness information for each pixel detected by the brightness detection unit 7. For example, the difference adjustment unit 4 lowers the threshold value for the corresponding location in the difference point cloud data for pixels with high brightness values.

[0036] Figure 8 shows the relationship between object brightness and threshold in the third embodiment. In Figure 8, the horizontal axis represents object brightness, and the vertical axis represents the threshold. As shown in Figure 7, the threshold decreases as brightness increases, and increases as brightness decreases. When brightness is high, it is easy to distinguish from noise contained in the difference point cloud data, so the threshold is lowered. On the other hand, when brightness is low, it is difficult to distinguish from noise contained in the difference point cloud data, so the threshold is higher.

[0037] Thus, in the third embodiment, the threshold for extracting third point cloud data from the difference is adjusted by brightness, thereby improving the extraction accuracy of the third point cloud data.

[0038] (Fourth embodiment) Figure 9 is a block diagram illustrating the schematic configuration of the information processing device 1 according to the fourth embodiment. As shown in Figure 9, the information processing device 1 according to the fourth embodiment has a block configuration similar to that of Figure 7, but the difference adjustment unit 4 in Figure 9 performs a different processing operation than the difference adjustment unit 4 in Figure 7, and the difference extraction unit 3 in Figure 9 performs a different processing operation than the difference extraction unit 3 in Figure 7.

[0039] The difference adjustment unit 4 in Figure 9 generates a threshold value as an indicator for extracting third point cloud data from difference point cloud data, based on distance information and brightness information. More specifically, the difference adjustment unit 4 variably controls the correspondence between the object's distance and the threshold value based on the object's brightness.

[0040] The difference extraction unit 3 in Figure 9 extracts third point cloud data based on the difference point cloud data that exceeds a threshold. More specifically, the difference extraction unit 3 variably controls the threshold according to the distance and brightness of the object.

[0041] Figure 10 shows the correspondence between object distance, brightness, and threshold in the fourth embodiment. In Figure 10, the horizontal axis represents the object distance, and the vertical axis represents the threshold. Figure 10 illustrates the correspondence w1 for high brightness and the correspondence w2 for low brightness. Correspondences w1 and w2 have different slopes. That is, the degree of change in the threshold when the object distance changes differs between high brightness and low brightness. The higher the brightness of the object, the smaller the degree of change in the threshold. Correspondences w1 and w2 in Figure 10 are just examples.

[0042] Thus, in the fourth embodiment, the threshold is variably controlled according to the distance and brightness of the object, so that an optimal threshold can be set according to the distance and brightness of the object, and the accuracy of extracting third point cloud data from difference point cloud data can be further improved.

[0043] (Fifth embodiment) Figure 11 is a block diagram showing the schematic configuration of the information processing device 1 according to the fifth embodiment. As shown in Figure 11, the information processing device 1 according to the fifth embodiment comprises a distance detection unit 2, a difference extraction unit 3 including a voxel generation unit 8, and a difference adjustment unit 4.

[0044] The distance detection unit 2 generates first point cloud data that includes distance information of objects contained in the origin point cloud data (RAW data).

[0045] The voxel generation unit 8 divides the first point cloud data into multiple voxels based on an index, and also divides the second point cloud data into multiple voxels based on an index. The first point cloud data includes distance information of objects. Specifically, the first point cloud data is point cloud data with density corresponding to the distance of the objects, and distance information is added to the data of each point. If the first point cloud data includes multiple point cloud data corresponding to multiple objects, each point cloud data is stored in a voxel of a size corresponding to the distance.

[0046] The multiple voxels that divide the first point cloud data are not all the same size; their sizes depend on the distance information of the object corresponding to the point cloud data within each voxel. For example, if the point cloud data contained in a voxel relates to a distant object, the voxel size will be larger. On the other hand, if the point cloud data contained in a voxel relates to a nearby object, the voxel size will be smaller.

[0047] The difference adjustment unit 4 generates voxel sizes that correlate with distance as an index. The difference adjustment unit 4 can adjust the degree of change in voxel size in response to changes in distance using this index.

[0048] The voxel generation unit 8 generates multiple voxels by dividing the first point cloud data based on an index, and multiple voxels by dividing the second point cloud data based on an index. The difference extraction unit 3 extracts the point cloud data contained in the difference voxels between the multiple voxels divided from the first point cloud data based on an index and the multiple voxels divided from the second point cloud data based on an index, as the third point cloud data.

[0049] The difference between the multiple voxels obtained by dividing the first point cloud data based on an index and the multiple voxels obtained by dividing the second point cloud data based on the index contains voxels that are different from the multiple voxels that divided the second point cloud data. The voxels in the difference contain point cloud data representing objects. The size of the voxels in the difference is proportional to the distance of the objects within that voxel.

[0050] Figure 12 shows the index generated by the difference adjustment unit 4 in the fifth embodiment. In Figure 12, the horizontal axis represents the distance of the object, and the vertical axis represents the voxel size. As shown in Figure 12, the index includes information indicating the degree of change in voxel size in response to a change in the distance of the object.

[0051] By adjusting the index in the difference adjustment unit 4, the voxel size of the voxels extracted by the difference extraction unit 3 is variably controlled.

[0052] Thus, in the fifth embodiment, the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index is taken, and the third point cloud data is extracted from the voxels of the difference. In the fifth embodiment, the degree of change in voxel size in response to changes in the distance of an object is adjusted by the index, and the third point cloud data is extracted using the adjusted voxel size, thereby improving the accuracy of extracting the third point cloud data.

[0053] (Sixth embodiment) Figure 13 is a block diagram showing the schematic configuration of the information processing device 1 according to the sixth embodiment. As shown in Figure 13, the information processing device 1 according to the sixth embodiment includes a brightness detection unit 7 in addition to the configuration shown in Figure 11. The brightness detection unit 7 detects brightness information for each pixel based on two-dimensional image data input together with the origin cloud data.

[0054] The difference adjustment unit 4 generates voxel sizes correlated with distance as indicators based on luminance information. The difference extraction unit 3 has a voxel generation unit 8. The voxel generation unit 8 generates multiple voxels by dividing the first point cloud data based on the indicators, and multiple voxels by dividing the second point cloud data based on the indicators. The difference extraction unit 3 extracts the third point cloud data within the voxels of the difference between the multiple voxels divided from the first point cloud data based on the indicators and the multiple voxels divided from the second point cloud data based on the indicators.

[0055] Figure 14 shows the index generated by the difference adjustment unit 4 in the sixth embodiment. In Figure 14, the horizontal axis represents the brightness of the object, and the vertical axis represents the voxel size. The index contains information indicating the degree of change in voxel size in response to changes in the brightness of the object. As shown in Figure 14, as the brightness of the object increases, the voxel size, which is the index, decreases.

[0056] The difference extraction unit 3 extracts the point cloud data contained in the voxel of the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index, as the third point cloud data. This allows for the extraction of third point cloud data with less noise.

[0057] As one modified example of the information processing device 1 according to the sixth embodiment, the difference adjustment unit 4 may generate voxel size as an indicator based on distance information and brightness information.

[0058] Figure 15 shows the index generated by the difference adjustment unit 4 in one modified example of the sixth embodiment. In Figure 15, the horizontal axis represents the brightness of the object, and the vertical axis represents the voxel size. Figure 15 shows the index w3 for high brightness and the index w4 for low brightness. Indicators w3 and w4 have different slopes. The higher the brightness of the object, the smaller the degree of change in voxel size. Indicators w3 and w4 in Figure 15 are just examples.

[0059] Thus, in the sixth embodiment, the voxel size, which is an index, is adjusted according to the brightness of the object, and the third point cloud data is extracted from the voxel of the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index. This makes it possible to extract third point cloud data with less noise, regardless of brightness.

[0060] (Seventh Embodiment) Figure 16 is a block diagram showing the schematic configuration of the information processing device 1 according to the seventh embodiment. The block configuration of the information processing device 1 according to the seventh embodiment shown in Figure 16 is the same as in Figure 1, but the differential adjustment unit 4 in Figure 16 has a different processing operation than the differential adjustment unit 4 in Figure 1.

[0061] The difference adjustment unit 4 in Figure 1 adjusts the index based on, for example, first point cloud data or two-dimensional image data, whereas the difference adjustment unit 4 in Figure 16 adjusts the index based on an adjustment signal α and a function f (distance or brightness). The function f (distance or brightness) is a function that calculates the index using at least one of distance or brightness as an input parameter. The shape of the function f is arbitrary. The index is, for example, a threshold or voxel size. The difference adjustment unit 4 adjusts the index based on, for example, the following equation (1). The adjustment signal α is a real number other than 0.

[0062] Index = f (distance or brightness) × α …(1) Figure 17 is a graph representing equation (1). In Figure 17, the horizontal axis represents the distance of the object, and the vertical axis represents the threshold or voxel size. The shape of the function f, for example, the slope of the function f, can be adjusted by the adjustment signal α.

[0063] The adjustment signal α may be input from outside the information processing device 1, or it may be a signal generated inside the information processing device 1.

[0064] Figure 18 is a block diagram illustrating the schematic configuration of an information processing device 1 that generates an adjustment signal α internally. The information processing device 1 in Figure 18 includes processing blocks related to the generation of the adjustment signal α. Specifically, the information processing device 1 in Figure 18 includes, in addition to the configuration of the information processing device 1 in Figure 1, an evaluation unit 9 and a test image storage unit 10.

[0065] The object recognition unit 6 recognizes objects based on the third point cloud data extracted by the difference extraction unit 3. The object recognition unit 6 has, for example, an object recognition model, and by inputting the third point cloud data extracted by the difference extraction unit 3 into the object recognition model, it outputs the object recognition result from the object recognition model.

[0066] The difference extraction unit 3 receives training images for training the object recognition model and test images for evaluating the trained object recognition model.

[0067] The evaluation unit 9 evaluates the object recognition accuracy of the object recognition unit 6 based on the object recognition results output from the object recognition unit 6 when a test image is input to the object recognition unit 6.

[0068] The difference adjustment unit 4 adjusts the adjustment signal α based on the evaluation result of the evaluation unit 9 and updates the index based on the adjusted adjustment signal α. The difference extraction unit 3 repeats the processing of the evaluation unit 9 and the difference adjustment unit 4 a predetermined number of times, and then extracts the third point cloud data from the difference point cloud data based on the highest accuracy index evaluated by the evaluation unit 9.

[0069] Figure 19 is a flowchart showing the processing operation of the information processing device 1 shown in Figure 18. First, the difference extraction unit 3 sets an index using the initial value α0 of the adjustment signal (step S1). The index is, for example, a threshold or a box size.

[0070] Next, the object recognition unit 6 trains an object recognition model using the indicators and training images set in step S1 (step S2). Here, the object recognition model is a model that takes the input image and indicators as input parameters and outputs the recognition result of objects contained in the input image. As for the training images, since the object information contained in the training images is known in advance, in step S2, the object recognition result output from the object recognition model is compared with the known object information to train the object recognition model.

[0071] Next, the evaluation unit 9 inputs test images and metrics that were not used during training into the object recognition model and evaluates the accuracy of the object recognition model (step S3).

[0072] Next, it is determined whether the processes in steps S2 to S5 have been repeated the prescribed number of times (step S4). If step S4 is NO, the adjustment signal α is adjusted and the index is updated (step S5). After that, the processes from step S2 onwards are performed using the updated index.

[0073] If step S4 is YES, the index corresponding to the most accurate adjustment signal α is ultimately selected (step S6).

[0074] Thus, in the seventh embodiment, the index can be arbitrarily adjusted by the adjustment signal α, making it easy to find the index that provides the best object recognition accuracy for the object recognition unit 6.

[0075] [Note] [Item 1] A distance detection unit that generates first point cloud data including distance information of objects included in the origin point cloud data, A difference extraction unit extracts a third point cloud data from the difference between the first point cloud data and the second point cloud data acquired in the past. The system includes a difference adjustment unit that generates an index for adjusting the amount of difference extracted, The difference extraction unit extracts the third point cloud data from the difference based on the index. Information processing device. [Item 2] The aforementioned second point cloud data constitutes a three-dimensional background image for one frame. The first point cloud data constitutes a three-dimensional distance image including the background image. The information processing device described in item 1. [Item 3] The difference adjustment unit generates a threshold value as the index for adjusting the amount of difference extracted based on the distance information included in the first point cloud data. The difference extraction unit extracts the third point cloud data based on the difference that exceeds the threshold. An information processing device as described in item 1 or 2. [Item 4] The difference adjustment unit controls the threshold value variably according to the distance of the object. The information processing device described in item 3. [Item 5] The difference adjustment unit increases the threshold as the distance of the object increases. The information processing device described in item 4. [Item 6] The system includes a brightness detection unit that detects brightness information for each pixel based on two-dimensional image data input together with the aforementioned origin cloud data. The difference adjustment unit generates a threshold value as the index for adjusting the amount of difference extracted based on the brightness information. The difference extraction unit extracts the third point cloud data based on the difference that exceeds the threshold. An information processing device as described in item 1 or 2. [Item 7] The difference adjustment unit variably controls the threshold according to the brightness of each pixel. The information processing device described in item 6. [Item 8] The difference adjustment unit reduces the threshold as the brightness increases. Information processing device as described in item 6 or 7. [Item 9] The system includes a brightness detection unit that detects brightness information for each pixel based on two-dimensional image data input together with the aforementioned origin cloud data. The difference adjustment unit generates a threshold value as the index for adjusting the amount of difference extracted, based on the distance information and the brightness information. The difference extraction unit extracts the third point cloud data based on the difference that exceeds the threshold. An information processing device as described in item 1 or 2. [Item 10] The difference extraction unit variably controls the threshold according to the distance and brightness of the preceding object. The information processing device described in item 9. [Item 11] The difference adjustment unit variably controls the correspondence between the distance of the object and the threshold value based on the brightness of the object. The information processing device described in item 10. [Item 12] The difference extraction unit extracts the third point cloud data within the voxel representing the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index. The difference adjustment unit generates the size of the voxel used to adjust the amount of difference extracted as the index. An information processing device as described in item 1 or 2. [Item 13] The difference adjustment unit controls the size of the voxel variably according to the distance of the object. The information processing device described in item 12. [Item 14] The difference adjustment unit increases the size of the voxel as the distance of the object increases. The information processing device described in item 13. [Item 15] The system includes a brightness detection unit that detects the brightness information of each of the multiple voxels obtained by dividing the first point cloud data and the second point cloud data based on the index, based on two-dimensional image data input together with the origin point cloud data. The difference extraction unit extracts the third point cloud data within the voxel of the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index, based on the brightness information. An information processing device as described in item 1 or 2. [Item 16] The difference adjustment unit variably controls the index that represents the degree of change in the size of the voxel in response to a change in the distance of the object, based on the brightness information for each voxel. The information processing device described in item 15. [Item 17] The system includes an object recognition unit that recognizes an object based on the aforementioned third point cloud data. An information processing device as described in any one of items 1 through 16. [Item 18] The difference adjustment unit adjusts the index based on the adjustment signal. The difference extraction unit extracts the third point cloud data from the difference based on the index adjusted by the difference adjustment unit. An information processing device as described in any one of items 1 through 17. [Item 19] The system includes an evaluation unit that evaluates the accuracy of the third point cloud data extracted by the difference extraction unit using the first point cloud data of the test image, The difference adjustment unit adjusts the index based on the adjustment signal based on the evaluation result of the evaluation unit. The difference extraction unit, after the evaluation unit and the difference adjustment unit have alternately repeated the processing of the evaluation unit and the difference adjustment unit a predetermined number of times, extracts the third point cloud data from the difference based on the index with the highest accuracy evaluated by the evaluation unit. The information processing device described in item 18. [Item 20] A third point cloud data is extracted based on the difference between the newly acquired first point cloud data and the previously acquired second point cloud data. Based on the first point cloud data, an index is generated to adjust the amount of difference extracted. Based on the aforementioned indicator, the third point cloud data is extracted from the difference. Information processing methods.

[0076] The aspects of this disclosure are not limited to the individual embodiments described above, but include various modifications that a person skilled in the art could conceive, and the effects of this disclosure are not limited to those described above. In other words, various additions, modifications, and partial deletions are possible, as long as they do not depart from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents. [Explanation of Symbols]

[0077] 1. Information processing unit, 2. Distance detection unit, 3. Difference extraction unit, 4. Difference adjustment unit, 5. Light detection unit, 6. Object recognition unit, 7. Brightness detection unit, 8. Voxel generation unit, 9. Evaluation unit, 10. Test image storage unit

Claims

1. A distance detection unit that generates first point cloud data including distance information of objects included in the origin point cloud data, A difference extraction unit extracts a third point cloud data from the difference between the first point cloud data and the second point cloud data acquired in the past. The system includes a difference adjustment unit that generates an index for adjusting the amount of difference extracted, The difference extraction unit extracts the third point cloud data from the difference based on the index. Information processing device.

2. The aforementioned second point cloud data constitutes a three-dimensional background image for one frame. The first point cloud data constitutes a three-dimensional distance image including the background image. The information processing apparatus according to claim 1.

3. The difference adjustment unit generates a threshold value as the index for adjusting the amount of difference extracted based on the distance information included in the first point cloud data. The difference extraction unit extracts the third point cloud data based on the difference that exceeds the threshold. The information processing apparatus according to claim 1.

4. The difference adjustment unit controls the threshold value variably according to the distance of the object. The information processing apparatus according to claim 3.

5. The difference adjustment unit increases the threshold as the distance of the object increases. The information processing apparatus according to claim 4.

6. The system includes a brightness detection unit that detects brightness information for each pixel based on two-dimensional image data input together with the aforementioned origin cloud data. The difference adjustment unit generates a threshold value as the index for adjusting the amount of difference extracted based on the brightness information. The difference extraction unit extracts the third point cloud data based on the difference that exceeds the threshold. The information processing apparatus according to claim 1.

7. The difference adjustment unit variably controls the threshold according to the brightness of each pixel. The information processing apparatus according to claim 6.

8. The difference adjustment unit reduces the threshold as the brightness increases. The information processing apparatus according to claim 6.

9. The system includes a brightness detection unit that detects brightness information for each pixel based on two-dimensional image data input together with the aforementioned origin cloud data. The difference adjustment unit generates a threshold value as the index for adjusting the amount of difference extracted, based on the distance information and the brightness information. The difference extraction unit extracts the third point cloud data based on the difference that exceeds the threshold. The information processing apparatus according to claim 1.

10. The difference extraction unit variably controls the threshold according to the distance and brightness of the preceding object. The information processing apparatus according to claim 9.

11. The difference adjustment unit variably controls the correspondence between the distance of the object and the threshold value based on the brightness of the object. The information processing apparatus according to claim 10.

12. The difference extraction unit extracts the third point cloud data within the voxel representing the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index. The difference adjustment unit generates the size of the voxel used to adjust the amount of difference extracted as the index. The information processing apparatus according to claim 1.

13. The difference adjustment unit controls the size of the voxel variably according to the distance of the object. The information processing apparatus according to claim 12.

14. The difference adjustment unit increases the size of the voxel as the distance of the object increases. The information processing apparatus according to claim 13.

15. The system includes a brightness detection unit that detects the brightness information of each of the multiple voxels obtained by dividing the first point cloud data and the second point cloud data based on the index, based on two-dimensional image data input together with the origin point cloud data. The difference extraction unit extracts the third point cloud data within the voxel of the difference between the multiple voxels obtained by dividing the first point cloud data based on the index and the multiple voxels obtained by dividing the second point cloud data based on the index, based on the brightness information. The information processing apparatus according to claim 1.

16. The difference adjustment unit variably controls the index that represents the degree of change in the size of the voxel in response to a change in the distance of the object, based on the brightness information for each voxel. The information processing apparatus according to claim 15.

17. The system includes an object recognition unit that recognizes an object based on the aforementioned third point cloud data. The information processing apparatus according to claim 1.

18. The difference adjustment unit adjusts the index based on the adjustment signal. The difference extraction unit extracts the third point cloud data from the difference based on the index adjusted by the difference adjustment unit. The information processing apparatus according to claim 1.

19. The system includes an evaluation unit that evaluates the accuracy of the third point cloud data extracted by the difference extraction unit using the first point cloud data of the test image, The difference adjustment unit adjusts the index based on the adjustment signal based on the evaluation result of the evaluation unit. The difference extraction unit, after the evaluation unit and the difference adjustment unit have alternately repeated the processing of the evaluation unit a predetermined number of times, extracts the third point cloud data from the difference based on the index with the highest accuracy evaluated by the evaluation unit. The information processing apparatus according to claim 18.

20. A third point cloud data is extracted based on the difference between the newly acquired first point cloud data and the previously acquired second point cloud data. Based on the first point cloud data, an index is generated to adjust the amount of difference extracted. Based on the aforementioned indicator, the third point cloud data is extracted from the difference. Information processing methods.

Images