Information processing device, information processing method, and program

Abstract

To determine whether a feature point indicating a feature of an object has been obtained from a space in a field of view of imaging means with high accuracy while suppressing a processing load.SOLUTION: An information processing device acquires a stereo image, performs matching for a first number of feature points less than the number of pixels of a first image, estimates a three-dimensional position of the feature point including a distance in a real space from a stereo camera to a position indicated by the feature point, sets as an attention point the feature point for determining that it has been obtained from the space set in the field of view of the imaging means among the first number of feature points, sets a second number of peripheral points existing in an image area in a predetermined range from the attention point in the first image and more than the number of feature points estimated in the image area, and determines whether the attention point is the feature point indicating a feature of an object in the space on the basis of a difference between a distance from a predetermined position of the stereo camara to a position indicated by the peripheral point and a distance from the predetermined position of the stereo camera to a position indicated by the attention point, which is obtained by performing matching for the peripheral point.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technique for obtaining object features from a range image. [Background technology]

[0002] There is a technology that uses images captured by a stereo camera as input, calculates a distance image based on stereo matching, and detects whether there is an object in front of the camera (Non-Patent Document 1). This technology is used, for example, to detect objects (people, obstacles, etc.) around robots and automobiles.

[0003] The range image obtained by stereo matching may contain noise due to matching failure. Methods for detecting and removing this noise include noise removal filters such as median filters and speckle filters. These noise detection and removal methods refer to multiple distance values ​​within a local image region of the range image, find distance values ​​that are statistically in the minority, and remove them as noise. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] NBNaveen Appiah,“Obstacle detection using stereo vision for self-driving cars”.IEEE Intelligent Vehicles Symposium,2011 Summary of the Invention [Problem to be solved by the invention]

[0005] To use a noise detection method based on statistical calculations that reference multiple distance values, it is assumed that the original range image has dense distance values ​​for each local image region. However, generating a dense range image based on stereo matching and then performing noise detection from that dense range image imposes a heavy processing load. When detecting noise using a camera installed on a moving body, it is desirable to ensure the accuracy of noise detection while keeping the processing load low.

[0006] The present invention has been made in consideration of the above problems, and aims to determine whether feature points indicating the characteristics of an object have been obtained from a three-dimensional space in the direction of travel of a moving body, with high accuracy and while reducing the processing load. [Means for solving the problem]

[0007] The information processing device according to the present invention has the following configuration: That is, images are obtained from an imaging means including a first optical system and a second optical system arranged on a moving body so that at least a part of the imaging field of view overlaps, and the image acquisition means acquires a first image from the first optical system and a second image from the second optical system, an estimation means performs stereo matching between the first image and the second image for a first number of feature points, the first number being less than the number of pixels in the first image, and estimates the three-dimensional positions of the feature points relative to the imaging means, and an estimation means determines, based on the three-dimensional positions, that the first number of feature points have been obtained from within a three-dimensional space set in the traveling direction of the moving body. a surrounding point setting means for setting a second number of surrounding points, the second number being greater than the number of feature points whose three-dimensional positions have been estimated by the estimation means, within an image area within a predetermined distance range from the attention point in the first image; and a determination means for determining whether the attention point is a feature point that indicates a characteristic of an object existing in the three-dimensional space based on the difference between the three-dimensional positions of the surrounding points relative to the imaging means, which are obtained by stereo matching the surrounding points using the first image and the second image, and the three-dimensional position of the attention point relative to the imaging means. [Effects of the Invention]

[0008] According to the present invention, it is possible to determine with high accuracy whether feature points indicating the features of an object have been obtained from within the three-dimensional space within the field of view of the imaging means while suppressing the processing load. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the arrangement conditions of a stereo camera and an object. [Figure 2] FIG. 10 is a diagram showing an example of an image captured by a stereo camera. [Figure 3] FIG. 2 is a diagram showing an example of the functional configuration of an information processing apparatus. [Figure 4] FIG. 10 is a flowchart showing the flow of information processing. [Figure 5] FIG. 10 is a flowchart showing the flow of information processing. [Figure 6] FIG. 10 is a flowchart showing the flow of information processing. [Figure 7] 1 shows an example of the hardware configuration of an information processing device. [Figure 8] 1A and 1B are diagrams showing variations of a stereo camera. DETAILED DESCRIPTION OF THE INVENTION

[0010] [First embodiment] In the first embodiment, a method for detecting the presence or absence of an object within the forward field of view of a stereo camera (in the direction of travel of a moving object on which the stereo camera is installed) is considered. Stereo cameras are mounted, for example, on autonomous mobile robots (AMRs), automatic guided vehicles (AGVs), and autonomous mobile vehicles. Stereo cameras can be mounted on moving devices (vehicles) as well as stationary devices such as surveillance cameras. FIG. 1 shows an example of the arrangement conditions of a stereo camera and an object in this embodiment. In FIG. 1, reference numeral 100 denotes a stereo camera, reference numeral 110 denotes a three-dimensional detection space for determining the presence or absence of an object, and reference numeral 200 denotes the object. The stereo camera 100 performs distance measurement based on stereo matching of two images (a first image obtained from a first imaging device (optical system) and a second image obtained from a second imaging device (optical system)) as shown in FIG. 2 (300 and 310). The presence or absence of an object is determined based on whether the calculated point is within the detection space 110. The measured distance is, for example, the distance from the imaging plane of the left camera of the stereo camera to a feature of the object (a feature point on the image). Of course, the distance from the imaging plane of the right camera of the stereo camera to the feature of the object can be used, or the distance from the midpoint between the predetermined positions of each camera to the feature of the object can be used. The height of the lowest part of the detection space 110 is set higher than the moving surface. This is to prevent feature points obtained from an image capturing the moving surface from being used in the subsequent determination process. The height of the detection space 110 is set, for example, to a height obtained by applying a predetermined coefficient to the height of the moving object. Furthermore, the width and depth of the detection space 110 are set based on the traveling speed of the moving object, to lengths that make it highly likely that it will be difficult to avoid the obstacle, even if, for example, the steering wheel is turned or the brakes are applied.

[0011] A range image obtained by stereo matching may contain noise due to matching failures. In this embodiment, noise refers to data containing distance values ​​for each pixel in the range image that deviate from the actual distance value (true value) of the part of the image represented by each pixel. Methods for removing this noise include noise removal filters such as median filters and speckle filters. These noise removal methods reference distance values ​​within local image regions of the range image, find distance values ​​that deviate significantly from the average (or median) among the distance values, and remove these as noise. To perform such statistical calculations, the original range image must have multiple distance values ​​for each local image region. Hereinafter, such a condition will be referred to as a dense range image. For example, a range image in which distance values ​​are estimated for all pixels in a captured stereo image is dense. Conversely, a range image in which distance values ​​are estimated for a small number of pixels and distance values ​​are sparsely distributed is referred to as a sparse range image.

[0012] In this embodiment, to reduce computational costs, stereo matching is performed only on sparsely set points, as shown by 301 in FIG. 2. These set points are called candidate points. Next, if any of the candidate points 301 exist within the detection space 110, the point is designated as a focus point 302 and a determination is made as to whether it is noise. When determining whether noise exists, multiple points 303 within a predetermined range from the focus point are designated, and distance values ​​for these multiple points are calculated based on stereo matching. Then, based on the distribution of the calculated distance values, a determination is made as to whether the distance value of the focus point is noise. If it is not noise, a determination is made that an object exists within the detection space 110. When determining whether noise exists, stereo matching is performed using a limited number of candidate points, as shown by 301, and surrounding points set around them. This allows for object detection with reduced computational costs compared to a method of determining noise in a dense range image and then detecting an object.

[0013] Hereinafter, the present embodiment will be described in detail. First, the module configuration of the present embodiment will be described. In FIG. 3, reference numeral 100 denotes a stereo camera, 400 denotes an information processing device, 410 denotes a candidate point setting unit, and 420 denotes a candidate point distance estimation unit. The information processing device 400 comprises an image acquisition unit 401, a point of interest setting unit 402, a surrounding point setting unit 403, a surrounding point distance acquisition unit 404, and a determination unit 405. The information processing device 400, the candidate point setting unit 410, and the candidate point distance estimation unit 420 each include a storage unit and a calculation unit (not shown), and the information processing device 400 can be realized by, for example, a general-purpose computer.

[0014] 7 shows the hardware configuration of information processing device 400. It is composed of a CPU, ROM, RAM, a storage unit such as an HDD or SSD, a general-purpose I / F such as USB, a communication I / F, and a system bus. The CPU uses RAM as work memory to execute an operating system (OS) and various computer programs stored in the ROM, storage unit, etc., and controls each unit via the system bus. For example, the programs executed by the CPU include programs for performing the processes described below.

[0015] The stereo camera 100 is composed of two cameras (a first imaging device (optical system) and a second imaging device (optical system)) that capture two-dimensional images. The camera parameters are assumed to be known. The first imaging device and the second imaging device are arranged so that their imaging fields of view overlap at least partially. The candidate point setting unit 410 sets candidate points on images captured by the stereo cameras. The candidate point distance estimation unit 420 calculates distance values ​​of candidate points in real space by performing stereo matching based on two images captured by the stereo camera 100 (a first image acquired from the first imaging device and a second image acquired from the second imaging device). FIG. 3 illustrates a case where the candidate point setting unit 410 is located outside the information processing device 400. The distance values ​​of candidate points can be obtained from output data of a distance sensor (such as a distance camera) installed externally, separate from the stereo cameras, and input to the information processing device. When the distance values ​​of candidate points are obtained from output data of a sensor installed inside the information processing device, the candidate point setting unit 410 can be provided inside the information processing device 400. The image acquisition unit 401 acquires two images captured by the stereo camera 100. The interest point setting unit 402 sets a point that exists within the detection space 110 (within a three-dimensional space) from among the candidate points as an interest point. The surrounding point setting unit 403 sets a plurality of points around the interest point. In the area set around the interest point, the surrounding points are set so that the density of the surrounding points is higher than the density of the candidate points. The surrounding point distance acquisition unit 404 calculates the distance values ​​of the surrounding points by performing stereo matching based on the two images captured by the stereo camera 100. The determination unit 405 determines whether the interest point is noise based on the distance values ​​between the interest point and the surrounding points, and determines whether an object exists in the detection space 110 (within a three-dimensional space) based on the determination result.

[0016] Next, a specific processing procedure of this embodiment will be described. Fig. 4 is a flowchart showing the processing procedure. Hereinafter, it is assumed that the flowchart is realized by a CPU executing a control program. The processing described in Fig. 4 starts when the moving object starts moving.

[0017] (Step S500) The information processing unit 400 performs initialization for image acquisition and calculation. Specifically, it performs processes such as starting up the program and stereo camera, and acquiring and loading parameters required for processing from a storage unit (not shown) included in the information processing unit 400. Here, the parameters include camera parameters of the stereo camera. The camera parameters are required when performing stereo matching in the candidate point distance estimation unit 420 and the surrounding point distance acquisition unit 404, which will be described later.

[0018] (Step S510) The image acquisition unit 401 acquires two images captured by the stereo camera 100 .

[0019] (Step S520) The candidate point setting unit 410 sets candidate points on the image. In this embodiment, M points are set in a grid pattern at predetermined intervals on the image as shown in 301 of FIG. 2. The set candidate points are designated as Ai (i=1 to M), where M is a first number smaller than the number of pixels. This is synonymous with setting candidate points so that the density of the candidate points is low, that is, a density of m.

[0020] (Step S530) The candidate point distance estimation unit 420 calculates the distance value of each candidate point Ai by performing stereo matching based on the two images captured by the stereo camera 100. In this embodiment, stereo matching is a process of performing block matching on the epipolar line based on the camera parameters of the stereo camera, and calculates the distance value by performing triangulation based on the positions of corresponding pixels. The three-dimensional position of each candidate point is also determined. Here, the distance value calculated for each candidate point is defined as D(Ai).

[0021] (Step S540) The attention point setting unit 403 sets a point that exists in the detection space 110 (within a three-dimensional space) from among the candidate points as an attention point. The set attention point is denoted as As, and the distance value of the attention point is denoted as D(As). 302 in FIG. 2 shows an example of the attention point As. In this embodiment, the detection space 110 is a rectangular parallelepiped of a predetermined size set in front of the stereo cameras. The front of the stereo cameras refers to the traveling direction of the moving body on which the stereo cameras are placed. When the moving body is backing up, the traveling direction of the moving body is behind the moving body.

[0022] (Step S550) The surrounding point setting unit 403 sets multiple points 303 around the attention point As. In this embodiment, N points are randomly selected from pixels present within a distance range of radius R centered on the attention point As in the image acquired in step S510 and set as surrounding points. The set surrounding points are designated as Bj (j = 1 to N). (Setting points within a circular distance range of radius R as surrounding points is one example, and points in a partial area of ​​the image are set as surrounding points.) The density of points within the range of radius R centered on the attention point As is set higher than m. The set N points is a second number that is greater than the number of candidate points present within the range of radius R centered on the attention point As.

[0023] (Step S560) The surrounding point distance acquisition unit 404 calculates the distance value of each surrounding point Bj by performing stereo matching based on the two images captured by the stereo camera 100. Here, the calculated distance value for the surrounding point is defined as D(Bj).

[0024] (Step S570) The determination unit 405 determines whether the point of interest is noise based on the distance between the point of interest and the surrounding points, and determines whether an object exists in the detection space 110 based on the determination result.

[0025] First, for the distance value D(Bj) of each surrounding point, the ratio p of the number of points similar to the distance value D(As) of the point of interest is calculated. Specifically, the ratio of surrounding points having distance values ​​within a certain range from D(As) to the set number of surrounding points N is calculated. If p is equal to or less than a threshold T, the point of interest is determined to be noise. If p is greater than the threshold T, it is determined to be not noise but a pixel point indicating the distance value of an object present in the detection space 110. In this case, an object is detected. When an object is detected, measurement of the three-dimensional shape of the entire object is started in order to allow the moving body to continue traveling while avoiding the object. Then, the moving body continues traveling along a path that avoids the object. Alternatively, when an object is detected, the moving body is stopped.

[0026] (Step S580) The above operations of S540 to S570 are executed for all candidate points by changing the point of interest. This determines whether each candidate point set on the image is a point on an object included in the detection space 110. If it is simply desired to determine the "presence or absence" of an obstacle, once an obstacle is found for one candidate point in S570, there is no need to repeat the process for other candidate points.

[0027] As described above, stereo matching is performed using a limited number of candidate points and surrounding points set around them, rather than targeting all pixels in the image. In the image area around the candidate points, the density of surrounding points is higher than that of the candidate points. This makes it possible to detect the presence or absence of an object with high accuracy while reducing computational costs compared to methods that generate dense range images.

[0028] In S520, the candidate point setting unit 410 set candidate points at equal intervals on the image. However, the calculation cost can be reduced if the candidate points can be set spatially sparsely on the image by a method that allows one or more points to be set spatially sparsely on the image. The candidate points may be set at equal intervals on the image, or at random positions. Furthermore, when capturing moving images, the positions of the set candidate points may be changed according to time. In this case, by shifting the positions on the image to fill in the gaps between candidate points set at a certain time and setting candidate points at a later time, it is possible to prevent missed detections due to the gaps between the candidate points. Furthermore, the distance values ​​of the candidate points may be calculated based on the output data of a distance sensor (such as a distance camera) installed separately from the stereo camera.

[0029] In S550, the surrounding point setting unit 403 set pixels at random positions around the point of interest. However, as long as one or more surrounding points can be set around the point of interest, noise at the point of interest can be determined. The surrounding points may be set at random positions or at equally spaced positions. When calculating the distribution of surrounding points in the object determination unit 405, it is desirable that the distribution of the surrounding points to be set is not biased. "Biased" means, for example, using only the right half of the area around the point of interest. If the distribution of the surrounding points to be set is biased, the statistics will depend on that biased portion, which may make accurate noise determination difficult. Therefore, to avoid bias, it is desirable to set the surrounding points so that they are spaced apart by a predetermined distance or more with a probability equal to or greater than a predetermined value.

[0030] Furthermore, the surrounding point setting unit 403 calculates the ratio p of surrounding points having distance values ​​similar to the distance value of the point of interest, and determines whether or not there is noise. Any method for determining noise can be used as long as it can evaluate the number of surrounding points having distance values ​​similar to that of the point of interest. The aforementioned ratio p may be used, or the number of surrounding points having distance values ​​similar to that of the point of interest may be used.

[0031] [Variation 1] In step S550, the number of points set in surrounding point setting unit 403 is a fixed number, N. In general, the error e expected in ratio p increases or decreases depending on the number N of surrounding points used in the calculation. Specifically, the error e decreases as N increases, and the error e increases as N decreases.

[0032] It is desirable that the error e can be calculated to be equal to or less than a predetermined value in order to accurately perform the determination in the determination unit 405. On the other hand, the more surrounding points are set in order to reduce the error e, the longer it takes to calculate the distance value in the surrounding point distance acquisition unit 404.

[0033] A method for reducing the number of surrounding points to be set in the surrounding point setting unit 403 while ensuring that the error e is equal to or less than a predetermined value will be described.

[0034] Specifically, the determination unit 405 calculates the ratio p of the number of points similar to the distance value D(Bj) of each surrounding point to the distance value D(As) of the point of interest, and also calculates the error e of the ratio p. If the error e is equal to or less than a predetermined value, it is determined that the number of surrounding points set is sufficient, and it is determined whether an object exists based on the ratio p. On the other hand, if the error e exceeds the predetermined value, it is determined that the number of surrounding points set is insufficient, and the process returns to the processing in the surrounding point setting unit 402 to increase the number N of surrounding points. The above process is repeated until the error e falls within a predetermined value. This makes it possible to suppress an increase in calculation time while ensuring that the error e of the ratio p is equal to or less than the predetermined value.

[0035] Next, a specific processing procedure of this embodiment will be described. Fig. 5 is a flowchart showing the processing procedure. The processing of S650 related to the surrounding point setting unit 403 and S670 and S671 related to the determination unit 405 differs from that of Fig. 4. The specific processing contents will be described below.

[0036] (Step S650) A surrounding point setting unit 403 sets a plurality of points around the attention point As. In this embodiment, N points are randomly selected from pixels existing within a range of radius R centered on the attention point As and set as surrounding points. The set surrounding points are designated as Bj (j=1 to N).

[0037] If the determining unit 405 determines that the error e is greater than a predetermined value, the number N is increased by x points. In this embodiment, x=1, and the number N is increased by one point at a time.

[0038] (Step S670) The determination unit 405 determines whether an object exists in the detection space 110 by calculating whether the point of interest is noise based on the distance value between the point of interest and the surrounding points.

[0039] First, for the distance value D(Bj) of each surrounding point, the ratio p of the number of points similar to the distance value D(As) of the point of interest is calculated. Specifically, the ratio (ratio) of surrounding points with distance values ​​that exist within a certain range from D(As) to the set number of surrounding points N is calculated. In addition, the error e for the ratio p is calculated. The error for the ratio p is calculated based on formula (1).

[0040]

number

[0041] Here, k is a coefficient for adjusting the degree of error, and in this embodiment, k = 1. The method for calculating the error e for sampled data shown in equation (1) is well known, and is explained, for example, in H. Taherdoost, "Determining Sample Size; How to Calculate Survey Sample Size," Mathematics Leadership & Organizational Behavior eJournal, 2017.

[0042] (Step S671) If the error e is equal to or smaller than the threshold value U, the determination unit 405 determines the presence of an object based on the ratio p, as in the procedure of the flowchart in Fig. 4. If the error e is greater than the threshold value U, it determines that the number N of surrounding points set by the surrounding point setting unit 403 is insufficient, and the process returns to S650, where the surrounding point setting unit 403 performs processing. The surrounding point setting unit 403 then increases the number of points to N+x, so that the error e can be reduced when the ratio is calculated again.

[0043] As described above, the number of surrounding points set by the surrounding point setting unit 403 is optimized based on the distribution of distance values ​​of the surrounding points. Since the number of surrounding points is not increased too much in order to reduce errors, calculation costs can be reduced.

[0044] The object determination unit 405 in this embodiment calculates the error e based on equation (1). The error assumed in the ratio p may be calculated using other methods. For example, the error e may be calculated by referring to a table prepared in advance that lists the values ​​of the error e for the ratio p and the number of points N. The table may be created, for example, by generating distance images using stereo images under various conditions as input, placing points of interest in various locations, calculating the ratio p from the distribution of distance values ​​of surrounding points, and recording the error. In this case, the true value of the ratio p for each point of interest is calculated using all points surrounding the point of interest.

[0045] Furthermore, the surrounding point setting unit in this embodiment sets x=1 and increases the number N of surrounding points by one point at a time. However, x may be one point or a number greater than one. For example, if the error e is large, it is more efficient to add multiple points at once rather than adding surrounding points one by one. Therefore, x may be determined so that the larger e becomes, the larger x becomes.

[0046] Furthermore, when the number of points is increased by the surrounding point setting unit 403, it is also desirable that there is no bias in the distribution of the surrounding point positions. Therefore, the number of points may be increased preferentially to positions with low density in the distribution of surrounding points that have already been set.

[0047] [Variation 2] In step S580, the presence or absence of an object is detected at multiple locations by repeating the process of selecting a point of interest. In contrast, in this modified example, the presence of an object is determined when a single point in the detection space is detected.

[0048] In this modification, when the object determination unit 405 determines that an object exists in the detection space 110, calculations for subsequent candidate points that are set as attention points are skipped, thereby reducing unnecessary calculation costs.

[0049] Next, a specific processing procedure of this modified example will be described. Fig. 6 is a flowchart showing the processing procedure. The processing in S772 related to the object determination unit 405 differs from that in Figs. 4 and 5. The specific processing content will be described below.

[0050] (Step S772) If the object determination unit 405 determines that no object exists, it returns to the processing of the attention point setting unit 402 and sets the subsequent attention points. If it determines that an object exists, it skips the determination of the subsequent candidate points and terminates the processing.

[0051] As described above, if it is determined that an object exists, the determination of subsequent candidate points is skipped, thereby reducing the calculation cost.

[0052] [Variation 3] In this modification, the attention point setting unit 402 sets attention points in ascending order of distance values ​​of candidate points, thereby enabling objects closer to the stereo camera to be preferentially detected while reducing calculation costs.

[0053] Next, a specific processing procedure will be described. Fig. 6 is a flowchart showing the processing procedure. The processing in S740 related to the attention point setting unit 402 differs from the processing described in the above flowchart. The specific processing content will be described below.

[0054] (Step S740) The attention point setting unit 402 sets points that exist in the detection space 110 among the candidate points as attention points. At this time, the candidate points are sorted in advance based on their distance values, and are set in order of the smallest distance value from the stereo camera. Sorting based on distance values ​​does not mean that attention points do not need to be set in exact order of distance value. By setting attention points with shorter distance values ​​with priority, it is possible to reduce calculation costs and enable objects that are close to the stereo camera to be detected with priority.

[0055] As a result, the determination unit 405 determines whether the point of interest is a point on an object in ascending order of distance value, and if it determines that an object exists, it skips the determination of subsequent candidate points and terminates the process.

[0056] As described above, by setting the points of interest in ascending order of distance from the stereo camera, it is possible to ensure that objects at close range are detected preferentially, while skipping the processing of other candidate points, thereby reducing computational costs.

[0057] In this embodiment, the attention point setting unit 402 sets the attention point according to the distance value. Alternatively, for example, by focusing on an object present in the center of the image and setting the attention point based on the distance from the center of the image to each candidate point, if it is determined that the object is present in the center of the image, subsequent determinations can be skipped.

[0058] [Variation 4] The image acquisition unit can apply a method capable of acquiring two images captured from different viewpoints. For example, as shown in the first embodiment, there is a method of acquiring images captured using a stereo camera, or a method of acquiring images captured from two viewpoints by moving one camera. Furthermore, as an example of an imaging device having two optical systems and two optical paths, there is an imaging device such as the imaging device 800 shown in FIG. 8 that uses one imaging element (801, such as a CMOS sensor or a CCD sensor) to form an image from light from multiple imaging optical systems (802a, 802b). The imaging device 800 records light entering from each of the two lenses through a single CMOS sensor. The image acquisition unit can also acquire captured images using this imaging device.

[0059] The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the programs. The programs may also be provided by recording them on a computer-readable recording medium.

Claims

1. an image acquisition unit that acquires a first image from the first optical system and a second image from the second optical system, the first image being obtained from an imaging unit including a first optical system and a second optical system that are arranged in the device so that at least a portion of an imaging field of view overlaps; an estimation means for performing stereo matching between the first image and the second image using a first number of feature points, the first number being smaller than the number of pixels in the first image, and estimating three-dimensional positions of the feature points relative to the imaging means; a focus point setting means for setting, as a focus point, a feature point that is determined to have been obtained from a three-dimensional space set within a field of view of the imaging means based on the three-dimensional position of the feature point among the first number of feature points; a surrounding point setting means for setting a second number of surrounding points, the second number being greater than the number of feature points whose three-dimensional positions have been estimated by the estimation means, within an image region within a predetermined distance range from the point of interest in the first image; and a determination means for determining whether the target point is a feature point that indicates a feature of an object existing in the three-dimensional space, based on a difference between the three-dimensional positions of the surrounding points relative to the imaging means, obtained by stereo matching the surrounding points using the first image and the second image, and the three-dimensional position of the target point relative to the imaging means.

2. The determination means determines whether the point of interest is a feature point that indicates a feature of an object existing in the three-dimensional space based on a similar number or ratio of distance values ​​from the imaging means to the position indicated by the point of interest and distance values ​​from the imaging means to the positions indicated by the surrounding points.

2. The information processing apparatus according to claim 1, wherein:

3. The information processing device described in claim 1 or 2, characterized in that if the error in the number or ratio of similarities between the distance value from the imaging means to the position indicated by the target point and the distance value from the imaging means to the position indicated by the surrounding points exceeds a predetermined value, the surrounding point setting means increases the number of surrounding points.

4. The information processing device according to any one of claims 1 to 3, characterized in that the surrounding point setting means sets surrounding points in the first image so that the spacing between surrounding points is equal to or greater than a predetermined spacing, with the spacing being centered around the target point, with a probability exceeding a predetermined value.

5. 5. The information processing apparatus according to claim 1, wherein the attention point setting means sets the attention point by giving priority to a feature point that is short in distance from the imaging means to the position indicated by the feature point.

6. 5. The information processing device according to claim 1, wherein the focus point setting means sets the focus point by prioritizing a feature point having a short two-dimensional distance from the center of the first image, omitting depth information.

7. an estimation step of estimating three-dimensional positions of the feature points, including distances in real space from predetermined positions of the imaging means to positions indicated by the feature points, using results of stereo matching between a first image and a second image obtained from imaging means including a first optical system and a second optical system arranged in the apparatus so that at least a portion of the imaging fields of view overlap, using a first number of feature points that is smaller than the number of pixels in the first image; a focus point setting step of setting, as a focus point, a feature point determined to have been obtained from a three-dimensional space set within a field of view of the imaging means based on the three-dimensional position of the feature point among the first number of feature points; a surrounding point setting step of setting a second number of surrounding points in the image region, the second number being greater than the number of feature points estimated in the estimation step, the surrounding points being present in an image region within a predetermined distance range from the point of interest in the first image; and a determination step of determining whether the attention point is a feature point that indicates a feature of an object existing in the three-dimensional space based on the difference between the distance from a predetermined position of the imaging means to a position indicated by the surrounding points, obtained by stereo matching of the surrounding points using the first image and the second image, and the distance from the predetermined position of the imaging means to a position indicated by the attention point.

8. A program that causes a computer to function as each of the means of the information processing device according to any one of claims 1 to 6.

Images