Reliability estimation device, position estimation device, and reliability estimation method

Abstract

A reliability estimation device (20) estimates the reliability of an estimated position of an object estimated based on a camera image, and the reliability estimation device (20) comprises: a feature quantity calculation unit (21) that calculates a feature quantity of the object based on feature point information related to a plurality of feature points of the object in the camera image; and a reliability calculation unit (22) that calculates reliability based on the feature quantity calculated by the feature quantity calculation unit (21).

Description

Technical Field

[0001] The present invention relates to a reliability estimation device. Background Art

[0002] A 3D position estimation algorithm has been developed that estimates the 3D position of an object based on an image (in addition, in this specification, the 3D position of an object is defined as synonymous with the 3D posture). For example, Non-Patent Document 1 describes a 3D position estimation method that applies a neural network model to a camera image obtained by photographing an object with a monocular camera, thereby recognizing an object in the camera image, estimating the position of the feature point of the object, and estimating the 3D position of the object based on the estimated position of the feature point. In addition, the position of an object estimated by such a 3D position estimation method (for example, the position of the feature point of the object or the 3D position of the object, etc.) is referred to as an estimated position.

[0003] Non-patent literature 1: S. Peng, Y. Liu, Q. Huang, H. Bao, X. Zhou, “PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation,” In Computer Vision and Pattern Recognition (CVPR), 2019. Summary of the invention

[0004] In the above-mentioned conventional 3D position estimation method that applies a neural network model to an image, the reliability of the estimated position is sometimes calculated using the same neural network model as the machine learning model used when estimating the estimated position, and the calculated reliability is used to evaluate the estimated position.

[0005] In this case, it is considered that there is a correlation between the reliability of the estimated position and the reliability of the reliability itself. Therefore, there is a problem that, when the reliability of the estimated position is insufficient, the reliability of the reliability itself also becomes insufficient, and the appropriateness of the evaluation of the estimated position using the reliability becomes uncertain.

[0006] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a technology for improving the reliability of the estimated position itself.

[0007] The reliability estimation device involved in the present invention estimates the reliability of the estimated position of an object estimated based on a camera image, and is characterized in that it has: a feature quantity calculation unit, which calculates the feature quantity of the object based on feature point information related to multiple feature points of the object in the camera image; and a reliability calculation unit, which calculates the reliability based on the feature quantity calculated by the feature quantity calculation unit, wherein the feature point information includes the estimated positions of the multiple feature points and the area occupied by the object in the camera image, namely, the object area, and the feature quantity calculation unit calculates the feature quantity that is a function of the area composed of the multiple feature points and the object area as input based on the estimated positions of the multiple feature points and the object area.

[0008] Effects of the Invention

[0009] According to the present invention, the reliability of the estimated position itself can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a block diagram showing the structure of the position estimation system involved in the first embodiment.

[0011] Figure 2 This is a flowchart showing a reliability estimation method implemented by the position estimation device according to the first embodiment.

[0012] Figure 3 This is a flowchart showing the details of a feature quantity calculation method in the reliability estimation method implemented by the position estimation device according to the first embodiment.

[0013] Figure 4A This is a block diagram showing a hardware configuration for realizing the functions of the position estimating device according to the first embodiment. Figure 4B This is a block diagram showing the hardware configuration for executing software that realizes the functions of the position estimating device according to the first embodiment. DETAILED DESCRIPTION

[0014] Hereinafter, in order to explain the present invention in more detail, embodiments for implementing the present invention will be described with reference to the accompanying drawings.

[0015] Implementation Method 1

[0016] Figure 1 is a block diagram showing the structure of the position estimation system 100 according to Embodiment 1. Figure 1 As shown, the position estimation system 100 includes a position estimation device 1, N cameras 2 (N is a positive integer), and a storage device 3. The position estimation device 1 includes a camera selection unit 40, a feature point estimation unit 10, a reliability estimation device 20, and a position calculation unit 30. The reliability estimation device 20 includes a feature quantity calculation unit 21 and a reliability calculation unit 22.

[0017] The N cameras 2 are composed of the first camera 2 to the Nth camera 2. Each of the N cameras 2 acquires a camera image (camera images D11 to D1N) by photographing an object. Each of the N cameras 2 outputs the acquired camera image to the camera selection unit 40 of the position estimation device 1.

[0018] The camera selection unit 40 selects at least one camera image D2 from the plurality of camera images. More specifically, in the first embodiment, the camera selection unit 40 selects at least one camera image D2 from the plurality of camera images D11 to D1N acquired by the N cameras 2. The detailed structure of the camera selection unit 40 will be described later. The camera selection unit 40 outputs the selected camera image D2 to the feature point estimation unit 10 and the feature amount calculation unit 21, respectively.

[0019] The feature point estimation unit 10 estimates feature point information about a plurality of feature points of an object based on the camera image. More specifically, in the first embodiment, the feature point estimation unit 10 estimates feature point information based on the camera image D2 selected by the camera selection unit 40 .

[0020] In further detail, in the first embodiment, the storage device 3 stores a convolutional neural network model for object position detection, namely, PVNet. The feature point estimation unit 10 applies the PVNet stored in the storage device 3 to the camera image D2 selected by the camera selection unit 40, thereby estimating each estimated position D3 of a plurality of feature points, the variance D4 of the candidate point group of the feature points, and the object area D5, which is the area occupied by the object in the camera image, as feature point information.

[0021] The feature point estimating unit 10 outputs the estimated positions D3 of the plurality of feature points to the position calculating unit 30 . The feature point estimating unit 10 also outputs the estimated positions D3 of the plurality of feature points, the variance D4 of the candidate point cluster of the feature points, and the object region D5 to the feature quantity calculating unit 21 .

[0022] The feature quantity calculation unit 21 calculates the feature quantity of the object based on the feature point information about the plurality of feature points of the object in the camera image. More specifically, in the first embodiment, the feature quantity calculation unit 21 calculates the feature quantity of the object based on the feature point information estimated by the feature point estimation unit 10 .

[0023] More specifically, in Embodiment 1, the feature quantity calculation unit 21 includes m feature quantity calculation units from the first feature quantity calculation unit 201 to the m-th feature quantity calculation unit 20m.

[0024] The first feature quantity calculation unit 201 calculates the feature quantity D61 of the object based on the variance D4 of the candidate point group of the feature point estimated by the feature point estimation unit 10. In addition, in the first embodiment, the feature quantity D61 calculated by the first feature quantity calculation unit 201 is a feature quantity that is proportional to the inverse of the variance D4 of the candidate point group of the feature point. In other words, the feature quantity D61 calculated by the first feature quantity calculation unit 201 is a feature quantity that is inversely proportional to the variance D4 of the candidate point group of the feature point. The first feature quantity calculation unit 201 outputs the calculated feature quantity D61 to the reliability calculation unit 22.

[0025] The second feature quantity calculation unit 202 calculates a feature quantity D62 that is a function that takes the region formed by the plurality of feature points and the object region as inputs, based on the estimated positions D3 of the plurality of feature points estimated by the feature point estimation unit 10 and the object region D5. More specifically, in the first embodiment, the second feature quantity calculation unit 202 calculates the area ratio of the convex plane formed by the plurality of feature points to the object region based on the estimated positions D3 of the plurality of feature points estimated by the feature point estimation unit 10 and the object region D5, and calculates a feature quantity D62 that is proportional to the inverse of the calculated area ratio or the inverse of the difference between the calculated area ratio and 1. Here, the "convex plane formed by the plurality of feature points" is, for example, the largest convex plane formed by the plurality of feature points or the smallest convex plane that includes all the plurality of feature points. The second feature quantity calculation unit 202 outputs the calculated feature quantity D62 to the reliability calculation unit 22.

[0026] Each feature quantity calculation unit from the third feature quantity calculation unit to the m-th feature quantity calculation unit 20m (not shown) calculates feature quantities (feature quantities D63 to D6m) that can be calculated based on at least one information among the camera image D2 selected by the camera selection unit 40 and each estimated position D3 of a plurality of feature points estimated by the feature point estimation unit 10, the variance D4 of the candidate point group of feature points, and the object area D5. Each feature quantity calculation unit from the third feature quantity calculation unit to the m-th feature quantity calculation unit 20m (not shown) outputs the calculated feature quantities to the reliability calculation unit 22.

[0027] The reliability calculation unit 22 calculates the reliability of the estimated position of the object based on the feature amount calculated by the feature amount calculation unit 21. More specifically, in the first embodiment, the reliability calculation unit 22 calculates the reliability D7 (for example, the reliability D7 having a positive correlation with the estimation error) related to the estimation error of the estimated position of the object based on the plurality of feature amounts (feature amounts D61 to D6m) calculated by the feature amount calculation unit 21.

[0028] In more detail, in the first embodiment, the storage device 3 stores a neural network model for calculating reliability based on feature quantities. The reliability calculation unit 22 calculates the reliability D7 which is inversely proportional to the estimation error of the estimated position of the object by applying the neural network model stored in the storage device 3 to the plurality of feature quantities (feature quantities D61 to D6m) calculated by the feature quantity calculation unit 21. The reliability calculation unit 22 outputs the calculated reliability D7 to the camera selection unit 40 and the outside of the position estimation device 1.

[0029] The camera selection unit 40 selects at least one camera image D2 from the N camera images D11 to D1N based on the reliability D7 calculated by the reliability calculation unit 22. The camera selection unit 40 outputs the camera image D2 selected based on the reliability D7 to the feature point estimation unit 10 and the feature amount calculation unit 21 respectively.

[0030] The position calculation unit 30 calculates the estimated position D8 of the object based on the feature point information estimated by the feature point estimation unit 10. More specifically, in the first embodiment, the storage device 3 stores the feature point positions of the object. The position calculation unit 30 solves the PnP problem based on the estimated positions D3 of the plurality of feature points estimated by the feature point estimation unit 10 and the feature point positions of the object stored in the storage device 3, thereby calculating the estimated position D8 of the object. As an example of the estimated position D8 calculated by the position calculation unit 30, a 3D position (3D posture) of the object is cited. The position calculation unit 30 outputs the calculated estimated position D8 to the outside of the position estimation device 1.

[0031] Furthermore, reliability D7 outputted from reliability calculation unit 22 to the outside of position estimation device 1 is used, for example, for evaluation of estimated position D8 outputted from position calculation unit 30 to the outside of position estimation device 1 .

[0032] Hereinafter, the operation of the position estimating device 1 according to the first embodiment will be described with reference to the drawings. Figure 2 1 is a flowchart showing a reliability estimation method implemented by the position estimation device 1 according to Embodiment 1. It is assumed that, before each step described below, the N cameras 2 each capture an object to obtain a camera image and output the obtained camera image to the camera selection unit 40 .

[0033] like Figure 2 As shown, the camera selection unit 40 selects at least one camera image D2 from among the plurality of camera images (step ST1 ). The camera selection unit 40 outputs the selected camera image D2 to each of the feature point estimation unit 10 and the feature amount calculation unit 21 .

[0034] Furthermore, in step ST1, the camera selection unit 40 selects an arbitrary camera image D2 from the plurality of camera images when the reliability D7 has not yet been calculated in the process of step ST4 described later. Alternatively, in step ST1, the camera selection unit 40 selects at least one camera image D2 from the N camera images D11 to D1N based on the reliability D7 when the reliability D7 has already been calculated in the process of step ST4 described later.

[0035] Next, the feature point estimation unit 10 estimates each estimated position D3 of the plurality of feature points, the variance D4 of the candidate point group of the feature points, and the object area D5 as feature point information based on the camera image D2 selected by the camera selection unit 40 (step ST2). The feature point estimation unit 10 outputs the estimated each estimated position D3 of the plurality of feature points, the variance D4 of the candidate point group of the feature points, and the object area D5 to the feature quantity calculation unit 21.

[0036] Next, the feature quantity calculation unit 21 calculates feature quantities D61 to D6m of the object based on the camera image D2 selected by the camera selection unit 40, and at least one of the estimated positions D3 of the plurality of feature points estimated by the feature point estimation unit 10, the variance D4 of the candidate point group of the feature points, and the object area D5 (step ST3). The feature quantity calculation unit 21 outputs the calculated feature quantities D61 to D6m to the reliability calculation unit 22.

[0037] Next, the reliability calculation unit 22 calculates the reliability D7 related to the estimated error of the estimated position of the object based on the feature quantities D61 to D6m calculated by the feature quantity calculation unit 21 (step ST4). Although not shown, the position estimation device 1 repeatedly performs the series of steps from step ST1 to step ST4 described above.

[0038] Hereinafter, step ST3 will be described in detail. Figure 3 This is a flowchart showing the details of the feature amount calculation method in step ST3.

[0039] like Figure 3 As shown, the first feature quantity calculation unit 201 calculates the feature quantity D61 of the object proportional to the inverse of the variance D4 based on the variance D4 of the candidate point group of the feature points estimated by the feature point estimation unit 10 (step ST31). The first feature quantity calculation unit 201 outputs the calculated feature quantity D61 to the reliability calculation unit 22.

[0040] The second feature quantity calculation unit 202 calculates the area ratio of the convex plane formed by the plurality of feature points and the object area based on the estimated positions D3 of the plurality of feature points estimated by the feature point estimation unit 10 and the object area D5, and calculates a feature quantity D62 proportional to the inverse of the calculated area ratio or the inverse of the difference between the calculated area ratio and 1 (step ST32). The second feature quantity calculation unit 202 outputs the calculated feature quantity D62 to the reliability calculation unit 22.

[0041] Each feature quantity calculation unit from the third feature quantity calculation unit to the m-th feature quantity calculation unit 20m calculates other feature quantities D63 to D6m based on the camera image D2 selected by the camera selection unit 40, and each estimated position D3 of a plurality of feature points estimated by the feature point estimation unit 10, the variance D4 of the candidate point group of the feature points, and at least one of the object area D5 (step ST33). Each feature quantity calculation unit from the third feature quantity calculation unit to the m-th feature quantity calculation unit 20m outputs the calculated feature quantities to the reliability calculation unit 22.

[0042] Hereinafter, a specific example of the camera selection method (step ST1) in the reliability estimation method implemented by the position estimation device 1 according to Embodiment 1 will be described in detail. In addition, it is assumed that the process of step ST1 described below is executed again after a series of processes from step ST1 to step ST4 are executed more than or equal to one time.

[0043] In this specific example, in step ST1, the camera selection unit 40 selects at least one camera image D2 from the N camera images D11 to D1N based on the reliability D7 calculated by the reliability calculation unit 22 in the above-mentioned step ST4. More specifically, in this specific example, the camera selection unit 40 selects any one of a mode of performing subsequent position estimation using one camera image or a mode of performing position estimation using all camera images in sequence based on the N camera images D11 to D1N and the reliability D7 calculated by the reliability calculation unit 22.

[0044] In further detail, in this specific example, in step ST1, the camera selection unit 40 selects the camera image D2 from the camera 2 corresponding to the camera image selected last time when the reliability D7 calculated by the reliability calculation unit 22 is greater than or equal to the threshold. In other words, the camera selection unit 40 selects the camera image D2 from the camera 2 that captured the camera image selected last time when the reliability D7 calculated by the reliability calculation unit 22 is greater than or equal to the threshold. In this case, in the above-mentioned step ST2, the feature point estimation unit 10 estimates the feature point information based on the camera image D2 selected by the camera selection unit 40 as described above.

[0045] On the other hand, in step ST1, the camera selection unit 40 selects all of the N camera images D11 to D1N when the reliability calculated by the reliability calculation unit 22 is less than the threshold value. Next, in the above-mentioned step ST2, the feature point estimation unit 10 estimates feature point information for each camera image when the camera selection unit 40 selects all of the N camera images D11 to D1N in step ST1.

[0046] Next, in the above-mentioned step ST3 , the feature amount calculation unit 21 calculates the feature amount of the object for each camera image based on the feature point information estimated by the feature point estimation unit 10 for each camera image in step ST2 .

[0047] Next, in the above-mentioned step ST4 , the reliability calculation unit 22 calculates the reliability of the estimated position of the object for each camera image based on the feature amount calculated by the feature amount calculation unit 21 for each camera image.

[0048] Next, the position estimation device 1 returns to the process of step ST1 described above, and the camera selection unit 40 selects at least one camera image D2 from the N camera images D11 to D1N based on the maximum reliability among the reliabilities of each camera image calculated by the reliability calculation unit 22, when the reliability calculation unit 22 calculates the reliability for each camera image. More specifically, for example, the camera selection unit 40 selects the camera image D2 captured by the camera 2 corresponding to the maximum reliability among the reliabilities of each camera image calculated by the reliability calculation unit 22, when the reliability calculation unit 22 calculates the reliability for each camera image.

[0049] The functions of the camera selection unit 40, the feature point estimation unit 10, the feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1 are realized by the processing circuit. Figure 2 and Figure 3 The processing circuit for the processing of each step shown may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.

[0050] Figure 4A It is a block diagram showing a hardware configuration for realizing the functions of the position estimating device 1 . Figure 4B 1 is a block diagram showing a hardware configuration for executing software that realizes the functions of the position estimating device 1 .

[0051] In the above processing circuit is Figure 4A In the case of the dedicated hardware processing circuit 50 shown, the processing circuit 50 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0052] The functions of the camera selection unit 40, the feature point estimation unit 10, the feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation unit 20, and the position calculation unit 30 in the position estimation device 1 can be implemented by independent processing circuits, or these functions can be combined and implemented by one processing circuit.

[0053] In the above processing circuit is Figure 4B In the case of the processor 51 shown, the functions of the camera selection unit 40, the feature point estimation unit 10, the feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1 are implemented by software, firmware, or a combination of software and firmware.

[0054] In addition, the software or firmware is described as a program and stored in the memory 52 .

[0055] The processor 51 reads out and executes the program stored in the memory 52, thereby realizing the functions of the camera selection unit 40, the feature point estimation unit 10, the feature amount calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1. That is, the position estimation device 1 has a memory 52 for storing the following program, which results in the execution of the above-mentioned functions by the processor 51. Figure 2 and Figure 3 The processing of each step shown is performed.

[0056] These programs cause the computer to execute the processes or methods of the camera selection unit 40, the feature point estimation unit 10, the feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation unit 20, and the position calculation unit 30 in the position estimation device 1. The memory 52 may be a computer-readable storage medium storing a program for causing the computer to function as the camera selection unit 40, the feature point estimation unit 10, the feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation unit 20, and the position calculation unit 30 in the position estimation device 1.

[0057] The processor 51 is, for example, a CPU (Central Processing Unit), a processing device, a computing device, a processor, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor).

[0058] The memory 52 is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), a magnetic disk such as a hard disk or a floppy disk, an optical disk, a compressed disk, a mini disk, a CD (CompactDisc), a DVD (Digital Versatile Disc), etc.

[0059] Regarding the functions of the camera selection unit 40, the feature point estimation unit 10, the feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20, and the position calculation unit 30 in the position estimation device 1, part of them can be implemented by dedicated hardware, and part of them can be implemented by software or firmware.

[0060] For example, the functions of the camera selection unit 40, the feature point estimation unit 10, and the position calculation unit 30 are realized by a processing circuit as dedicated hardware. The feature quantity calculation unit 21 and the reliability calculation unit 22 of the reliability estimation device 20 may also be realized by the processor 51 reading and executing a program stored in the memory 52.

[0061] As described above, the processing circuit can implement the above functions through hardware, software, firmware or a combination thereof.

[0062] As described above, the reliability estimation device 20 involved in embodiment 1 estimates the reliability of the estimated position of the object estimated based on the camera image, and the reliability estimation device 20 has: a feature quantity calculation unit 21, which calculates the feature quantity of the object based on feature point information related to multiple feature points of the object in the camera image; and a reliability calculation unit 22, which calculates the reliability of the estimated position of the object based on the feature quantity calculated by the feature quantity calculation unit 21.

[0063] According to the above structure, the reliability of the estimated position of the object is calculated based on the feature amount of the object calculated based on the feature point information. Therefore, compared with the case where the reliability is directly calculated based on the estimated position, the reliability of the estimated position itself does not depend on the reliability of the estimated position, so the reliability of the estimated position itself can be improved. Therefore, for example, even when an image is input that is difficult to estimate the estimated position, the reliability of the estimated position itself can be improved.

[0064] The feature point information in the reliability estimation device 20 according to the first embodiment includes the variance of the candidate point group of the feature points, and the feature quantity calculation unit 21 calculates the feature quantity of the object based on the variance of the candidate point group of the feature points.

[0065] According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature amount of the object calculated from the variance of the candidate point group of feature points. This makes it possible to appropriately improve the reliability of the estimated position itself.

[0066] The feature quantity calculation unit 21 in the reliability estimation device 20 according to the first embodiment calculates a feature quantity proportional to the inverse of the variance.

[0067] According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature quantity proportional to the inverse of the variance. This makes it possible to appropriately improve the reliability of the estimated position itself.

[0068] The multiple feature point information in the reliability estimation device 20 involved in implementation mode 1 includes the estimated positions of the multiple feature points and the area occupied by the object in the camera image, namely, the object area. The feature quantity calculation unit 21 calculates the feature quantity that is a function of the area composed of the multiple feature points and the object area as input based on the estimated positions of the multiple feature points and the object area.

[0069] According to the above configuration, the reliability of the estimated position of the object is calculated based on the feature quantity which is a function of the region composed of a plurality of feature points and the object region as inputs. This makes it possible to appropriately improve the reliability of the estimated position itself.

[0070] The feature point information in the reliability estimation device 20 involved in implementation mode 1 includes the estimated positions of multiple feature points and the area occupied by the object in the camera image, that is, the object area. The feature quantity calculation unit 21 calculates the area ratio of the convex plane composed of multiple feature points and the object area based on the estimated positions of the multiple feature points and the object area, and calculates the feature quantity that is proportional to the inverse of the calculated area ratio, or the feature quantity that is proportional to the inverse of the difference between the calculated area ratio and 1.

[0071] According to the above configuration, the reliability of the estimated position of the object is calculated based on the estimated positions of the plurality of feature points and a feature quantity proportional to the inverse of the area ratio calculated from the object region or a feature quantity proportional to the inverse of the difference between the area ratio and 1. Thus, the reliability of the estimated position itself can be appropriately improved.

[0072] The feature amount calculation unit 21 in the reliability estimation device 20 according to the first embodiment calculates a plurality of feature amounts of the object, and the reliability calculation unit 22 calculates the reliability associated with the estimation error of the estimated position of the object based on the plurality of feature amounts calculated by the feature amount calculation unit 21 .

[0073] According to the above configuration, the estimated position of the object can be evaluated using the reliability associated with the estimation error of the estimated position of the object.

[0074] The position estimation device 1 involved in embodiment 1 has the reliability estimation device 20 involved in embodiment 1, the feature point estimation unit 10 that estimates the feature point information based on the camera image, and the position calculation unit 30 that calculates the estimated position of the object based on the feature point information estimated by the feature point estimation unit 10, and the feature quantity calculation unit 21 calculates the feature quantity of the object based on the feature point information estimated by the feature point estimation unit 10.

[0075] According to the above structure, the above effects of the reliability estimation device 20 according to the first embodiment can be realized in the position estimation device 1. For example, the position estimation device 1 according to the first embodiment can be applied to the position estimation of an object based on a camera image. For example, it can be used in FA equipment such as an autonomous driving vehicle or a robot arm.

[0076] For example, when an object is held by a robot arm, the position of the object needs to be detected. Therefore, by applying an image captured by a single-lens camera or the like mounted on the robot arm as the above-mentioned camera image to the reliability estimation device 20 involved in Embodiment 1, even when an image with poor shooting conditions and difficult position estimation is input, the above-mentioned reliability with high reliability can be calculated, and the estimated position can be evaluated based on the reliability. Thus, stable robot control can be achieved.

[0077] The position estimation device 1 according to Embodiment 1 further includes a camera selection unit 40 for selecting at least one camera image from a plurality of camera images based on the reliability calculated by the reliability calculation unit 22 , and the feature point estimation unit 10 estimates feature point information based on the camera image selected by the camera selection unit 40 .

[0078] According to the above configuration, feature point information is estimated based on the camera image selected based on the reliability of the estimated position. This can improve the reliability of the estimated feature point information. This can improve the reliability of the estimated position and reliability based on the feature point information.

[0079] The camera selection unit 40 in the position estimation device 1 involved in embodiment 1 selects the camera image from the camera 2 corresponding to the camera image selected last time when the reliability calculated by the reliability calculation unit 22 is greater than or equal to the threshold, and selects all multiple camera images when the reliability calculated by the reliability calculation unit 22 is less than the threshold.

[0080] According to the above configuration, it is possible to appropriately select an estimated camera image suitable for the estimated position, thereby improving the reliability of the feature point information, the estimated position, and the reliability based on the camera image.

[0081] In the position estimation device 1 involved in the first embodiment, the feature point estimating unit 10 estimates feature point information for each camera image when the camera selecting unit 40 selects all of the multiple camera images, the feature quantity calculating unit 21 calculates the feature quantity of the object for each camera image based on the feature point information estimated by the feature point estimating unit 10 for each camera image, the reliability calculating unit 22 calculates the reliability of the estimated position of the object for each camera image based on the feature quantity calculated by the feature quantity calculating unit 21 for each camera image, and the camera selecting unit 40 selects at least one camera image from the multiple camera images based on the maximum reliability among the reliabilities of each camera image calculated by the reliability calculating unit 22 when the reliability calculating unit 22 calculates the reliability for each camera image.

[0082] According to the above configuration, even if the camera image selected last time is not suitable for estimating the estimated position, a camera image suitable for estimating the estimated position can be selected next time, thereby improving the reliability of the feature point information, estimated position and reliability based on the camera image.

[0083] The reliability estimation method involved in embodiment 1 estimates the reliability of the estimated position of an object estimated based on a camera image, and the reliability estimation method includes: a feature quantity calculation step, calculating the feature quantity of the object based on feature point information related to multiple feature points of the object in the camera image; and a reliability calculation step, calculating the reliability based on the feature quantity calculated in the feature quantity calculation step.

[0084] According to the above-mentioned structure, the same effect as the above-mentioned effect achieved by the reliability estimation device 20 according to the first embodiment is achieved.

[0085] Furthermore, any structural elements of the embodiments may be modified or omitted.

[0086] Industrial Applicability

[0087] The reliability estimating device according to the present invention can improve the reliability of the estimated position itself, and therefore can be used in a position estimating device.

[0088] Description of the label

[0089] 1 position estimation device, 2 camera, 3 storage device, 10 feature point estimation unit, 20 reliability estimation device, 21 feature quantity calculation unit, 22 reliability calculation unit, 30 position calculation unit, 40 camera selection unit, 50 processing circuit, 51 processor, 52 memory, 100 position estimation system, 201 first feature quantity calculation unit, 202 second feature quantity calculation unit, 20m mth feature quantity calculation unit.

Claims

1. A reliability estimation device for estimating the reliability of an estimated position of an object estimated based on a camera image, The reliability estimation device is characterized by having: a feature quantity calculation unit that calculates a feature quantity of the object based on feature point information related to a plurality of feature points of the object in the camera image; and a reliability calculation unit that calculates the reliability based on the feature quantity calculated by the feature quantity calculation unit, The feature point information includes the estimated positions of the plurality of feature points and the area occupied by the object in the camera image, that is, the object area. The feature quantity calculation unit calculates a feature quantity that is a function having a region constituted by the plurality of feature points and the object region as inputs, based on the estimated positions of the plurality of feature points and the object region.

2. The reliability estimation device according to claim 1, characterized in that: The feature point information includes the variance of the candidate point group of the feature point. The feature quantity calculation unit calculates the feature quantity based on a variance of a candidate point group of the feature points.

3. The reliability estimation device according to claim 2, characterized in that: The feature quantity calculation unit calculates a feature quantity proportional to the inverse of the variance.

4. The reliability estimation device according to claim 1, characterized in that: The feature quantity calculation unit calculates a plurality of the feature quantities. The reliability calculation unit calculates reliability related to an estimation error of the estimated position of the object based on the plurality of feature quantities calculated by the feature quantity calculation unit.

5. A position estimation device, characterized in that: have: The reliability estimation device according to claim 1; a feature point estimating unit that estimates the feature point information based on the camera image; as well as a position calculation unit that calculates an estimated position of the object based on the feature point information estimated by the feature point estimation unit, The feature amount calculation unit calculates the feature amount based on the feature point information estimated by the feature point estimation unit.

6. The position estimation device according to claim 5, characterized in that: further comprising a camera selection unit that selects at least one camera image from among the plurality of camera images based on the reliability calculated by the reliability calculation unit, The feature point estimating unit estimates the feature point information based on the camera image selected by the camera selecting unit.

7. The position estimation device according to claim 6, characterized in that: the camera selection unit, When the reliability calculated by the reliability calculation unit is greater than or equal to a threshold value, selecting a camera image from a camera corresponding to the camera image selected last time, When the reliability calculated by the reliability calculation unit is smaller than a threshold value, all of the plurality of camera images are selected.

8. The position estimation device according to claim 7, characterized in that: The feature point estimating unit estimates the feature point information for each camera image when the camera selecting unit selects all of the plurality of camera images. The feature quantity calculation unit calculates the feature quantity for each camera image based on the feature point information estimated by the feature point estimation unit for each camera image. The reliability calculation unit calculates the reliability for each camera image based on the feature amount calculated by the feature amount calculation unit for each camera image. The camera selection unit selects at least one camera image from the plurality of camera images based on a maximum reliability among the reliabilities of each camera image calculated by the reliability calculation unit when the reliability calculation unit calculates the reliability for each camera image.

9. A reliability estimation method for estimating the reliability of an estimated position of an object estimated based on a camera image, The reliability estimation method is characterized by comprising: a feature quantity calculating step of calculating, by a feature quantity calculating unit, a feature quantity of the object based on feature point information related to a plurality of feature points of the object in the camera image; and a reliability calculation step of calculating the reliability by a reliability calculation unit based on the feature quantity calculated by the feature quantity calculation unit in the feature quantity calculation step, The feature point information includes the estimated positions of the plurality of feature points and the area occupied by the object in the camera image, that is, the object area. In the feature quantity calculation step, the feature quantity calculation unit calculates a feature quantity that is a function having a region composed of the plurality of feature points and the object region as inputs, based on the estimated positions of the plurality of feature points and the object region.

Images