Information processing device, information processing method, and computer program

Abstract

To provide an information processing device, an information processing method, and a computer program capable of reducing errors when estimating the position and orientation of a mobile body .SOLUTION: The processing in an information processing device includes acquiring detection information detected by a sensor mounted on a mobile body, calculating degrees of similarity of a plurality of pieces of detection information detected at different positions or different orientations, determining, on the basis of the degrees of similarity, positions of additional objects to be added within a detection range of the sensor when the detection information is detected, and notifying a user of at least positions of the additional objects.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a computer program suitable for estimating the position and orientation of a moving object.

Background Art

[0002] A method of mounting sensors such as cameras and distance sensors on a moving object and estimating the position and orientation of the moving object based on information acquired by the cameras and sensors is known. As such a method, for example, the SLAM (Simultaneous Localization and Mapping) technique is known.

[0003] The moving object is, for example, a small mobility such as an AGV (Automatic Guides Vehicle), an AMR (Autonomous Mobile Robot), a cleaning robot, a drone, or the like.

[0004] SLAM simultaneously and parallelly performs a process of generating a three-dimensional environmental map and a self-position and orientation estimation process using the environmental map. In Non-Patent Document 1, as a method for resuming self-position and orientation estimation after failure of self-position and orientation estimation, a process called relocalize is performed.

[0005] In relocalize, in order to estimate the current self-position and orientation, a map element including information (for example, an image) similar to the information acquired by the sensor at the current position and orientation is searched from the three-dimensional environmental map.

Prior Art Documents

Patent Documents

[0006]

Non-Patent Document 1

[0007] However, with the method described in Non-Patent Document 1, if multiple similar map elements exist, errors in estimating the position and orientation of the moving object may reduce the accuracy of the relocalization results.

[0008] In view of the above problems, the present invention aims to provide an information processing device that can reduce errors in the estimation of the position and orientation of a moving object. [Means for solving the problem]

[0009] To solve the above problems, the information processing apparatus according to the present invention is: A means for acquiring detection information that acquires detection information detected by a sensor mounted on a mobile body, A similarity calculation means for calculating the similarity of a plurality of detection pieces of information obtained by the detection information acquisition means, which are detected at different positions or orientations from each other; An additional object determination means determines the position of an additional object to be added to the detection range of the sensor when the detection information is detected, based on the similarity calculated by the similarity calculation means. The system is characterized by having a notification means for notifying the position of the additional object determined by the additional object determination means. [Effects of the Invention]

[0010] According to the present invention, it is possible to realize an information processing device that can reduce errors in the estimation of the position and orientation of a moving object. [Brief explanation of the drawing]

[0011] [Figure 1] This is a functional block diagram showing an example configuration of an information processing device according to Embodiment 1 of the present invention. [Figure 2] This is a hardware block diagram of the information processing device according to Example 1. [Figure 3] This is a flowchart showing the overall processing flow of the information processing device according to Example 1. [Figure 4] This figure shows an example of an object information list to be added to the environment according to Example 1. [Figure 5] This figure shows an example of a method for notifying additional objects according to Example 1. [Figure 6] This figure shows the configuration of an information processing device in another embodiment. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same reference numeral is used for the same member or element, and redundant explanations are omitted or simplified. [Examples]

[0013] Example 1 describes an example of a mobile body that acquires detection information, such as images, detected by sensors such as cameras mounted on the mobile body, and uses this detection information to perform automatic (autonomous) driving using SLAM. Note that the above-mentioned sensors may also be distance measuring means such as LIDAR that output distance information as the detection information, as will be described later.

[0014] In this embodiment, with respect to multiple detection information such as images that constitute map elements, it is determined whether the similarity of multiple detection information detected at different positions or orientations is above a predetermined threshold. If there is detection information with a similarity above the predetermined threshold, the position, pattern, or orientation of an object to be added within the detection range (field of view) of a sensor such as a camera is determined in order to reduce the similarity.

[0015] FIG. 1 is a functional block diagram showing a configuration example of an information processing apparatus according to Embodiment 1 of the present invention. Note that some of the functional blocks shown in FIG. 1 are realized by causing a computer included in the information processing apparatus to execute a computer program stored in a memory as a storage medium. However, some or all of them may be realized by hardware. As the hardware, a dedicated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used.

[0016] Moreover, each of the functional blocks shown in FIG. 1 does not have to be built in the same housing, and may be constituted by separate devices connected to each other via signal paths.

[0017] 100 is an image acquisition unit that acquires a plurality of images captured by an imaging device as imaging means mounted on a moving body that moves in a predetermined area. Here, the image acquisition unit 100 (detection information acquisition means) acquires detection information such as an image detected by an imaging device as a sensor mounted on the moving body.

[0018] Note that in the present embodiment, a key frame that associates an image stored in the storage device with the position and orientation of the imaging device when the image was captured is acquired. Note that in the present embodiment, a key frame is an image of a structure, a building, a signboard, a sign, or the like that serves as a landmark for specifying the location, and is an image acquired by an imaging device mounted on the moving body.

[0019] 101 is a similarity calculation unit that calculates the similarity between a plurality of images acquired by the image acquisition unit 100. Here, the similarity calculation unit 101 (similarity calculation means) calculates the similarity between a plurality of detection information detected at different positions or orientations and acquired by the image acquisition unit 100 as detection information acquisition means.

[0020] 102 is an additional object determination unit that determines the pattern and the position and orientation of an object to be added to the visual field corresponding to a plurality of image groups acquired by the image acquisition unit 100 based on the similarity calculated by the similarity calculation unit 101.

[0021] The additional object determination unit 102 (additional object determination means) determines which additional objects to be added to the detection range of the sensor (within the camera's field of view) when detection information is detected, based on the similarity calculated by the similarity calculation unit 101, which acts as a similarity calculation means. In this embodiment, the additional object determination unit 102 determines not only the position of the additional object, but also at least one of the orientation and pattern of the additional object.

[0022] 103 is a notification unit that notifies the user of the information determined by the additional object determination unit 102. The notification unit 103 (notification means) notifies the user of at least the location of the additional object determined by the additional object determination unit 102, which is an additional object determination means.

[0023] Figure 2 is a hardware block diagram of the information processing device according to Embodiment 1. 200 is a bus for connecting various devices, 201 is a CPU that acts as a computer to read and execute control and processing steps and programs for the various devices connected to the bus 200, and 202 is a ROM that stores the BIOS program and boot program.

[0024] 203 is RAM used as the main memory of the CPU 201, 204 is external memory that stores computer programs processed by the CPU 201, and 205 is an input unit that processes information input. 206 is a display unit that outputs the results of the information processing device to a display device according to instructions from the CPU 201, and 207 is a communication I / O that performs information communication via a network.

[0025] Figure 3 is a flowchart showing the overall processing flow of the information processing device according to Embodiment 1. The CPU 201, acting as a computer, executes a computer program stored in, for example, external memory 204, thereby performing each step in the flowchart of Figure 3. Furthermore, the flow in Figure 3 represents a process that is repeatedly performed while a moving object moves towards a target location.

[0026] In step S300, the CPU 201 initializes the information processing device. That is, it reads the computer program from the external memory 204 and makes it executable. It also reads the information necessary for processing, such as the similarity threshold used by the additional object determination unit 102 when determining whether multiple image groups are similar, and the additional object information list shown in Figure 4, into the RAM 203. The additional object information list shown in Figure 4 will be described later.

[0027] In step S301 (detection information acquisition step), the CPU 201 acquires multiple image groups (keyframes) from the image acquisition unit 100 at different positions or orientations. That is, in step S301, images are acquired as detection information output from an imaging device, which is a sensor mounted on a moving object.

[0028] Then, in step S302 (similarity calculation step), the CPU 201 uses the similarity calculation unit 101 to calculate the similarity R between the multiple images taken at different positions or orientations, which were acquired in step S301. That is, in step S302, the similarity of the multiple images, which are detection information acquired in step S301 as a detection information acquisition step, and which were detected at different positions or orientations, is calculated.

[0029] To calculate the similarity R between multiple images, we apply a well-known Bag of Words (BoW) method. Specifically, we extract feature vectors from the images and calculate the similarity of these feature vectors as the similarity of the images. The similarity of the feature vectors is calculated as the dot product of the vectors. Further details on BoW can be found in, for example, Non-Patent Document 1.

[0030] In step S303 (additional object determination step), the CPU 201, using the additional object determination unit 102, determines the pattern, position, and orientation of the object to be added to the camera's field of view (environment) corresponding to the multiple image groups acquired in step S301.

[0031] Specifically, in step S303, based on the similarity calculated in step S302, which is a similarity calculation step, the position of an object to be added to the sensor's detection range (field of view) when the image as detection information is detected (hereinafter referred to as "additional object") is determined. The method for determining the additional object will be described later.

[0032] Furthermore, the position of the additional object determined in step S303, which is the additional object determination step, is notified to the user in a notification step (not shown) that follows step S303. This allows the user to place (attach, paint, etc.) the additional object with the pattern notified in step S303 at the determined position and orientation, thereby reducing position and orientation estimation errors and errors in relocalization.

[0033] Figure 4 is a diagram showing an example of an additional object information list according to Example 1, and shows an example of an object information list used in step S303 when determining the pattern and position / orientation of the additional object.

[0034] In Figure 4, 400 is a list of subjects that may be present in the image set acquired in step S301, and 401 is a list of additional objects related to (associated with) the subject list 400.

[0035] 402 is a list of patterns for additional objects associated with (linked to) the additional object list 401. 403 is a list of installation methods for additional objects associated with (linked to) the subject list 400 and the additional object list 401. Here, the installation method indicates the positional relationship and method when installing additional objects relative to the subjects listed in the subject list 400.

[0036] Next, in the method for determining the additional object performed in step S303 of Example 1, the pattern and position / orientation of the object are determined for each similar image group such that the similarity between multiple images within the group is less than the similarity threshold.

[0037] Similar image groups are created by sorting images based on whether or not they are similar to a specific image. Specifically, for example, using a specific image I1 as a reference, the similarity (R12, R13, ..., R1i) is calculated for all images (I2, I3, ..., Ii) except for image I1, which is stored in the memory acquired in step S301.

[0038] Then, images that exceed the aforementioned similarity threshold are designated as similar images and formed into similar image group G1. Next, using a specific image different from image I1 as a reference, the same process is performed on all images that do not belong to the similar image group to create similar image groups (G1, G2, ..., Gi) for all images. Additionally, images for which no images exceed the similarity threshold are designated as dissimilar image group GN.

[0039] Images belonging to a similar image group are determined based on the subject (imaged object) of each image, such that the patterns of additional objects within the same image group are always different. For images belonging to a dissimilar image group, the patterns and position / orientation of the objects are not determined.

[0040] The method for determining additional objects in each image is as follows: Specifically, an image recognition method using a pre-trained model such as a CNN (Convolutional Neural Network) is used to detect subjects in the image. From the detected subjects, the subject that is present in the subject list 400 and has the highest pattern matching score in image recognition is extracted. Then, the additional object is determined from the additional object list 401 associated with (linked to) that subject.

[0041] The pattern of the additional object in each image is determined sequentially from the patterns listed in the pattern list 402 associated with the additional object, ensuring that there are no duplicates within the same group of similar images. If there are no non-duplicate objects among the objects listed in the pattern list, the pattern of the additional object is modified using rotation or scaling to obtain the pattern of the additional object.

[0042] The position and orientation of the additional object in each image is determined by sequentially selecting an installation method listed in the installation method list 403 associated with the additional object, and then determining the position and orientation of the object relative to the coordinates in the image based on the selected installation method.

[0043] For all images within the similar image group, after determining the pattern and position / orientation of the aforementioned additional object, the determination of the object's pattern and position / orientation is repeated until the similarity between images, assuming the object is in the camera's field of view (environment), falls below a similarity threshold. The similarity between images at this time is calculated using the similarity calculation method used in step S302. An example of how to notify the notification content of the notification unit 103 using a GUI (Graphical User Interface) is explained with reference to Figure 5.

[0044] Figure 5 shows an example of a method for notifying an additional object according to Embodiment 1. 500 is the image display screen of the terminal operated by the user, and 501 is an image display unit that displays multiple images taken at different positions or orientations, acquired in step S301. 502 is a reference image display unit that displays a reference image, which is the first type of image among the multiple images taken at different positions or orientations, acquired in step S301.

[0045] 503 is a reference image display unit that displays a reference image which is the second image among multiple images taken at different positions or orientations acquired in step S301; 504 is a computer graphics (CG) of an additional object determined and notified in step S303; 505 is an environmental map display unit that displays an environmental map (SLAM map); and 506 is a display of a path formed by connecting the position and orientation of the imaging device that took the images acquired in step S301.

[0046] 507 is a first mark indicating the position and orientation of the imaging device that captured the image when the pattern and position and orientation of the first additional object were determined in step S303. 508 is a first similarity that represents the similarity of the multiple images calculated in step S302, which were referenced when the pattern and position and orientation of the first additional object were determined in step S303.

[0047] 509 is a second marker indicating the position and orientation of the imaging device that captured the image when the pattern and position and orientation of the second additional object were determined in step S303. 510 is a second similarity value representing the similarity of the multiple images calculated in step S302, which was referenced when the pattern and position and orientation of the second additional object were determined in step S303.

[0048] 511 is a third mark indicating the position and orientation of the imaging device that captured the image when the pattern and position and orientation of the third additional object were determined in step S303. As explained above, the accuracy of relocalization can be improved by applying the method of Example 1. [Examples]

[0049] In Example 1, the pattern and position / orientation of additional objects to be added to the field of view (environment) corresponding to multiple images acquired by the image acquisition unit 100 are determined based on the subjects in the images. In contrast, in Example 2, the patterns of additional objects are determined to be different from each other within the same image group, based on the distribution of feature points in multiple images. That is, the position of the additional objects is determined based on the distribution of feature points used to calculate the position / orientation of the sensor.

[0050] The distribution of feature points in the aforementioned image group is calculated by applying the SUSAN (Smallest Univalue Segment Assimilating Nucleus) operator to each image.

[0051] Furthermore, the position and orientation of additional objects in each image are determined based on the distribution of feature points in the image group and the number of feature points present in each section, which is obtained by dividing each image into a predetermined number of sections. Specifically, after calculating the feature point distribution for each image, each image is divided into a predetermined number of sections j, and the number of feature points (N11, N12, ..., Nj) in each section is calculated.

[0052] Then, the section with the fewest total number of feature points (N11+N21+...+Ni1, N12+N22+...+Ni2, ...) calculated in each of the multiple i images in the similar image group is determined. The calculated section is then designated as the section within the image in the similar image group to which the additional object should be added.

[0053] Subsequently, from the multiple planes in the area estimated by the Hough transform, the center position of the largest plane (the plane with the most pixels) is taken as the position and orientation of the object relative to the coordinates in the image. In Example 2, the position where the additional object should be added is determined in this way.

[0054] The pattern of the additional object in each image is selected sequentially from a set of system-specific patterns stored in ROM202, ensuring that there are no duplicates within the same group of similar images. In this case, as in Example 1, if there are no non-duplicate patterns among the system-specific patterns stored in ROM202, the pattern of the additional object is modified using rotation or scaling to obtain the pattern of the additional object. [Examples]

[0055] In Example 3, the image acquisition unit 100 independently determines the pattern and position / orientation of an object to be added to the field of view corresponding to multiple images acquired by the image acquisition unit 100. Based on the results, the similarity between the images is calculated, and the determination of the pattern and position / orientation of the object is repeated until the similarity between the images falls below a similarity threshold.

[0056] In Embodiment 3, the image acquisition unit 100 determines the patterns and positional orientations of objects to be added to the field of view corresponding to multiple images acquired by the unit, based on common regions which are similar local areas between the images.

[0057] The method for calculating the common region involves detecting subjects in each image, and if a region similar to the subject exists in the other image, that region is considered the common region. Specifically, the detection of subjects in the images is performed using image recognition techniques with pre-trained models such as CNNs. Then, for the detected subject, a pattern matching score is calculated using SSD (Sum of Squared Difference) against the other image.

[0058] If the score is equal to or greater than the subject similarity threshold read in step S300, that region is determined to be a common region (C1, C2, ..., Ci). If there is no region where the score is equal to or greater than the subject similarity threshold, it is determined to be a non-common region (N1, N2, ..., Ni).

[0059] If it is determined that an object should be added to a common area, the pattern and position / orientation of the object are determined so that the patterns differ between the objects in the area containing the common area with the highest similarity to the subject. If it is determined that an object should be added to a non-common area, the pattern and position / orientation of the object are determined so that the patterns are the same between the objects in the area containing the common area with the lowest similarity to the subject. In this case, the method for determining the pattern and position / orientation of the object is the same as in Example 1, based on the position / orientation obtained by detecting the subject and the information of the additional object associated with the subject.

[0060] The decision of whether to add an object to the common or non-common area is specified by the user via communication I / O 207, or by a system-specific setting stored in ROM 202, and the pattern and position / orientation of the object are determined in the same way as in Example 1. Then, while limiting the attributes of one area, only the attributes of the other area are determined.

[0061] Furthermore, a map generation unit (map generation means) may be provided that generates map information used to calculate the position and orientation of the imaging device, etc., as a sensor, using images captured by the imaging device mounted on the mobile body. In addition, similarity may be calculated using distance information acquired by LIDAR (Laser Imaging Detection And Ranging) in addition to images captured by the imaging device, or the map information described above may be created.

[0062] In that case, the similarity calculation unit 101 calculates the similarity of shapes based on distance information, and the additional object determination unit 102 determines the shape and position / orientation of the additional object in the same manner as in Example 1. When determining the shape and position / orientation of the additional object based on distance information, the subject list 400 is a list of information regarding the shape. In that case, the additional object list 401 is an additional object linked to a shape, and the pattern list 402 is information regarding the shape of the additional object.

[0063] Furthermore, the communication I / O 207 can be of any type, including Ethernet, USB, serial communication, or wireless communication. Also, in step S300, information is read into RAM 203 for initialization of the information processing device, but it is also possible to obtain user-specified information or system-specific information to be stored in ROM 202 via communication I / O 207.

[0064] Furthermore, in step S302, the similarity is calculated using a method based on Bag of Words (BoW), but it is sufficient to calculate the similarity between multiple images, and deep learning may be used to calculate the similarity. Alternatively, the similarity may be calculated based on the brightness values ​​or feature point distribution of the images. In addition, subject similarity may be determined by calculating the similarity in local regions.

[0065] Furthermore, in step S303, it is sufficient to determine the pattern and position / orientation of the additional object so that the similarity R between multiple images is less than the similarity threshold. Therefore, instead of determining them in a predetermined order, they may be determined in a random order or in an order specified by the user via communication I / O 207. Moreover, the additional object may be determined by identifying an additional object lacking features so that the similarity R between multiple images is less than the similarity threshold. Alternatively, multiple objects may be determined as candidates for the additional object, and the object with the smallest similarity between the images may be selected as the additional object.

[0066] Furthermore, in order to reduce the similarity between images, it may be decided to add objects that introduce environmental changes, such as lighting fixtures, which increase the average brightness difference between the images. If lighting fixtures or similar objects are to be added, it may be decided that they should be added in a position where they are not visible in the image.

[0067] Furthermore, while the decision of whether or not to add an object is made based on the similarity R between multiple images, this can be limited to system-specific values ​​or ranges specified by the user via communication I / O 207 or stored in ROM 202.

[0068] Furthermore, if the time difference between the images is less than a predetermined time, it may be determined that adding an object is unnecessary. This is because if the camera's movement distance is less than a predetermined distance, there is a high probability that the similarity between the multiple images is high. The predetermined time can be a system-specific value specified by the user via communication I / O 207 or stored in ROM 202.

[0069] Furthermore, although the information list for additional objects in Figure 4 is read during initialization in step S300, it is sufficient to read it at a time when it can be used in steps S302 and S303. Therefore, it is possible to read, create, and update the information as needed while the information processing device is running. In addition, the pattern list 402 and the installation method list 403 may be calculated by computation so that they can be calculated while the device is running.

[0070] Furthermore, while the installation method list 403 places the object at a position relative to the subject listed in the subject list 400, it is sufficient if the position and orientation of the object can be converted to coordinates within the image in step S303. Therefore, a position relative to the coordinates of the camera's field of view (environment) is also acceptable.

[0071] Furthermore, although the image display unit 501 is composed of two types of image display units, a reference image display unit 502 and a reference image display unit 503, it is acceptable as long as they display similar images. Therefore, it is not limited to two types. It may display one type or three or more types of images.

[0072] Furthermore, the marks indicating the position and orientation of the imaging devices 507, 509, and 511 can be any notification method that allows for the distinction between each position and orientation. For example, the marks may be distinguished and displayed by their size, color, or pattern based on the magnitude of the similarity R between multiple images calculated in step S302.

[0073] Alternatively, the position and orientation of the imaging device used to capture images in which the similarity R calculated in step S302 is determined to be equal to or greater than the similarity threshold may be displayed. In this case, the display may be limited to values ​​or ranges specified by the user via communication I / O 207, or to system-specific values ​​or ranges stored in ROM 202.

[0074] Furthermore, in Examples 1 and 2, the patterns and positions of the additional objects were not duplicated within the same image group, but it is sufficient that they are not similar across multiple images. Therefore, the patterns and positions of the objects may be determined so that they are not similar across all images.

[0075] Furthermore, in Examples 1 and 3, subject detection was performed using image recognition methods with pre-trained models such as CNNs, but it is sufficient if the objects in the image can be distinguished. Therefore, estimation can also be performed using pattern matching with SSD or SAD (Sum of Absolute Difference) based on image data and template data.

[0076] Furthermore, instead of detecting objects using direct methods, the subject may be estimated based on the distribution of feature points within the image. The feature point distribution may also be calculated using the SUSAN operator or based on 3D features such as SHOT (Signature of Histograms of OrienTations) from multiple images and feature points.

[0077] Furthermore, in Example 2, the position and orientation of the object is set to the center of the largest plane in a section with few feature points, but it does not have to be the center. It may be determined at the four corners of the plane or randomly. Also, within the same image group, the patterns of the additional objects are determined to be different from each other, but it is not necessary to limit it to patterns.

[0078] That is, for example, in areas where the distribution of feature points in each image is sparse, the position and orientation of the objects may be determined so that they do not overlap within the same image group, thereby determining the position and orientation of the objects.

[0079] Furthermore, the pattern and position / orientation of the object may be determined such that both the pattern and position / orientation attributes are different by combining the methods described above for different position / orientation and different patterns. That is, when multiple detection pieces of similarity exceeding a predetermined threshold are detected, at least one attribute of the pattern, position, and orientation of the additional object to be added to the detection range may be determined to be different from each other. Alternatively, the pattern and position / orientation may be selectively made different. In that case, the user may select the determination content of the additional object determination unit 102.

[0080] Furthermore, in Example 3, the similarity of subjects in each image is calculated, but it does not have to be a subject; it is sufficient if the image can be distinguished according to a certain rule. Therefore, the common area of ​​each image may be calculated based on the similarity of each section. This section is a system-specific area specified by the user via communication I / O 207 or stored in ROM 202, and the similarity of each section may be calculated using the same method as in step S302.

[0081] Figure 6 shows the configuration of an information processing device in another embodiment. As shown in Figure 6, the image captured by the imaging device 601 mounted on the mobile body 600 may be transmitted to an information processing device (such as a tablet or laptop computer) separate from the mobile body 600, and the user 603 may confirm information about additional objects via the display device 602. Alternatively, the information processing device may be mounted on the mobile body 600 and notify the display device 602 of information about objects to be added via the communication I / O 207.

[0082] In the above-described embodiment, an example of applying the present invention to an autonomous mobile vehicle was explained. However, the mobile vehicle in this embodiment is not limited to autonomous mobile vehicles such as AGVs (Automatic Guided Vehicles) or AMRs (Autonomous Mobile Robots). Furthermore, it may be used for driving assistance purposes even if it does not move completely autonomously.

[0083] Furthermore, the mobile body can be any mobile device that moves, such as an automobile, train, ship, airplane, robot, or drone. Also, as mentioned above, at least a part of the information processing device in the embodiment may or may not be mounted on the mobile body. Moreover, the present invention can also be applied when the mobile body is controlled remotely.

[0084] Although the present invention has been described in detail above based on preferred embodiments, the present invention is not limited to the above embodiments, and various modifications are possible in accordance with the spirit of the invention, and these modifications are not excluded from the scope of the present invention. Furthermore, the above multiple embodiments may be combined as appropriate. The present invention also includes the following combinations.

[0085] (Configuration 1) An information processing device comprising: detection information acquisition means for acquiring detection information detected by a sensor mounted on a moving body; similarity calculation means for calculating the similarity of a plurality of the detection information acquired by the detection information acquisition means, which are detected at different positions or orientations; additional object determination means for determining the position of an additional object to be added to the detection range of the sensor when the detection information was detected, based on the similarity calculated by the similarity calculation means; and notification means for notifying the position of the additional object determined by the additional object determination means.

[0086] (Configuration 2) The information processing device of Configuration 1, characterized in that the sensor includes an imaging means that outputs an image as the detection information.

[0087] (Configuration 3) The information processing device according to Configuration 1 or 2, characterized in that the sensor includes distance measuring means that outputs distance information as detection information.

[0088] (Configuration 4) The information processing apparatus according to any one of Configurations 1 to 3, characterized in that the additional object determination means determines at least one of the orientation and pattern of the additional object.

[0089] (Configuration 5) The information processing apparatus according to any one of Configurations 1 to 4, characterized in that the additional object determination means determines that at least one attribute of the position, pattern, and orientation of the additional object to be added to the detection range when a plurality of detection pieces of information with a similarity of a predetermined threshold or higher are detected is different from one another.

[0090] (Configuration 6) An information processing device according to any one of Configurations 1 to 5, characterized in that it has a map generation means for generating map information used to calculate the position and orientation of the sensor based on the detection information.

[0091] (Configuration 7) The information processing apparatus according to any one of Configurations 1 to 6, characterized in that the additional object determination means determines the position of the additional object based on the distribution of feature points used to calculate the position and orientation of the sensor.

[0092] (Method) An information processing method characterized by comprising: a detection information acquisition step of acquiring detection information output from a sensor mounted on a moving body; a similarity calculation step of calculating the similarity of a plurality of the detection information acquired in the detection information acquisition step, which are detected at different positions or orientations; an additional object determination step of determining the position of an additional object to be added to the detection range of the sensor when the detection information is detected, based on the similarity calculated in the similarity calculation step; and a notification step of notifying the position of the additional object determined in the additional object determination step.

[0093] A computer program for controlling each of the means of the information processing device described in any one of configurations 1 to 7 by a computer.

[0094] Furthermore, the present invention may also be realized by supplying a storage medium containing software program code (control program) that realizes the functions of the embodiments described above to a system or device. It can also be realized by the computer (or CPU or MPU) of the system or device reading and executing the computer-readable program code stored on the storage medium.

[0095] In that case, the program code read from the storage medium itself will realize the function of the embodiment described above, and the storage medium storing that program code will constitute the present invention. [Explanation of Symbols]

[0096] 100: Image acquisition unit 101: Similarity calculation part 102: Additional Object Determination Unit 103: Notification Department

Claims

1. A means for acquiring detection information that acquires detection information detected by a sensor mounted on a mobile body, A similarity calculation means for calculating the similarity of a plurality of detection pieces of information obtained by the detection information acquisition means, which are detected at different positions or orientations from each other; An additional object determination means determines the position of an additional object to be added to the detection range of the sensor when the detection information is detected, based on the similarity calculated by the similarity calculation means. An information processing apparatus characterized by having a notification means for notifying the position of the additional object determined by the additional object determination means.

2. The information processing apparatus according to claim 1, characterized in that the sensor includes an imaging means for outputting an image as the detection information.

3. The information processing apparatus according to claim 1, characterized in that the sensor includes distance measuring means for outputting distance information as detection information.

4. The information processing apparatus according to claim 1, characterized in that the additional object determination means determines at least one of the orientation and pattern of the additional object.

5. The information processing apparatus according to claim 1, characterized in that the additional object determination means determines that at least one attribute of the position, pattern, and orientation of the additional object to be added to the detection range when a plurality of the detection pieces of information with a similarity of a predetermined threshold or higher are detected is different from one another.

6. The information processing apparatus according to claim 1, characterized in that it has a map generation means for generating map information used to calculate the position and orientation of the sensor based on the detection information.

7. The information processing apparatus according to claim 1, characterized in that the additional object determination means determines the position of the additional object based on the distribution of feature points used to calculate the position and orientation of the sensor.

8. A computer-based information processing method, A detection information acquisition step involves acquiring detection information output from a sensor mounted on a mobile object, A similarity calculation step which calculates the similarity of a plurality of detection information obtained in the detection information acquisition step, which are detected at different positions or orientations from each other; An additional object determination step, which determines the position of an additional object to be added to the detection range of the sensor when the detection information is detected, based on the similarity calculated in the similarity calculation step, An information processing method characterized by comprising: a notification step of notifying the position of the additional object determined in the additional object determination step.

9. A computer program for causing a computer to function as one of the means of an information processing apparatus according to any one of claims 1 to 7.

Images