Information processing apparatus, control method, and program

Abstract

To improve the measurement accuracy of the position and orientation of a moving object.SOLUTION: An information processing apparatus for performing position and orientation measurement using sensor data obtained by a sensor, comprises: first characteristic acquisition means for acquiring object variation characteristics indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; nearby object determination means for determining a nearby object interfering with the object whose object variation characteristics have been acquired; second characteristic acquisition means for acquiring, for each of the nearby objects, an object variation factor characteristic that indicates a characteristic that affects accuracy of the position and orientation measurement by interfering with other objects; and position and orientation measurement control means for controlling a position and orientation measurement process so as to limit use of information of the object within the measurement range of the sensor, which is estimated to have a change, based on the object variation characteristics and the object variation factor characteristics.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method, and a program. [Background technology]

[0002] Currently, there is technology for autonomous driving robots (hereinafter referred to as "mobile bodies") that move and work autonomously in various places such as office buildings, homes, and logistics centers. Such mobile bodies calculate feature points from captured images, grasp the space as a map such as point cloud data or a collection of feature points, and move autonomously by comparing the map with the surrounding environment of the mobile body.

[0003] Patent Document 1 discloses an invention that reduces the load of collision avoidance processing by calculating the apparent change in an object from an image captured by an onboard camera of an automobile, recognizing objects with a low risk of collision from the change, and excluding them from the captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-17612 A

[0005] [Non-Patent Document 1] Joseph et al., “You Only Look Once: Unified, Real-Time Object Detection”. CVPR 2016 [Non-Patent Document 2] Ashish et al., “Attention is All you Need”, NeuralIPS2017 [Non-Patent Document 3] Jacob et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, arXiv 2018 [Non-Patent Document 4] Benjamin et al., “I2T: Image Parsing to Text Description,” IEEE, Summary of the Invention [Problem to be solved by the invention]

[0006] However, in Patent Document 1, it was sometimes impossible to recognize an object whose characteristics change. As a result, the accuracy of position and orientation measurement was reduced by the object whose characteristics change.

[0007] The present invention has been made in consideration of the above problems, and has an object to improve the measurement accuracy of the position and orientation of a moving object. [Means for solving the problem]

[0008] An information processing device according to one embodiment of the present invention is an information processing device that performs position and orientation measurement using sensor data from a sensor, and is characterized by comprising: a first characteristic acquisition means that acquires object variation characteristics indicating characteristics that affect the accuracy of the position and orientation measurement for each object within the measurement range of the sensor; a nearby object determination means that determines a nearby object that interferes with the object about which the object variation characteristics have been acquired; a second characteristic acquisition means that acquires object variation factor characteristics for each nearby object that indicate characteristics that affect the accuracy of the position and orientation measurement by interfering with other objects; and a position and orientation measurement control means that controls the position and orientation measurement to limit the use of information about objects within the measurement range of the sensor that is estimated to have variation, based on the object variation characteristics and the object variation factor characteristics. Effect of the Invention

[0009] According to the present invention, it is possible to improve the measurement accuracy of the position and orientation of a moving object. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of a usage scene of the first embodiment of the present invention. [Diagram 2]1 is a configuration diagram of a mobile system including an information processing device according to a first embodiment of the present invention. [Diagram 3] 1 is a hardware configuration diagram of an information processing device according to a first embodiment of the present invention. [Figure 4] 4 is a flowchart showing a process flow of a mobile system including an information processing device according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is an explanatory diagram of a GUI of a mobile system including an information processing device according to a first embodiment of the present invention. [Figure 6] FIG. 11 is a configuration diagram of a mobile system including an information processing device according to a modified example of the first embodiment of the present invention. [Figure 7] 13 is a flowchart showing a detailed flow of acquiring a degree of variation in the information processing device according to a modified example of the first embodiment of the present invention. [Figure 8] 13 is a flowchart showing a detailed flow of acquiring a variation factor degree in the information processing device according to a modified example of the first embodiment of the present invention. [Figure 9] FIG. 11 is a configuration diagram of a mobile system including an information processing device according to a second embodiment of the present invention. [Figure 10] 10 is a flowchart showing a process flow of a mobile system including an information processing device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [Example 1] First, a first embodiment of the present invention will be described. The first embodiment relates to the autonomous control of a moving body. The autonomous control of a moving body calculates the position and orientation of the moving body from a sensor mounted on the moving body, and controls the moving body to follow a plurality of waypoints. In this embodiment, the position and orientation of the moving body is measured using Visual SLAM, which calculates the position and orientation by matching image information captured by an imaging device mounted on the moving body as a sensor with two-dimensional or three-dimensional map information. SLAM is an abbreviation for Simultaneous Localization and Mapping. In the Visual SLAM of this embodiment, feature points are extracted from a captured image, and the position and orientation of the imaging device is measured by feature point matching.

[0013] However, the object variability (the term will be defined later) does not depend only on the type information of the target object, but changes depending on the positional relationship between the target object and other nearby objects. Therefore, in this embodiment, the likelihood of the target object itself moving / rotating or changing in appearance, and the likelihood of objects nearby the target object moving / rotating or changing in appearance are evaluated. Based on the results of these two evaluations, the possibility of the target object moving / rotating or changing in appearance is evaluated. If this evaluation is higher than a predetermined threshold, the features on the target object from among the image features for position and orientation measurement are not used for position and orientation measurement processing, thereby improving the position and orientation measurement accuracy.

[0014] In this embodiment, the characteristics of an object that affect the accuracy of SLAM position and orientation measurement due to its movement, rotation, or change in appearance are called object variation characteristics, and the degree of the effect is called object variation. The larger the object variation, the greater the effect on the accuracy of SLAM position and orientation measurement.

[0015] Furthermore, the characteristics of an object that affect the accuracy of SLAM position and orientation measurement by interfering with another object and causing the other object to move, rotate, or change in appearance are called object variation factor characteristics, and the degree of influence is called object variation factor degree. The greater the object variation factor degree, the greater the influence on SLAM position and orientation measurement accuracy. In this embodiment, the object for which the object variation characteristics are sought is called the target object, and the object for which the object variation factor characteristics are sought is called the nearby object.

[0016] 1 is a diagram showing an example of a usage scene of a mobile body system including an information processing device according to this embodiment, viewed from above. Reference numeral 100 denotes a mobile body. The information processing device according to this embodiment is disposed on the mobile body 100. The information processing device according to this embodiment measures the position and orientation of the mobile body 100 using an image captured by an imaging device disposed on the mobile body 100.

[0017] Reference numeral 101 denotes an object whose movement affects the accuracy of position and orientation measurement of the moving body 100. Reference numeral 102 denotes an object identical to the object 101. Reference numeral 103 denotes a conveying device, which is capable of moving the object 101.

[0018] 2 is a diagram showing the functional configuration of a mobile body system including an information processing device according to this embodiment. 203 is an information processing device according to this embodiment. The information processing device 203 has a first characteristic acquisition unit 204, a nearby object determination unit 205, a second characteristic acquisition unit 205, and a position and orientation measurement control unit 207. The mobile body system according to this embodiment has an image acquisition unit 201, an object recognition unit 202, an information processing device 203, and a position and orientation measurement unit 208.

[0019] In this embodiment, the image acquisition unit 201 is an imaging device disposed on the moving body 100. The object recognition unit 202 recognizes object type information of a target object contained in a captured image acquired by the image acquisition unit 201, and pixel area information of the target object in the captured image. In this embodiment, a word representing an object name is used as the object type information. The image acquisition unit 201 is an example of a sensor. The captured image captured by the image acquisition unit 201 is an example of sensor data detected by a sensor. The object recognition unit 202 recognizes an object from the sensor data.

[0020] The first characteristic acquisition unit 204 acquires an object variation characteristic based on the object type information of the target object input from the object recognition unit 202. In this embodiment, the first characteristic acquisition unit 204 acquires an object variation degree, which is the degree of the object variation characteristic.

[0021] The nearby object determination unit 205 determines nearby objects that are close to the object whose characteristics have been acquired by the first characteristic acquisition unit 204, based on the object type information and pixel area information of the object input from the object recognition unit 202.

[0022] The second characteristic acquisition unit 206 acquires an object change factor characteristic based on the object type information of the nearby object determined by the nearby object determination unit 205. In this embodiment, the second characteristic acquisition unit 206 acquires an object change factor degree which is the degree of the object change factor characteristic.

[0023] The position and orientation measurement control unit 207 determines the control content of the position and orientation measurement process based on the object change characteristics acquired by the first characteristic acquisition unit 204, pixel area information about the target object in the image, and object change factor characteristics acquired by the second characteristic acquisition unit 205.

[0024] The position and orientation measurement unit 108 performs position and orientation measurement based on the control content determined by the position and orientation measurement control unit 207 .

[0025] 3 is a diagram showing a hardware configuration of the information processing device 203. Reference numeral 301 denotes a CPU, which executes a program describing the operation of this embodiment to realize the processing of this embodiment. CPU is an abbreviation for Central Processing Unit. CPU 301 also controls various devices connected to a system bus 309.

[0026] Reference numeral 302 denotes a ROM, which stores a BIOS program and a boot program. ROM is an abbreviation for Read Only Memory. BIOS is an abbreviation for Basic Input Output System. Reference numeral 303 denotes a RAM, which is used as a main storage device for the CPU 01. RAM is an abbreviation for Random Access Memory. Reference numeral 304 denotes an external memory, which stores programs to be processed by the information processing device 203.

[0027] The input unit 305 performs processing related to input of information, etc., via a keyboard, a mouse, etc. The display unit 306 outputs the calculation results of the information processing device 203 to a display device according to instructions from the CPU 301. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED indicator.

[0028] Reference numeral 307 denotes a communication I / F. I / F is an abbreviation for interface. The first characteristic acquisition unit 204, the nearby object determination unit 205, and the position and orientation measurement control unit 207 input object type information and output control contents via a system bus 309. The communication I / F 307 communicates information via a network. The communication interface supported by the communication I / F 307 may be Ethernet, or any type of interface such as USB, serial communication, or wireless communication. USB is an abbreviation for Universal Serial Bus. Reference numeral 308 denotes an I / O. I / O is an abbreviation for Input / Output.

[0029] 4 is a flowchart for explaining the processing of a mobile system including the information processing device 203. In step S401, the information processing device 203 executes an initialization process. That is, the CPU 301 reads and executes a program from the external memory 304, and puts the information processing device 203 into an operable state.

[0030] In the initialization process, for example, tables that store object type information, object change characteristics, object change factor characteristics, etc. acquired and used by each component, and control contents determined by the position and orientation measurement control 207 are expanded in the RAM 303. In this embodiment, it is assumed that the user inputs information for each table regarding objects in the environment in which the mobile object moves using an input device (not shown). Here, the data format of each table in this embodiment will be described.

[0031] In step S404, the first characteristic acquisition unit 204 refers to the object variation characteristic table. The object variation characteristic table can uniquely determine the object variation degree of each object based on the object type information of the object. In this embodiment, for objects that have a moving function, TRUE is stored in the table as the object variation degree associated with the object type information. On the other hand, for objects that do not have a moving function, FALSE is stored in the table as the object variation degree associated with the object type information.

[0032] In step S406, the second characteristic acquisition unit 206 refers to the object change factor characteristic table. The object change factor characteristic table can uniquely determine the object change factor degree of each object based on the object type information of the object. In this embodiment, for objects that have a function for moving, TRUE is stored in the table as the object change factor degree associated with the object type information. On the other hand, for objects that do not have a function for moving, FALSE is stored in the table as the object change factor degree associated with the object type information.

[0033] After the series of initialization processes are completed, in step S402, the image acquisition unit 201 acquires image data from the imaging device. In step S403, the object recognition unit 202 recognizes a target object and a nearby object included in the image acquired by the image acquisition unit 201. The contents described in Non-Patent Document 1 can be applied to a method of recognizing an object in an image and pixel area information on an object on an image.

[0034] In step S404, the first characteristic acquisition unit 204 acquires object variation characteristics based on the object type information. Specifically, the first characteristic acquisition unit 204 determines an object for which the control content of the position and orientation measurement has not been determined as a target object among the multiple objects recognized in step S403. The first characteristic acquisition unit 204 then acquires the object variation degree of the target object by searching the object variation characteristic table using the object type information of the target object as a key.

[0035] In step S405, the nearby object determination unit 205 determines a nearby object that is close to the object whose object fluctuation degree has been acquired by the first characteristic acquisition unit 204, and acquires object type information of the determined nearby object. In this embodiment, the nearby object determination unit 205 calculates the area center of gravity of each object from the area information of the multiple objects recognized by the object recognition unit 202. Then, the nearby object determination unit 205 determines the object having the smallest center of gravity distance from the target object as the nearby object.

[0036] In step S406, the second characteristic acquisition unit 206 acquires an object change factor characteristic based on the object type information of the nearby object. In this embodiment, the object change factor degree of the nearby object is acquired by searching the object change factor characteristic table using the object type information of the nearby object as a key.

[0037] In step S407, the position and orientation measurement control unit 207 determines the control content of the position and orientation measurement unit 208 based on the object variation degree and the object variation factor degree. In this embodiment, the position and orientation measurement control unit 207 calculates the logical sum of the object variation degree and the object variation factor degree, and determines the application of the control content read in step S401 based on the calculation result. Specifically, when the calculation result is TRUE, the position and orientation measurement control unit 207 does not use the features as the control content in the image area of ​​the target object for position and orientation measurement. When the calculation result is FALSE, the position and orientation measurement control unit 207 does not perform control and uses the features in the image area of ​​the target object for position and orientation measurement. The calculation performed by the position and orientation measurement control unit 207 on the object variation degree and the object variation factor degree may be a logical AND or an arithmetic calculation.

[0038] In step S408, the information processing device 203 judges whether or not to end the process. In this embodiment, the information processing device 203 judges to end the process when the control contents have been determined for all objects recognized in step S403, and judges not to end the process otherwise. If the information processing device 203 judges to end the process, the process of step S409 is executed. If the information processing device 203 judges not to end the process, the process of step S404 is executed.

[0039] In step S409, the position and orientation measurement unit 208 measures the position and orientation of the image capturing apparatus based on the control content determined in step S407.

[0040] In the above-described first embodiment, control is performed to exclude features within a pixel area of ​​an object with a large degree of object variation from a captured image based on the object variation characteristics of the target object and the object variation factor characteristics of a nearby object from the position and orientation measurement processing. This makes it possible to improve the accuracy of position and orientation measurement in the first embodiment.

[0041] <Variation 1-1> The control contents of the position and orientation measurement control unit 207 in the first embodiment may be confirmed by the user using a GUI. Fig. 5 shows an example of the control contents of the position and orientation measurement unit 208 being output on a display screen.

[0042] Reference numeral 500 denotes a captured image captured by an imaging device disposed on the moving body 100, acquired by the image acquisition unit 201. Reference numeral 501 denotes a target object whose movement or change in appearance affects the measurement accuracy of the position and orientation measurement unit 208. Reference numeral 502 denotes a driving device that is in close proximity to the target object 501 and has a function of moving the target object 501. Reference numeral 503 denotes a warning mark superimposed on the captured image based on the control content of the position and orientation measurement control unit 207. Reference numeral 504 denotes a pixel area that the target object 501 occupies in the captured image. Reference numeral 505 denotes a display field that displays the control content of the position and orientation measurement control unit 207 in a sentence format.

[0043] Display field 505 displays that the object (target object 501) in area 1 (pixel area 504) may be moved by the drive device (drive device 502), and therefore the features of area 1 will not be used for position and orientation measurement.

[0044] As described above, the user can visually recognize the control contents of the position and orientation measurement control unit 207 by using the GUI shown in FIG.

[0045] <Variation 1-2> In the first embodiment, the object fluctuation degree is defined as "presence or absence of a moving function", but the object fluctuation degree is not limited to the presence or absence of a moving function, and may be any fluctuation characteristic of the target object that affects the accuracy of the position and orientation measurement. For example, the object fluctuation degree may be determined based on the presence or absence of an opening and closing function such as a door or hinge, rather than the moving function. The object fluctuation degree may also be determined based on the presence or absence of a rotation mechanism such as a bearing. This makes it possible to evaluate the object fluctuation degree more accurately, and thus to further improve the accuracy of the position and orientation measurement.

[0046] In addition, the present invention may determine the object movement degree not based on the "presence or absence" of the function of moving, but based on the "degree of function" of moving. For example, the smaller the friction coefficient, the easier it is to move, so the object movement degree may be determined to be large.

[0047] As described above, the degree of function is not limited to the "function of moving." Here, a method of controlling position and orientation measurement depending on the degree of function will be described. Note that only differences from the first embodiment will be described.

[0048] In step S401, the information processing device 203 reads a table that can uniquely determine an object fluctuation degree from object type information of a target object. Here, the object fluctuation degree is set to a sliding friction coefficient. The information processing device 203 also reads a formula and a threshold value to be used in step S407. The formula and the threshold value will be described later.

[0049] In step S404, the first characteristic acquisition unit 204 acquires the sliding friction coefficient from the object type information of the target object as the object fluctuation degree. Here, the object fluctuation degree is acquired by referring to the table read in step S401.

[0050] In step S407, the position and orientation measurement control unit 207 determines the control content based on the object change degree acquired in step S404 and the object change factor degree acquired in step S406. Here, if the nearby object has a function to move the target object, the object change factor degree is acquired as "1". Also, if the nearby object does not have a function to move the target object, the object change factor degree is acquired as "0". The position and orientation measurement control unit 207 determines the control content according to formula 1.

[0051] 1 / ω×f>th...(Equation 1) In the formula 1, ω is the object fluctuation degree, that is, the sliding friction coefficient, f is the object fluctuation factor degree, and th is a threshold value that is set in advance and read in step S401.

[0052] The position and orientation measurement control unit 207 uses the features included in the image area of ​​the target object for position and orientation measurement processing when formula 1 is satisfied. That is, the position and orientation measurement control unit 207 uses the features included in the image area of ​​the target object for position and orientation measurement processing only when the target object has a sliding friction coefficient equal to or less than a certain level and the nearby object has a function to move it.

[0053] In formula 1, the control content is determined by the threshold value th, but the variable value of the control content may be determined from the left side value of formula 1. For example, the larger the left side value of formula 1, the smaller the weight in the position and orientation measurement process may be for the features included in the pixel region of the target object.

[0054] According to this modification, as described above, the object fluctuation degree is evaluated by degree, so that the object fluctuation degree can be evaluated more precisely, and the accuracy of position and orientation measurement can be further improved.

[0055] Similarly, the object variation factor degree is not limited to "presence or absence of a function to move". The object variation factor degree may be any property of a nearby object that affects the accuracy of position and orientation measurement by changing the position, orientation, or appearance of surrounding objects. The object variation factor degree may be, for example, a property of moving such as a cart, or a property of changing appearance such as paint and a brush. This makes it possible to more accurately evaluate the variation factor degree, and ultimately to further improve the accuracy of position and orientation measurement.

[0056] Furthermore, the object variation factor degree may be determined not only by the "presence or absence of a function" but also by the "degree of function". In this case, the greater the product of the object variation degree and the object variation factor degree of a feature, the smaller the weight in the position and orientation measurement process may be for the feature included in the image area of ​​the target object. In this way, by evaluating the object variation factor degree as a degree, it is possible to evaluate the object variation factor degree more precisely, and the accuracy of the position and orientation measurement can be further improved.

[0057] <Modification 1-3> In the first embodiment, the position and orientation measurement control unit 207 determines the control content regarding whether or not features can be used, but any control that improves the position and orientation measurement accuracy may be used. For example, a priority may be set for the use of features in position and orientation measurement according to the calculation result. Here, the priority in the position and orientation measurement process is set higher for features of a target object with a smaller object fluctuation degree. In this way, even if there is a limit to the number of features used in the position and orientation measurement process, the accuracy of the position and orientation measurement can be guaranteed.

[0058] <Modification 1-4> In the first embodiment, in step S407, the evaluation is performed by the logical sum of the object variation degree and the object variation factor degree, but other calculation methods may be used as long as they are means for determining control that improves the position and orientation measurement accuracy. For example, calculation may be performed by the logical product of the object variation degree and the object variation factor degree, and there is no need to reduce the features to be used. Alternatively, the object variation degree and the object variation factor degree may be calculated by arithmetic calculation, and the features may be weighted.

[0059] In this way, the accuracy of position and orientation measurement can be improved in various usage scenarios.

[0060] <Variation 1-5> The present invention may control the process of storing the features in the map information according to the calculation result in step S407. For example, features included in the pixel area of ​​the target object for which the calculation result is FALSE may not be stored in the map information. If the object variation degree and the object variation factor degree are degrees as described above, features in the image area of ​​the target object may not be stored in the map information if the product of the object variation degree and the object variation factor degree is less than a predetermined threshold value.

[0061] In this way, object features with high object fluctuations can be prevented from being stored in the map information, and a decrease in the accuracy of position and orientation measurement can be prevented.

[0062] <Variation 1-6> In the first embodiment, in step S405, an object whose center of gravity in the captured image is the smallest from the target object is determined as the nearby object. However, in the present invention, the number of nearby objects does not need to be limited to one. For example, in the area information of each object in the captured image, objects whose center of gravity of the object in the area is less than a threshold value are selected as a nearby object group. After that, if any one of the multiple nearby objects has a function of moving, the object change factor degree of the nearby object group composed of the multiple nearby objects may be set to "has a function of moving."

[0063] Alternatively, the object variation factor degree of a nearby object or an object included in a nearby object group may be evaluated so that the variation factor degree decreases as the distance from the target object increases.

[0064] According to this modification, by evaluating various object variation factor degrees in this way, it is possible to improve the accuracy of the evaluation.

[0065] <Variation 1-7> In the above description, the position and orientation control unit 207 uniformly determines whether to use, prioritize, or weight in the position and orientation measurement process for a group of features within a pixel region that the target object occupies in the image. However, in the present invention, the use or non-use, prioritize, or weight in the position and orientation measurement process may be determined according to the position of a feature, taking into account the shape, position, and orientation of the target object and nearby objects.

[0066] The process will now be described. In step S401, the information processing device 203 reads a table that can uniquely determine an address for the three-dimensional model data of the target object stored in a storage means (not shown) from the object type information of the target object. Furthermore, the information processing device 203 also reads a table that can refer to information on movement, rotation, and appearance changes that may occur in the target object from the object type information of the target object. In this modification, it is assumed that horizontal movement, rotation around the vertical axis, etc. are stored for each target object.

[0067] In step S407, the position and orientation measurement control unit 207 performs a calculation based on the object variation degree acquired in step S404 and the object variation factor degree acquired in step S406, and refers to a three-dimensional model of the target object according to the calculation result. In this modification, when the logical sum of the object variation degree and the object variation factor degree is TRUE, the three-dimensional model is referred to.

[0068] The position and orientation measurement control unit 207 measures the relative position and orientation of the target object and the imaging device by performing matching processing on the captured image and the three-dimensional model of the target object. After that, it calculates the relative position and orientation of the target object and the imaging device after changes in the target object, such as possible translation, rotation, and appearance, and superimposes the three-dimensional model after the changes on the captured image.

[0069] From the superimposition result, the position and orientation measurement control unit 207 selects feature points that can be measured unchanged before and after a change in the target object within the pixel area occupied by the target object in the captured image. In this modified example, features that cannot be measured after a change are not used. Alternatively, a probability distribution may be included in information on possible translation, rotation, and appearance changes, and weighting may be performed so that features that cannot be measured when a change with a low probability occurs are not used in position and orientation measurement.

[0070] In this way, according to this modification, by using a three-dimensional model to select features within a pixel region, it is possible to eliminate only a minimum number of features, thereby improving the accuracy of position and orientation measurement.

[0071] <Variation 1-8> In the first embodiment, the position and orientation control unit 207 determines the control content of the position and orientation measurement, but the control target may be anything as long as the accuracy of the position and orientation measurement is improved. For example, the driving device arranged on the moving body 100 may be controlled so that the moving body 100 does not pass near a target object having a large object fluctuation degree. Alternatively, the orientation of the moving body 100 may be changed so that the target object is not included in the observation range of a sensor for position and orientation measurement.

[0072] In this way, according to this modification, it is possible to more reliably suppress deterioration in the position and orientation measurement accuracy.

[0073] <Variation 1-9> In the first embodiment, the object variation characteristic and the object variation factor characteristic are acquired by referring to a table using the object type information as a key. In this modified example, a method of estimating the object variation characteristic and the object variation factor characteristic using an object arrangement characteristic database that holds the object type information and positional relationship of an object group will be described.

[0074] Fig. 6 is a block diagram showing the configuration of an information processing device 203 according to this modification, which estimates object variation characteristics and object variation factor characteristics by using an object arrangement characteristic database. In the following explanation of Fig. 6, the explanation will be limited to the difference from Fig. 2.

[0075] The object recognition unit 202 recognizes object type information regarding the target object and its surrounding objects, and pixel area information that each object occupies in the captured image. The object variation characteristic evaluation unit 601 evaluates object variation characteristics based on the object type information input from the first characteristic acquisition unit 204. The object variation factor characteristic evaluation unit 603 evaluates object variation factor characteristics based on the object type information input from the second characteristic acquisition unit 206.

[0076] The object placement characteristic database 602 is a database that holds placement characteristics that indicate the positional relationships of a plurality of objects. The placement characteristics are knowledge data that generalize the three-dimensional positional relationships of objects in the real world. Specifically, the object placement characteristic database 602 holds within the database characteristics such as "a chair is likely to be next to a desk, but is never on top of a desk." In addition, the object placement characteristic database 602 holds within the database characteristics such as what placements and configurations are common in reality (what placements and configurations are not common).

[0077] The object arrangement characteristic database 602 in this embodiment is a pre-trained neural network trained to infer object characteristic information of component parts of an object from surrounding object characteristic information. Specifically, it is a trained neural network in which 24 layers of Transformers described in Non-Patent Document 2 are stacked. In addition, in this example, the number of input dimensions and the number of output dimensions of the Transformer are 512, that is, a maximum of 512 pieces of object characteristic information are input, and the same number of 512-dimensional outputs are obtained. Specifically, the encoder network used in Non-Patent Document 3 is used.

[0078] 7 is a flowchart showing the flow of processing in the object variation characteristic evaluation unit 601 in this modified example. In step 701, the first characteristic acquisition unit 204 inputs object type information and position information of the target object and its surrounding objects to the object variation characteristic evaluation unit 601 based on the captured image acquired by the image acquisition unit 201.

[0079] In step 702, the object variation characteristic evaluation unit 601 generates a feature vector that lists object type information of the target object and its peripheral objects, and a position vector that indicates the positions of these objects. Also in step 702, the object variation characteristic evaluation unit 601 MASKs the position coordinate element that corresponds to the target object in the generated position vector. Here, masking the position coordinate element means replacing the position element with a MASK token (a special label that indicates that the position coordinate is unknown). Here, the object type vector is a one-dimensional column vector, and object type information is placed in each element.

[0080] In step 703 , the object variation characteristic evaluation unit 601 inputs the vector generated and masked in step 702 to the object arrangement characteristic database 602 .

[0081] In step 704 , the object variation characteristic evaluation unit 601 acquires the output vector of the object arrangement characteristic database 602 .

[0082] In step 705, the object variation characteristic evaluation unit 601 evaluates the object variation characteristic based on the output vector of the object arrangement characteristic database 602. In this modification, the object variation characteristic evaluation unit 601 evaluates the object variation characteristic based on the likelihood of the inference result for the position and orientation elements of the output vector. The object variation characteristic evaluation unit 601 evaluates so that the object variation degree of the target object is high when the likelihood of the inference result of the position coordinates of an object other than the target object is low, and the object variation degree of the target object is low when the likelihood is low. The likelihood is determined based on whether it is large or small relative to the threshold value read in step S401. In this modification, the object variation characteristic evaluation unit 601 inputs the evaluation result to the first characteristic acquisition unit 204.

[0083] 8 is a flowchart showing a process flow in the object change factor characteristic evaluation unit 603 in this modified example. In step 801, the second characteristic acquisition unit 206 inputs object type information of the nearby object and the surrounding objects to the object change factor characteristic evaluation unit 603 based on the captured image acquired by the image acquisition unit 201.

[0084] In step 802, the object change factor characteristic evaluation unit 603 generates a position vector and an object type vector in which the position coordinate elements are masked. In this modification, the object change factor characteristic evaluation unit 603 masks the position coordinate elements of all objects.

[0085] In step 803 , the object variation factor characteristic evaluation unit 603 inputs the vector generated in step 802 into the object arrangement characteristic database 602 .

[0086] In step 804 , the object variation factor characteristic evaluation unit 603 acquires the output vector of the object arrangement characteristic database 602 .

[0087] In step 805, the object change factor characteristic evaluation unit 603 evaluates the object change factor characteristic based on the output vector of the object arrangement characteristic database 602. In this modification, the object change factor characteristic evaluation unit 603 evaluates the object change characteristic based on the likelihood of the inference result for the position and orientation elements of the output vector. The object change characteristic evaluation unit 601 evaluates so that the object change factor degree of the nearby object is high when the likelihood of the inference result of the position coordinates of an object other than the nearby object is low, and the object change factor degree of the nearby object is low when the likelihood is low. The level of the likelihood is determined based on the magnitude of the threshold read in step S401. In this modification, the object change factor characteristic evaluation unit 603 inputs the evaluation result to the second characteristic acquisition unit 206.

[0088] In this way, according to this modified example, by utilizing the object arrangement characteristic database 602, the characteristics of the target object and nearby objects can be obtained by prediction, thereby reducing the complexity of setting the object variation degree and object variation factor degree.

[0089] In this modification, the object variation characteristic evaluation unit 601 and the object variation factor characteristic evaluation unit 603 evaluate the object variation degree and the object variation factor degree by comparing the input vector and the output vector of the object placement characteristic database 602. The evaluation method of the present invention is not limited to the comparison of the input vector and the output vector. For example, a plurality of types of output vectors, which are the prediction results of the object placement characteristic database 602, may be obtained, and the more the number of output vectors, the easier it is to move or to make it move, and the object variation degree and the object variation factor degree may be evaluated as being high. Alternatively, the likelihood of the prediction result may be obtained together with the prediction result, and the lower the likelihood, the easier it is to move, and the object variation degree may be evaluated as being high.

[0090] In this way, according to this modification, the object variation degree and the object variation factor degree can be evaluated with high accuracy even in a variety of environments.

[0091] In addition, in this modification, the object variation degree is evaluated from a vector generated based on the captured image by the image acquisition unit 201, but any method may be used as long as the object variation degree can be evaluated using a plurality of captured images. For example, the object variation degree may be evaluated by taking into account the object variation degree of the target object evaluated from one image and the object variation degree evaluated from another image.

[0092] In this manner, according to this modification, the object fluctuation degree of the target object can be evaluated with high accuracy without being affected by a specific shooting environment or object.

[0093] Furthermore, the nearby object determination unit 205 may utilize the object arrangement characteristic database 602. For example, the nearby object determination unit 205 inputs a feature vector in which object type information other than the target object is masked to the object arrangement characteristic database 602, based on the object type information input from the object recognition unit 202 and the first characteristic acquisition unit 204. Thereafter, the nearby object determination unit 205 can predict objects generally close to an object by acquiring the output of the object arrangement characteristic database 602.

[0094] In this way, according to this modification, by using the object arrangement characteristic database 602, it is possible to reduce the failure to recognize nearby objects and further improve the accuracy of the position and orientation measurement process.

[0095] Furthermore, the position and orientation measurement control unit 207 may use the object arrangement characteristic database 602 for features in a pixel area occupied by the target object in the image to simulate and select the position of a target object with a high object variation degree or a nearby object with a high object variation factor degree. Specifically, the position and orientation measurement control unit 207 inputs a feature vector and a position vector including the target object and the nearby object to the object arrangement characteristic database 602. At this time, the position coordinate elements corresponding to the target object and the nearby object are masked. Thereafter, the position and orientation measurement control unit 207 acquires an output vector from the object arrangement characteristic database 602, changes the position coordinates that were masked at the time of input, and acquires the positional relationship before and after the change by setting the position coordinates of the output vector as the positional relationship before the change. The simulated result is superimposed on the captured image from the three-dimensional model read in the initialization process of step S401 and the positional relationship before and after the change. The position and orientation measurement control unit 207 performs control such that features that cannot be measured between cases where the result of the superposition is a predetermined general arrangement and cases where it is not are not used for position and orientation measurement, and features that can be measured regardless are used for position and orientation measurement.

[0096] In this way, according to this modification, by estimating the positional relationships between multiple objects using the object arrangement characteristic database 602 and simulating the features, it is possible to improve the accuracy of position and orientation measurement even in an environment where multiple objects are present.

[0097] In this modification, the object variation characteristic evaluation unit 601 evaluates the object variation degree by referring to the object placement characteristic database 602, but the database does not have to be the object placement characteristic. For example, in the environment in which the moving body 100 moves, a storage unit that stores information on the movement of the object may be provided, and the object variation characteristic evaluation unit 601 may evaluate the variation degree by referring to the storage unit.

[0098] The information on the movement of an object here may be information acquired from a sensor (for example, an image sensor such as a surveillance camera and an object recognition device) installed in the environment in which the moving body moves. The information on the movement of an object may also be the difference in position coordinates in a horizontal two-dimensional coordinate system of the environment in which the object exists. The information on the movement of an object may also be the difference in coordinates between a certain time and a different time. The times different from the certain time may be any two times.

[0099] The storage unit stores object type information and the difference in position coordinates in association with each other, and the object change characteristic evaluation unit 601 refers to the storage unit, and if there is information that the object position and orientation have changed, evaluates the object change degree as 1, otherwise as 0. Alternatively, for information on movement, the object change degree may be evaluated by comparing a numerical value such as distance or rotation angle with a threshold value.

[0100] The storage unit may also store document information. Document information is data including sentences such as documents and news articles, and is data created for the purpose of being read, seen, and heard by humans. For example, in document information showing a work process, attention may be paid to a portion describing a target object and surrounding objects, and it may be determined that the greater the number of types of arrangements of the target object and surrounding objects, the greater the degree of variation that can be used in various work locations. A method for recognizing the positional relationship between objects from sentences in document information is described in Non-Patent Document 4.

[0101] Similarly to the object change characteristic evaluation unit 601, the object change factor characteristic evaluation unit 603 may not use the object placement characteristic database 602. The object change factor characteristic evaluation unit 603 may use a storage unit that stores information on an object moving another object. This may include the presence or absence of information on an object moving another object, or the distance of movement within the information on the object being moved.

[0102] As described above, by controlling the position and orientation measurement process based on the actual amount of movement of the target object and the amount of movement of surrounding objects caused by nearby objects, this modified example can more reliably improve the accuracy of position and orientation measurement.

[0103] [Example 2] Next, a second embodiment of the present invention will be described. In the first embodiment, a method for improving accuracy by controlling position and orientation measurement based on the characteristics of an object and a nearby object close to the object has been described. In the second embodiment, a method for improving accuracy by controlling position and orientation measurement based on the characteristics of an area in which an object exists will be described.

[0104] In this embodiment, the characteristics of an area that an object in the area has on the accuracy of SLAM position and orientation measurement due to its movement, rotation, or change in appearance is called the area variation characteristics, and the degree of influence is called the area variability. In this embodiment, the larger the area variability is, the lower the weight in the position and orientation measurement process is given to the features in the area that an object existing inside occupies on the image.

[0105] Fig. 9 shows the configuration of an information processing device 203 according to the second embodiment. In Fig. 9, the same components as those in Fig. 2, which is a configuration diagram of the first embodiment, are denoted by the same reference numerals, and the description will be omitted, and the description will be limited to the differences from Fig. 2.

[0106] The area determination unit 901 determines an area in which a target object exists, and outputs area type information to the area characteristic acquisition unit 902 .

[0107] The area characteristic acquisition unit 902 acquires area variation characteristics based on the input area type information. In this embodiment, the area characteristic acquisition unit 902 acquires an area variation degree, which is the degree of the area variation characteristic. The area characteristic acquisition unit 902 outputs the acquired area variation degree to the position and orientation measurement unit 207.

[0108] Fig. 10 is a flowchart showing a process flow of a mobile system including an information processing device 203 according to the embodiment 2. In Fig. 10, the same components as those in Fig. 4, which is the flowchart of the embodiment 1, are denoted by the same reference numerals, and the description is omitted, and the description is limited to the difference from Fig. 4.

[0109] In step S1001, the region determination unit 901 determines region type information. In this embodiment, the region determination unit 901 determines the region type information by taking the region name input by the user from the input / output unit as the region type information.

[0110] In step S1002, the area characteristic acquisition unit 902 acquires area variation characteristics based on the input area type information. In this embodiment, the area variation characteristics are acquired by referring to the area variation characteristics table read in the initialization process of step S401. Here, the area variation characteristics table is a table that allows the area variation degree to be referenced from the area type information. In this embodiment, the area variation degree is determined by the ratio of the number of movable objects to the total number of objects existing in the area at any time. That is, if more than half of the objects in the area are movable, the area characteristic acquisition unit 902 sets the area variation degree of the area to TRUE, and if less than half of the objects are movable, sets the variation degree to FALSE.

[0111] In step S404, the first characteristic acquisition unit 204 acquires an object variation degree as an object variation characteristic based on the object type information.

[0112] In step S407, the position and orientation measurement control unit 207 determines the control content for position and orientation measurement based on the input object variability and area variability. In this embodiment, the object variability and area variability are calculated by logical sum, and the application of the control content read in step S401 is determined according to the calculation result. Specifically, the control content is determined such that if the calculation result is TRUE, the features in the image area of ​​the target object are not used for position and orientation measurement, and if the calculation result is FALSE, the features are used.

[0113] According to the second embodiment described above, a control is performed to remove a pixel area of ​​an object with a large degree of fluctuation from a captured image based on the fluctuation characteristics of the target object and the area fluctuation characteristics of the area where the target object exists. This control can improve the control of position and orientation measurement.

[0114] <Variation 2-1> In the second embodiment, the region variability is determined based on the number of movable objects in the region, but the region variability may be any characteristic that affects the accuracy of position and orientation measurement. For example, the region variability may be determined based on the presence or absence of a function to move and rotate objects in the region. Alternatively, the region variability may be determined based on the degree of the function, rather than the presence or absence of the function. For example, as in modification 1-2, the position and orientation control content may be determined by threshold processing using a friction coefficient as in Equation 1 as a variable.

[0115] In this way, according to this modification, by setting the degree of the area variation characteristic, it is possible to more reliably improve the accuracy of position and orientation measurement.

[0116] <Variation 2-2> Also, a storage unit (e.g., a storage device) may be provided that stores position and orientation information of an object in the region together with a time series, and the region fluctuation degree may be determined from the change in position and orientation stored in the storage unit. For example, if information is stored showing that an object is moved from inside the region to outside the region at two different times, the region fluctuation degree may be set to be greater as the difference between the two different times is smaller. Alternatively, the region fluctuation degree may be set to be greater as the moving distance of the object at the two different times is greater, instead of the difference in time.

[0117] As described above, according to this modification, the position and orientation measurement process is controlled based on the past movement history of the object, thereby making it possible to more reliably improve the accuracy of the position and orientation measurement.

[0118] <Modification 2-3> The area variability may use the object placement characteristic database described in the modified example 1-7 for objects in the area. For example, a task database is added that infers the type information vector and position vector of a general object in the area when the area type name is input based on the object placement envoy database. The task database may be used to obtain the likelihood of the inference result, and the area variability may be evaluated so that the area variability is high in areas with low likelihood and low in areas with high likelihood.

[0119] In this way, according to this modified example, it is possible to reduce the time and effort required for setting the area variation degree.

[0120] In the second embodiment, the type information of the area is determined by an input from a user, but the system may determine the type information of the area as long as the type information can be uniquely determined. For example, the area name may be determined from map information such as a floor plan and a measurement result of a position and orientation measurement unit such as SLAM.

[0121] Alternatively, the feature vector and the position vector may be input to an input object placement characteristic database, and the output may be input to a place prediction database that predicts a place label, thereby determining the area type information. Here, the place prediction database will be described. The place prediction database connects a 1-input 1-output fully connected layer of a neural network to the beginning of the object placement characteristic database, and outputs the area name. In this way, recognition can be performed by taking into account information on multiple objects rather than simply recognizing an object, so that the area type can be recognized with higher accuracy.

[0122] <Variation 2-4> In the above description, the position and orientation measurement control unit 207 uniformly determines whether to use or not use and the priority order of the feature groups in the pixel area that the target object occupies in the image in the position and orientation measurement process. However, the determination may be made according to the position of the feature, taking into account the shape, position, and orientation of the target object or the object in the area.

[0123] Only the differences from the modified example 1-4 will be described regarding the processing content. In this modified example, a table that can uniquely determine an address to 3D model data of an object in a region using region type information as a key is read in the initialization process of step S401. In step S407, the position and orientation measurement control unit 207 determines whether or not to use a feature, as in modified example 1-5.

[0124] Alternatively, the position and orientation measurement control unit 207 may obtain information on the placement of objects within a region using the task database described in variant example 2-3, which estimates general objects in the region and their position information when region type information is input.

[0125] In this way, according to this modification, by using a three-dimensional model to select features within a pixel region, it is possible to eliminate only a minimum number of features, thereby improving the accuracy of position and orientation measurement.

[0126] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0127] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0128] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An information processing device that performs position and orientation measurement using sensor data obtained by a sensor, a first characteristic acquisition means for acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a nearby object determination means for determining a nearby object interfering with the object whose object fluctuation characteristics have been acquired; a second characteristic acquisition means for acquiring, for each of the nearby objects, an object variation factor characteristic that indicates a characteristic that affects accuracy of the position and orientation measurement by interfering with other objects; a position and orientation measurement control means for controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the object change factor characteristics; An information processing device comprising: (Configuration 2) The object change characteristic is information that indicates the presence or absence of a function that causes a change in the movement, rotation, or appearance of an object, or the likelihood that a change in the movement, rotation, or appearance of an object will occur, The object change factor characteristic is information that indicates whether or not the object has a function of moving, rotating, or changing the appearance of another object, or a likelihood that the object will move, rotate, or change the appearance of another object; 2. The information processing device according to configuration 1, (Configuration 3) an object fluctuation characteristic evaluation means for evaluating the object fluctuation characteristic, The object movement characteristic evaluation means evaluates the object movement characteristic based on an object arrangement characteristic database that stores object type information and three-dimensional positional relationships of objects in space in association with each other. 3. The information processing device according to configuration 1 or 2. (Configuration 4) an object variation factor characteristic evaluation means for evaluating the object variation factor characteristic, The object change factor characteristic evaluation means evaluates the object change factor characteristic based on an object arrangement characteristic database that stores object type information and a three-dimensional positional relationship of the object in space in association with each other. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) An information processing device that performs position and orientation measurement using sensor data obtained by a sensor, a first characteristic acquisition means for acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a region determining means for determining a region in which the object having the object fluctuation characteristics is present; an area characteristic acquisition means for acquiring area variation characteristics that indicate characteristics that affect accuracy of the position and orientation measurement of the area; a position and orientation measurement control means for controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the area change characteristics; An information processing device comprising: (Configuration 6) The area change characteristic is information that indicates the presence or absence of a function that causes an object in the area to move, rotate, or change in appearance, or the likelihood that an object in the area will move, rotate, or change in appearance. 6. The information processing device according to configuration 5. (Configuration 7) a region variation characteristic evaluation means for evaluating the region variation characteristic, The area change characteristic evaluation means evaluates the area change characteristic based on an object arrangement characteristic database that stores object type information and three-dimensional positional relationships of objects in space in association with each other. 7. The information processing device according to configuration 6. (Method 1) A method for controlling an information processing device that performs position and orientation measurement using sensor data from a sensor, comprising: a first characteristic acquisition step of acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a nearby object determination step of determining a nearby object that interferes with the object whose object fluctuation characteristics have been acquired; a second characteristic acquisition step of acquiring, for each of the nearby objects, an object variation factor characteristic that indicates a characteristic that affects accuracy of the position and orientation measurement by interfering with other objects; a position and orientation measurement control step of controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the object change factor characteristics; A control method comprising: (Method 2) A method for controlling an information processing device that performs position and orientation measurement using sensor data from a sensor, comprising: a first characteristic acquisition step of acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a region determining step of determining a region in which the object whose object fluctuation characteristics have been acquired exists; a region characteristic acquisition step of acquiring a region variation characteristic indicating a characteristic that affects accuracy of the position and orientation measurement of the region; a position and orientation measurement control step of controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the area change characteristics; A control method comprising: (Program 1) A program for causing a computer to function as each of the means according to any one of configurations 1 to 7. [Explanation of symbols]

[0129] 201 Image Acquisition Unit 202 Object recognition section 203 Image Processing Device 208 Position and Orientation Measurement Unit

Claims

1. An information processing device that performs position and orientation measurement using sensor data obtained by a sensor, a first characteristic acquisition means for acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a nearby object determination means for determining a nearby object interfering with the object whose object fluctuation characteristics have been acquired; a second characteristic acquisition means for acquiring, for each of the nearby objects, an object variation factor characteristic that indicates a characteristic that affects accuracy of the position and orientation measurement by interfering with other objects; a position and orientation measurement control means for controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the object change factor characteristics; An information processing device comprising:

2. The object change characteristic is information that indicates the presence or absence of a function that causes a change in the movement, rotation, or appearance of an object, or the likelihood that a change in the movement, rotation, or appearance of an object will occur, The object change factor characteristic is information that indicates whether or not the object has a function of moving, rotating, or changing the appearance of another object, or a likelihood that the object will move, rotate, or change the appearance of another object; 2. The information processing apparatus according to claim 1,

3. an object fluctuation characteristic evaluation means for evaluating the object fluctuation characteristic, The object movement characteristic evaluation means evaluates the object movement characteristic based on an object arrangement characteristic database that stores object type information and three-dimensional positional relationships of objects in space in association with each other.

2. The information processing apparatus according to claim 1,

4. an object variation factor characteristic evaluation means for evaluating the object variation factor characteristic, The object change factor characteristic evaluation means evaluates the object change factor characteristic based on an object arrangement characteristic database that stores object type information and a three-dimensional positional relationship of the object in space in association with each other.

2. The information processing apparatus according to claim 1,

5. An information processing device that performs position and orientation measurement using sensor data obtained by a sensor, a first characteristic acquisition means for acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a region determining means for determining a region in which the object having the object fluctuation characteristics is present; an area characteristic acquisition means for acquiring area variation characteristics that indicate characteristics that affect accuracy of the position and orientation measurement of the area; a position and orientation measurement control means for controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the area change characteristics; An information processing device comprising:

6. The area change characteristic is information that indicates the presence or absence of a function that causes an object in the area to move, rotate, or change in appearance, or the likelihood that an object in the area will move, rotate, or change in appearance.

6. The information processing apparatus according to claim 5,

7. a region variation characteristic evaluation means for evaluating the region variation characteristic, The area change characteristic evaluation means evaluates the area change characteristic based on an object arrangement characteristic database that stores object type information and three-dimensional positional relationships of objects in space in association with each other.

7. The information processing apparatus according to claim 6,

8. A method for controlling an information processing device that performs position and orientation measurement using sensor data from a sensor, comprising: a first characteristic acquisition step of acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a nearby object determination step of determining a nearby object that interferes with the object whose object fluctuation characteristics have been acquired; a second characteristic acquisition step of acquiring, for each of the nearby objects, an object variation factor characteristic that indicates a characteristic that affects accuracy of the position and orientation measurement by interfering with other objects; a position and orientation measurement control step of controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the object change factor characteristics; A control method comprising:

9. A method for controlling an information processing device that performs position and orientation measurement using sensor data from a sensor, comprising: a first characteristic acquisition step of acquiring an object variation characteristic indicating a characteristic that affects the accuracy of the position and orientation measurement for each object within a measurement range of the sensor; a region determining step of determining a region in which the object having the object fluctuation characteristics is present; a region characteristic acquisition step of acquiring a region variation characteristic indicating a characteristic that affects accuracy of the position and orientation measurement of the region; a position and orientation measurement control step of controlling the position and orientation measurement so as to limit use of information of an object within a measurement range of the sensor, the information being estimated to have a change, based on the object change characteristics and the area change characteristics; A control method comprising:

10. A program for causing a computer to function as each of the means according to any one of claims 1 to 7.

Images