Information processing apparatus, control method and program

The information processing device improves the accuracy of position and orientation measurement for autonomous mobile bodies by adjusting sensor settings and glass transparency to counteract the effects of glass, enhancing navigation and obstacle detection.

JP2024167609AInactive Publication Date: 2024-12-04CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023083807
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-12-04
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing methods for measuring the position and orientation of autonomous mobile bodies, such as AGVs and AMRs, are compromised by the presence of glass due to light refraction and reflection, leading to reduced accuracy in obstacle detection and measurement.

Method used

An information processing device that acquires sensor and material information to adjust the contribution rate of feature points and control the sensor or environment to minimize the impact of glass on measurement accuracy, including changing the sensing range or glass transparency.

Benefits of technology

Enhances the accuracy of position and orientation measurement by reducing the influence of glass on the measurement process, ensuring precise navigation and obstacle detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167609000001_ABST
    Figure 2024167609000001_ABST
Patent Text Reader

Abstract

To reduce deterioration of measurement accuracy of position and posture of a moving body.SOLUTION: An information processing apparatus includes: detection means for detecting the position of a sensor disposed on a moving body; material information detection means for detecting object information including material information of an object around the moving body based on the position of the sensor; and control means for controlling at least one of the moving body, the sensor of the moving body, the object, and a method for measuring the position or attitude of the moving body based on the object information detected by the material information detection means.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a control technique for an autonomous moving object. [Background technology]

[0002] In recent years, mobile objects that operate in environments such as factories and logistics warehouses, called automated guided vehicles (e.g., AGVs) and autonomous mobile robots (e.g., AMRs), have been attracting attention. AGV is an abbreviation for Automated Guided Vehicle. AMR is an abbreviation for autonomous mobile robot. Known mobile objects include those that measure their own position and orientation (hereafter referred to as "position and orientation") while traveling, and those that detect nearby equipment or people as obstacles on the traveling path of the mobile object while traveling (hereafter referred to as "obstacle detection").

[0003] Incidentally, position and orientation measurement and obstacle detection are performed based on, for example, surrounding images captured by a moving object, but depending on the material of the surrounding objects, the measurement accuracy of position and orientation and obstacle detection may be reduced or may not be possible. For example, glass may cause the position of an object to deviate from its actual position due to refraction of light, or the reflected image reflected on glass may look different depending on the time of day, which may affect the position and orientation measurement results. In addition, glass and mirrors are imaged through reflected images and objects located behind them, so the glass and mirror themselves are not measured and cannot be detected as obstacles.

[0004] In contrast, the autonomous mobile robot in Patent Document 1 detects surrounding glass by determining that any measurement target for which the measurement results of the distance measuring sensor are unstable is glass, and sets the position information of the glass on a map used for position and orientation measurement. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2018-185768 A Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, there have been studies to improve the measurement accuracy of the position and orientation of a moving object even when there is glass around the moving object. However, since they did not pay close attention to the material and shape of the object, the measurement accuracy of the position and orientation of the moving object sometimes decreased.

[0007] An object of the present invention is to reduce degradation in measurement accuracy of the position and orientation of a moving object. [Means for solving the problem]

[0008] An information processing device according to one embodiment of the present invention has a sensor information acquisition means for acquiring the position of a sensor placed on a moving body, a material information acquisition means for acquiring object information including material information of objects around the moving body based on the position of the sensor, and a control means for controlling at least one of the measurement methods of the moving body, the sensor of the moving body, the object, and the position or attitude of the moving body based on the object information acquired by the material information acquisition means. Effect of the Invention

[0009] According to the present invention, it is possible to reduce the decrease in measurement accuracy of the position and orientation of a moving object. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing an example of a usage scene of an information processing device according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a module configuration of a system according to a first embodiment of the present invention. [Diagram 3] 3 is a block diagram showing a hardware configuration of an information processing device 202 shown in FIG. 2. [Figure 4] 3 is a flowchart showing the overall flow of processing executed by the information processing device 202 shown in FIG. 2. [Diagram 5]3 is a flowchart showing a process of controlling change of a contribution rate of a feature of a feature point of an object in the information processing device 202 shown in FIG. 2 in the first embodiment of the present invention. [Figure 6] 10 is a flowchart showing a process of controlling a moving object in the information processing device 202 shown in FIG. 2 in the second embodiment of the present invention. [Figure 7] 13 is a flowchart showing a process of performing object control in the information processing device 202 shown in FIG. 2 in the third embodiment of the present invention. [Figure 8] FIG. 13 is a diagram showing a display screen as an example of a notification means according to the fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims, and all of the combinations of features described in the embodiments are not necessarily essential to the solution of the invention.

[0012] [First embodiment] <Summary> First, an overview of an environment in which an information processing device according to a first embodiment of the present invention is used will be described. In this embodiment, a case in which a moving body autonomously travels based on a map in an environment where a glass object is present will be described, in which the information processing device according to the first embodiment of the present invention is applied.

[0013] The map used by the moving body is a position and orientation measurement map that measures at least one of the position and orientation (hereinafter referred to as "position and orientation") of the moving body itself using, for example, Visual SLAM technology. SLAM is an abbreviation for Simultaneous Localization and Mapping. Hereinafter, the position and orientation measurement map will also be simply referred to as the map. In this map, the visual feature points of the surrounding objects measured by the moving body are recorded as natural feature points of the map data and their features (quantities) (hereinafter referred to as "feature points" and "feature (quantity)"). A feature point is, for example, a point where the brightness gradient value is large in both the x and y directions. The feature obtained from the local region centered on that feature point is considered to be the "feature (quantity)." SIFT features, SURF features, ORB features, etc. are used.

[0014] While the moving body is traveling, the information processing device acquires the position and orientation of the moving body. Then, based on the position and orientation of the moving body, the information processing device acquires glass information, which is information on glass that exists near the moving body, from an object information database (for example, BIM). BIM is an abbreviation for Building Information Modeling. It has been recorded.

[0015] The information processing device refers to information on the material and the position and orientation of the acquired glass information. The glass information includes material information that is information on the material of the glass. The information processing device estimates a shift amount as an index of the degree of deterioration in the accuracy of the position and orientation measurement by the moving body due to the material of the glass. This shift amount is a spatial deviation amount caused by the displacement of the position of a feature point of an object located behind the glass, which is measured by the moving body, due to the material of the glass.

[0016] The information processing device performs position and orientation measurement using the feature points of each object and their features (quantity). The larger the shift amount for an object located behind the glass, the lower the contribution rate of the feature points of the object to the position and orientation measurement. The specific process will be described later.

[0017] <Usage form> 1 is a schematic diagram showing an example of a scene in which an information processing device according to a first embodiment of the present invention is used. A scene 101 shows a moving object 102 according to the present embodiment traveling along a path 105 toward a glass 103.

[0018] An information processing device 202 (see FIG. 2) is disposed on the moving body 102. The information processing device 202 may be one that moves together with the moving body 102, and is not limited to one that is fixedly attached to the moving body 102. A table 104 seen through the glass 103 is reflected on the glass, and is visible from the moving body 102. The table 104 is an example of an object located behind the glass 103.

[0019] At this time, the table 104 seen from the moving body 102 is seen through the glass 103, and therefore appears to be at a position displaced from the actual position due to the refraction and thickness of the glass 103. The moving body 102 measures the feature points and their features (quantities) of the table 104 using an image captured by a built-in image sensor 309 (see FIG. 3) to measure the position and orientation of the moving body 102. The position and orientation, shape, refractive index, thickness, etc. of the glass 103 are held in the object information database 106. The moving body 102 can communicate with the object information database 106 through a wireless communication network. The moving body 102 can acquire information on the position and orientation, shape, refractive index, thickness, etc. of the glass 103 from the object information database 106. The glass 103 can also communicate with an object control device 107. The object control device 107 can electrically change the degree of cloudiness of the surface of the glass 103.

[0020] <Configuration> The configuration of the system according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the module configuration of the system according to the first embodiment of the present invention.

[0021] 2 is disposed on the moving body 102. The information processing system 201 may be one that moves together with the moving body 102, and is not limited to one that is fixedly set on the moving body 102. The information processing system 201 includes an information processing device 202, a sensor control unit 203, and a position and orientation measurement unit 204.

[0022] The information processing device 202 includes a sensor position acquisition unit 205, a material information acquisition unit 206, and a control unit 207. The information processing device 202 wirelessly communicates with the object information database 106. The object information database 106 includes an object information transmission unit 208.

[0023] The sensor position acquisition unit 205 acquires the position and orientation of the moving body 102 by acquiring the position and orientation of the image sensor 309 arranged in the moving body 102. In this embodiment, the sensor position acquisition unit 205 acquires the current position and orientation of the moving body 102 while it is traveling.

[0024] The material information acquisition unit 206 acquires object information including material information of the glass around the mobile object 102 from the object information transmission unit 208 of the object information database 106 based on the position and orientation of the mobile object 102 acquired by the sensor position acquisition unit 205. In this embodiment, the material information acquisition unit 206 acquires the refractive index and thickness of the glass as the material information of the glass. In addition, the material information acquisition unit 206 acquires information including the position, orientation, and shape information of the glass in addition to the material information of the glass as object information.

[0025] The control unit 207 controls a target for reducing a decrease in the accuracy of measuring the position and orientation of the moving object 102, based on the object information acquired by the material information acquisition unit 206. The control of the target in this embodiment refers to control for changing the contribution rate of the feature (amount) of the feature point of the object located behind the glass, which is performed on the position and orientation measurement unit 204. Details of the processing by the control unit 207 will be described later with reference to Figs. 4 and 5.

[0026] <Hardware configuration description> Fig. 3 is a block diagram showing a hardware configuration of an information processing device according to a first embodiment of the present invention. The information processing device 202 of Fig. 2 has the hardware configuration shown in Fig. 3. The information processing device 202 has a CPU 19, a ROM 302, a RAM 303, an external memory 304, a system bus 305, an input unit 306, a display unit 307, an I / O 308, and an image sensor 309. CPU is an abbreviation for Central Processing Unit. ROM is an abbreviation for Read Only Memory. RAM is an abbreviation for Random Access Memory. I / O is an abbreviation for Input / Output.

[0027] The imaging sensor 203 is connected to the I / O 308. The sensor position acquisition unit 205 acquires an image captured by the imaging sensor 203. The imaging sensor 203 may be provided outside the information processing device 202.

[0028] The CPU 301 controls various devices connected to a system bus 305 and controls processing in the information processing device 202. The ROM 302 stores a BIOS program and a boot program. BIOS is an abbreviation for Basic Input Output System. The RAM 303 is used as a main storage device for the CPU 301.

[0029] The external memory 304 stores programs to be processed by the information processing device 202. The input unit 306 is a keyboard and a mouse, and performs processing related to the input of information. The display unit 307 outputs the results of calculations by the information processing device 202 to a display device according to instructions from the CPU 301. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED indicator. LED is an abbreviation for Light Emitting Diode.

[0030] The information processing device 202 may have a communication interface. The communication interface is for performing information communication via a network. The communication interface may be any type, such as Ethernet, USB, serial communication, or wireless communication. USB is an abbreviation for Universal Serial Bus.

[0031] <Processing procedure and detailed processing method> Hereinafter, a processing procedure and a detailed processing method of the information processing device 202 according to this embodiment will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a flowchart showing the overall flow of processing executed by the information processing device 202 shown in Fig. 2.

[0032] In step S401, the sensor position acquisition unit 205 acquires the position and orientation of the moving object 102. In step S402, the material information acquisition unit 206 acquires object information including material information of objects around the moving object 102 from the object information database 106, using the position and orientation of the moving object 102 acquired by the sensor position acquisition unit 205 as an input. In this embodiment, the material information acquisition unit 206 acquires object information including the refractive index, thickness, position, orientation, and shape information of glass as material information.

[0033] In step S403, the control unit 207 controls a target for reducing the decrease in the accuracy of measuring the position and orientation of the moving body 102, based on the object information acquired in step S402. The target in this embodiment refers to control for changing the contribution rate of the feature (amount) of the feature point located behind the glass. Details of this control will be described with reference to FIG. 5.

[0034] Fig. 5 is a flowchart showing a process for changing the contribution rate of the feature (quantity) of the feature point located behind the glass in the information processing device 202 shown in Fig. 2. Fig. 5 is also a flowchart showing the detailed process flow of step S403 in Fig. 4.

[0035] In step S501, the control unit 207 estimates the degree of degradation in the position and orientation measurement accuracy of the moving body 102 caused by the object information acquired by the material information acquisition unit 206, based on the object information. Specifically, using the refractive index and thickness of the material information, the control unit 207 calculates the distance between the light beam to a false feature point whose detection position is shifted due to observation through glass, and the light beam to the position of a feature point with no positional deviation when not observed through glass. The calculated distance is then estimated as a shift amount, which is the degree of degradation in the position and orientation measurement accuracy.

[0036] In step S502, the control unit 207 determines an object area within the sensing range of the sensor placed on the moving body 102 from the object information acquired by the material information acquisition unit 206. Specifically, in this embodiment, the area of ​​the object located behind the glass in the captured image is determined from the position, orientation, and shape of the object. Building materials are an example of an object.

[0037] In step S503, the control unit 207 determines the contribution rate of the feature (quantity) of the feature point located behind the glass according to the degree of deterioration of the position and orientation measurement accuracy determined in step S501. Specifically, the contribution rate is determined so that the contribution rate decreases as the shift amount increases. When the shift amount at that time is p, a predetermined shift amount upper limit p_max, and the contribution rate is m, the contribution rate m is determined as m=1.0-p / p_max, which is the ratio of the shift amount p to the shift amount upper limit p_max. Here, the shift amount upper limit is a threshold value of the shift amount that, if the shift amount exceeds this value, is ignored from the measurement of the moving object (the contribution to the position and orientation measurement is set to 0).

[0038] In step S504, the control unit 207 controls to change the method of measuring the position and orientation of the object region determined in step S502 based on the contribution rate determined in step S503. Specifically, the contribution rate of the feature (amount) of the feature point in the object region is lowered by applying the contribution rate determined in step S503 as the contribution rate of the feature (amount) of the feature point in the object region. The position and orientation measurement unit 204 executes a process of measuring the position and orientation of the moving body 102 by applying this contribution rate. The position and orientation measurement unit 204 may measure the position or orientation of the moving body by preferentially using the feature amount with a high contribution rate. Note that in the present invention, instead of lowering the contribution rate of the feature (amount) of the feature point in the glass region, the contribution rate of the feature (amount) of the feature point outside the glass region may be increased to relatively lower the contribution rate of the feature (amount) of the feature point in the glass region.

[0039] <Effects> As described above, according to this embodiment, it is possible to reduce the degradation of the accuracy of measuring the position and orientation of a moving object caused by objects such as glass around the moving object.

[0040] <Modification of the first embodiment> (Variation 1-1) Variation of position and orientation measurement method In the first embodiment, the position and orientation of the moving body 102 is measured using the Visual SLAM technique, but other methods may be used as long as they can measure the position and orientation of the moving body. For example, the LiDAR SLAM method or the Depth SLAM method may be used. Lidar is an abbreviation for Light Detection and Ranging. This makes it possible to reduce the decrease in the position and orientation measurement accuracy as described in the first embodiment, regardless of the method for measuring the position and orientation of the moving body.

[0041] (Variation 1-2) Variation of glass material information The material information in the first embodiment is the refractive index and thickness of the glass, but other material information may be used as long as it is material information that is a factor in reducing the position and orientation measurement accuracy of the moving object. For example, it may be the transparency or cloudiness of the glass. For example, the amount of blur of the image measured by the moving object may be estimated based on the transparency or cloudiness of the glass acquired from the object information database 106, and the greater the amount of blur, the lower the contribution rate of the feature (quantity) of the feature point located behind the glass. In this way, regardless of the specific material information of the glass around the moving object, as long as it is material information of the glass that is a factor in reducing the position and orientation measurement accuracy of the moving object, it is possible to reduce the reduction in the position and orientation measurement accuracy as described in the first embodiment.

[0042] (Variation 1-3) Variation of feature points with different contribution rates In the first embodiment, the control device 202 changes the contribution rate of feature points located behind the glass. However, other features (quantities) may be used as long as they are features (quantities) that reduce the accuracy of measuring the position and orientation of a moving object. For example, if there is no person or obstacle between the moving object and the glass, it is acceptable to simply change the contribution rate of the features (quantities) of all feature points that exist in the direction of the glass within the sensing range. This makes it possible to reduce the reduction in accuracy of measuring the position and orientation as described in the first embodiment, without being limited to the positions of feature points whose contribution rates are changed.

[0043] (Variation 1-4) Variations in object types Although the object in the first embodiment is glass, it may be any other object as long as it can determine an index of the degree of deterioration of the position and orientation measurement accuracy of the moving object. For example, it may be a mirror, a glossy floor or a wall. In the case of a mirror, the contribution rate of the feature (quantity) of the feature point in the mirror direction may be reduced based on the position and orientation and shape of the mirror acquired from the object information database 106. This also applies to the reflection of glass. In the case of a glossy floor or wall, the contribution rate of the glossy wall or floor direction may be reduced as the glossiness increases based on the glossiness of the floor or wall acquired from the object information database 106. This makes it possible to reduce the deterioration of the position and orientation measurement accuracy as described in the first embodiment, even if there is an object other than glass around the moving object and the object deteriorates the position and orientation measurement of the moving object.

[0044] Furthermore, in order to reduce the deterioration of the position and orientation measurement accuracy due to the reflection of the glass, the control unit 207 in the first embodiment performs control to change the contribution rate of the feature (quantity) of the feature points of the entire region of the target glass. If the location that affects the position and orientation measurement can be specified in detail, control to change the contribution rate of the feature (quantity) of the feature points of a specific region of the object to be controlled may be performed. For example, the position of the reflected image of the glass may be specified based on the position and orientation and shape of the glass acquired from the object information database 106 and the position and orientation of other objects, and the contribution rate of only the feature (quantity) of the feature points in the region of the reflected image in the object region may be changed. This allows the region of the object that affects the position and orientation to be specified in detail, and the influence of the object on the position and orientation measurement result of only that region can be reliably reduced.

[0045] (Variation 1-5) Object information taking into account the measurement characteristics of the sensor The control unit 207 in the first embodiment prevents the position and orientation measurement accuracy of the moving body caused by the object from decreasing based on the object information. The present invention may further control the contribution of the feature (quantity) of the feature point by taking into consideration the measurement characteristics of the sensor. For example, a case will be described in which the sensor arranged on the moving body is a distance measurement sensor that measures distance using the reflection of light irradiated from a light source, and the color information of the glass acquired from the object information database 106 is "red". In this case, if the sensor light of the moving body is also reddish, measurement cannot be performed before the feature point shifts. For this reason, in addition to the contents described in the first embodiment, the contribution rate may be further changed according to the luminance value of the reddish color, and the control may be changed so that the contribution rate of the feature (quantity) of the feature point located behind the glass is further reduced as the luminance value of the reddish color increases. On the other hand, if the glass is reddish and the sensor is greenish, the contribution rate may be changed in the same way as in the first embodiment. This makes it possible to reduce the decrease in the position and orientation measurement accuracy even when the degree of decrease in the position and orientation measurement accuracy changes depending on the sensor characteristics.

[0046] (Modification 1-6) Method for determining index of degree of deterioration in position and orientation measurement accuracy In the first embodiment, the index of the degree of deterioration of the position and orientation measurement accuracy is calculated and determined based on the material information. However, as long as an index of the degree of deterioration of the position and orientation measurement accuracy related to the material information of the object can be obtained, it may be determined by other methods. For example, it may be determined based on the material information obtained by the material information obtaining unit 206 from a table that holds a group of predetermined index candidates.

[0047] (Variation 1-7) Location of information processing device In the first embodiment, the information processing device 202 is installed inside the moving body 102, but the present invention is not limited to this. The information processing device 202 does not have to be installed inside the moving body 102 as long as it acquires sensor information and object information of the moving body 102 and selects and controls a control method.

[0048] For example, the information processing device 202 may be configured to be on a cloud, a server, or a tablet terminal, and to cooperate with the moving object 102 via wireless communication. This allows for centralized management of measurement results of multiple moving objects, even if the information processing device 202 is inside the moving object, on a cloud, a server, or a tablet terminal. The present invention can also be applied to an environment in which wireless communication between the information processing device 202 and the moving object 102 is not required, and can reduce the influence of the reflection characteristics of the object on the position and orientation measurement results. In the first embodiment, visual feature points of surrounding objects measured by the moving body are used to estimate the position and orientation of the moving body itself, but this is not limited to this as long as it is a feature of the surrounding objects that can estimate the position and orientation of the moving body. For example, an edge of an image showing a surrounding object may be used. This makes it possible to reduce the decrease in the position and orientation measurement accuracy as described in the first embodiment, regardless of the form of the feature (quantity) used by the moving body to measure the position and orientation.

[0049] [Second embodiment] <Summary> In the first embodiment, when glass is present around a moving object, the contribution rate of the feature (quantity) of the feature point located behind the glass used in measuring the position and orientation of the moving object is changed. This reduces the degradation of the position and orientation measurement accuracy caused by the glass. In the second embodiment, the sensing range of the sensor of the moving object is changed to reduce the degradation of the position and orientation measurement accuracy caused by the glass.

[0050] As in the first embodiment, the information processing device in the second embodiment estimates a shift amount from the position and orientation of the glass and material information acquired from the object information database as an index of the degree of degradation of the position and orientation measurement accuracy of the moving object caused by the material information of the glass. Then, the larger the shift amount, the more the pan-tilt control of the image sensor of the moving object is controlled to increase the displacement amount of the sensor so that the sensor is not directed toward the glass, thereby reducing the degradation of the position and orientation measurement accuracy caused by the glass. This control is an example of controlling the sensor control unit 203 according to the degree of degradation of the position and orientation measurement accuracy of the moving object caused by the material information of the surrounding objects.

[0051] <Usage form> The usage scene of the information processing device according to the second embodiment is similar to that in FIG. 1, and therefore the description thereof will be omitted.

[0052] <Configuration> The configuration of the information processing system according to the second embodiment will be described with reference to Fig. 2. Note that the sensor position acquisition unit 205 and the material information acquisition unit 206 in the information processing device 202 according to the second embodiment are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0053] The control unit 207 controls a target for reducing a decrease in accuracy of measuring the position and orientation of a moving body, based on the material information of the object acquired by the material information acquisition unit 206. The control of the target in this embodiment refers to change control of the sensing range by pan-tilt control of the moving body sensor implemented by the sensor control unit 203.

[0054] <Hardware configuration description> The hardware configuration of the information processing device 202 according to the second embodiment is similar to that shown in FIG. 3, and therefore will not be described.

[0055] <Processing procedure and detailed processing method> The overall process flow of the information processing device 202 according to the second embodiment is the same as that of the first embodiment, as shown in the flowchart in Fig. 4. The process contents of steps S401 and S402 are the same as those of the first embodiment, and therefore will not be described.

[0056] In the second embodiment, in step S403, the control unit 207 controls the target of reducing the decrease in the accuracy of measuring the position and orientation of the moving body based on the object information of the glass acquired in step S402. The target in this embodiment is the change control of the sensing range by the pan-tilt control of the moving body sensor. The details of this control will be described with reference to FIG. 6.

[0057] Fig. 6 is a flowchart showing a process of changing the sensing range of the sensor of the mobile body in the information processing device 202 shown in Fig. 2. Fig. 6 is also a flowchart showing a detailed process flow of step S403 in Fig. 4. The processes of steps S501 and S502 in the second embodiment are similar to those in the first embodiment, and therefore description thereof will be omitted.

[0058] In step S601, the control unit 207 determines the pan / tilt change amount for changing the sensing range of the sensor of the moving object 102 by treating the shift amount determined in step S501 as the degree of displacement of the sensor. Specifically, the horizontal angle of view of the glass and the angles of view at both ends excluding the horizontal angle of view are calculated from the sensor angle of view and the glass area size in the image according to the shift amount. If the angle of view of the glass is Δg and the smaller angle of the other angles of view is Δe_s, then (Δg+Δe_s) is determined as the pan / tilt angle at which the glass does not fall within the sensing range.

[0059] In step S602, the control unit 207 controls the sensor control unit 203 to pan and tilt the moving object, using the displacement angle of the sensor of the moving object 102 determined in step S601 as a control parameter.

[0060] <Effects> As described above, according to this embodiment, even if there is an object around a traveling mobile object that affects the position and orientation measurement of the mobile object, it is possible to reduce a decrease in the position and orientation measurement accuracy caused by the object.

[0061] <Modification of the second embodiment> (Variation 2-1) Variation of the moving object sensor control In the second embodiment, the control unit 207 changes the pan-tilt control of the sensor of the moving body, but other sensor control methods may be used as long as they can reduce the decrease in accuracy of measuring the position and orientation of the moving body caused by an object. For example, the camera of the moving body may be zoomed out. Alternatively, the brightness parameters during imaging may be adjusted. For example, the gain may be lowered depending on the shift amount to make it difficult to measure the feature points in the glass direction. In the second embodiment, at least one of the position and orientation of the moving body, or the measurement range of the sensor of the moving body is controlled so that the object does not enter the sensing range of the moving body.

[0062] In addition, if the sensor of the moving object is a distance measuring sensor that measures distance using the reflection of light irradiated from a light source, the frequency range to be detected may be changed or the frequency of the projected light may be increased or decreased according to the shift amount. In this case, the distance measurement method or the projection timing may be changed.

[0063] For example, when changing the frequency range to be detected, the control may be performed so that the greater the shift amount, the lower the reflected light frequency that is less susceptible to the effect of optical refraction is detected.

[0064] Furthermore, for example, when changing the frequency of the light to be projected, it is possible to control so that the larger the shift amount is, the lower the frequency of the light to be projected, which is less susceptible to the effects of light refraction.

[0065] In addition, when changing the distance measurement method, if the shift amount becomes larger than a predetermined threshold value, the distance measurement method may be switched from a method using a random dot pattern as the projection pattern to any known method, such as a method using a pattern of multiple dashed lines.

[0066] In addition, when changing the projection timing, if the shift amount is large, the projection timing may be changed so that the projection is performed with a positional relationship between the moving object and the glass such that the glass area occupies a smaller portion of the sensing range. This makes it possible to reduce the degradation of the position and orientation measurement accuracy caused by the surrounding objects described in the second embodiment, regardless of the sensor control method described in this embodiment.

[0067] (Modification 2-2) Method for determining sensor displacement amount In the second embodiment, the sensing range change parameter is calculated from the shift amount, which is an index of the degree of deterioration of the position and orientation measurement accuracy of the moving object, and is determined. Other parameter determination methods may be used as long as the sensing range can be changed so as not to face the direction of the object according to the shift amount. For example, the control unit 207 may determine parameters in advance that position the object outside the sensing range, and may determine the parameter according to the shift amount or whether the value of the shift amount is equal to or greater than a threshold value. This makes it possible to reduce deterioration of the position and orientation measurement accuracy caused by the surrounding objects described in the second embodiment, regardless of the parameter determination method for changing the sensing range of the moving object.

[0068] (Variation 2-3) Variation of the Controlled Object In the second embodiment, the control unit 207 controls the sensor of the moving body in the form of pan-tilt control, but other control contents may be used as long as the control contents reduce the deterioration of the position and orientation measurement accuracy of the moving body. For example, the traveling position or orientation of the moving body may be changed according to the shift amount. This makes it possible to reduce the deterioration of the position and orientation measurement accuracy caused by the surrounding objects described in the second embodiment, regardless of whether the control contents of the moving body are the control of the sensor or the control of the moving body itself.

[0069] [Third embodiment] <Summary> In the first and second embodiments, when glass is present around a moving object, the contribution rate of the feature (quantity) of a feature point located behind the glass used in measuring the position and orientation of the moving object is changed, or the sensing range of a sensor placed on the moving object is changed. This reduces a decrease in the accuracy of measuring the position and orientation caused by the glass. In the third embodiment, when the state of the glass can be changed by an external input, the moving object changes the operating state of the glass, thereby reducing a decrease in the accuracy of measuring the position and orientation caused by the glass. The glass is, for example, a glass object to which an instantaneous light control film that electrically switches the transmittance is attached.

[0070] In the third embodiment, similarly to the first and second embodiments, a shift amount is estimated from the position and orientation and material information of the glass acquired from the object information database 106 as an index of the degree of degradation of the position and orientation measurement accuracy of the moving body caused by the material information of the glass. Then, if the shift amount is present, the moving body performs control to change the transmittance of the glass so that the moving body cannot measure the features (quantity) of the feature points of the object located behind the shifted glass. By controlling the surrounding objects in this manner, degradation of the position and orientation measurement accuracy caused by the glass is reduced. This control is an example of controlling the object control unit 209 according to the degree of degradation of the position and orientation measurement accuracy of the moving body caused by the material information of the surrounding objects.

[0071] <Usage form> The scene in which the information processing device according to the third embodiment is used is the same as that shown in FIG.

[0072] <Configuration> The configuration of an information processing system according to the third embodiment will be described with reference to Fig. 2. Note that the sensor position acquisition unit 205 and the material information acquisition unit 206 in the information processing device 202 according to the third embodiment are similar to those in the first and second embodiments, and therefore description thereof will be omitted.

[0073] The control unit 207 controls the target for reducing the decrease in accuracy of measuring the position and orientation of the moving object, based on the object information acquired by the material information acquisition unit 206. The control of the target in this embodiment is the control of changing the transparency of the glass, which is performed by the object control unit 209. Details of the processing by the control unit 207 will be described later with reference to Figs. 4 and 7.

[0074] <Hardware configuration description> The hardware configuration of the information processing device 202 according to the third embodiment is the same as that shown in FIG.

[0075] <Processing procedure and detailed processing method> The overall process flow of the information processing device 202 according to the third embodiment is the same as that of the first and second embodiments, as shown in the flowchart of Fig. 4. The process contents of each step of steps S401 and S402 are the same as those of the first and second embodiments, and therefore will not be described.

[0076] In the third embodiment, in step S403, the control unit 207 controls the target of reducing the decrease in accuracy of measuring the position and orientation of the moving object based on the object information of the glass acquired in step S402. The target in this embodiment is the control of changing the transparency of the glass. Details of this control will be described with reference to FIG.

[0077] Fig. 7 is a flowchart showing a process for changing the transparency of glass in the information processing device 202 shown in Fig. 2. Fig. 7 is also a flowchart showing a detailed process flow of step S403 in Fig. 4. Note that step S501 is similar to the first and second embodiments, and therefore a description thereof will be omitted.

[0078] In step S701, the control unit 207 determines a control parameter for the amount of change in glass transparency according to the degree of deterioration in the position and orientation measurement accuracy determined in step S501. Specifically, the control parameter is determined so that the higher the current transparency is, the more the control parameter for changing transparency is increased to make it less transparent. When the transparency in the cloudy state of the glass is t0, the current transparency is t1, and the maximum amount of change in transparency is ΔD, the control parameter for changing transparency is determined as (t1-t0) / ΔD. Here, the maximum amount of change in transparency is the range of parameters that can be changed when electrically changing the transparency of the glass.

[0079] In step S702, the control unit 207 controls the object control unit 209 to instruct the object control unit 209 to perform processing for changing the transparency of the glass by applying the object control parameters determined in step S701.

[0080] <Effects> As described above, according to this embodiment, even if there is an object around a traveling moving body that reduces the accuracy of measuring the position and orientation of the moving body, the reduction in the accuracy of measuring the position and orientation of the moving body can be mitigated by controlling the object.

[0081] <Modification of the third embodiment> (Variation 3-1) Variations in Control Object and Its Material Information In the third embodiment, the object to be controlled is glass that changes its transparency, but other objects may be used as long as the object can be controlled from outside the object. For example, a movable partition, shutter, or door (hereinafter referred to as a "movable object") that controls the operating position may be used. For example, a movable object operating distance may be obtained as material information from the object information database 106, and the operating distance may be treated as a shift amount to be measured in the position and orientation of the moving body, and the position of the object may be controlled using the shift amount as a control parameter. This makes it possible to reduce the decrease in position and orientation measurement accuracy caused by the object as described in the third embodiment, regardless of the object to be controlled or its material information.

[0082] (Modification 3-2) Variation in Method for Determining Object Control Parameters In the third embodiment, the control parameters of the object are determined by calculation using the glass transparency according to the shift amount, but the parameters may be determined by other methods as long as the control parameters of the object can be determined. For example, a pattern of object control parameters may be determined by selecting a control parameter from among the determined patterns according to the shift amount. This makes it possible to reduce the deterioration of the position and orientation measurement accuracy caused by the object as described in the third embodiment, regardless of the specific method of determining the control parameters of the peripheral objects based on the index of the degree of deterioration of the position and orientation measurement accuracy.

[0083] (Variation 3-3) Variations in the installation location of the object control device In the third embodiment, the moving body communicates with an externally installed object control device 107 to issue control instructions. This does not matter if the object can be controlled from the moving body, the object control device 107 may be installed in another location. For example, the object control device 107 may be installed inside the moving body. This makes it possible to reduce the influence of the object described in the third embodiment on the position and orientation measurement results, regardless of the installation location of the object control device 107.

[0084] [Fourth embodiment] <Summary> In the fourth embodiment, a method for notifying a moving object monitor or a facility monitor of the contents for reducing the deterioration of the position and orientation measurement accuracy caused by an object according to the first, second, and third embodiments will be described. Note that in this embodiment, a method for notifying the contents of the countermeasure method will be described based on the first embodiment.

[0085] <Usage form> The scene in which the information processing device according to the fourth embodiment is used is the same as that shown in FIG.

[0086] <Configuration> The configuration of an information processing system according to a fourth embodiment will be described with reference to Fig. 2 and Fig. 8. An information processing system 201 according to the fourth embodiment newly includes a notification unit in addition to the configuration shown in Fig. 2. Note that other configurations are similar to those of the other embodiments, and therefore detailed description thereof will be omitted.

[0087] The notification unit notifies information about the object and the control content to be performed by the control unit 207 based on various results from at least one of the sensor position acquisition unit 205, the material information acquisition unit 206, and the control unit 207.

[0088] 8 is a diagram showing a display screen as an example of a notification unit according to the fourth embodiment. A display screen 801 notifies by displaying control contents for reducing a decrease in accuracy of position and orientation measurement caused by an object.

[0089] The display screen 801 has an inherent information display section 802. The inherent information display section 802 displays inherent information of the moving object. The display screen 801 has an inherent information display section 803. The inherent information display section 803 displays inherent information of the object information database 106 from which object information is obtained.

[0090] The display screen 801 has an object information display section 804. The object information display section 804 displays the position information of an object acquired from the object information database 106, which is mapped on a map, as well as the position and orientation information and material information of the object acquired from the object information database 106.

[0091] The display screen 801 has a control result display section 805. The control result display section 805 displays the contents of measures taken by the moving body 102 to improve the accuracy of object-related position and orientation measurement. In the display example of Fig. 8, the glass area is masked, and it is shown that the masking has reduced the contribution rate of the features (quantity) of the feature points in that area.

[0092] <Hardware configuration description> The hardware configuration of the information processing device 202 according to the fourth embodiment is similar to that shown in FIG. 3, and therefore will not be described.

[0093] <Effects> As described above, according to this embodiment, even if there is an object around a traveling moving object that affects the position and orientation measurement of the moving object, it is possible to notify a moving object monitor or equipment monitor of information that reduces the degradation of the object's position and orientation measurement accuracy caused by the reflection characteristics of the object.

[0094] <Modification of the Fourth Embodiment> (Variation 4-1) Variation of notification media In the fourth embodiment, the notification of the countermeasures for reducing the deterioration of the position and orientation measurement accuracy caused by an object is in GUI format. GUI is an abbreviation for Graphical User Interface. This is not limited to this as long as the countermeasures implemented by the moving object for reducing the deterioration of the position and orientation measurement accuracy caused by an object can be notified to the moving object monitor or the facility monitor. For example, the content may be notified in text format. This makes it possible to notify the countermeasures for reducing the influence on the position and orientation measurement result caused by the object described in the fourth embodiment, regardless of the GUI display support of the display unit or the hardware for CUI display. CUI is an abbreviation for Character User Interface.

[0095] The present invention may be such that at least one of the controls described in each of the above-described embodiments is performed, or may be such that all of the controls are performed.

[0096] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0097] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0098] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An acquisition means for acquiring a position of a sensor disposed on a moving object; a material information acquiring means for acquiring object information including material information of objects around the moving object based on a position of the sensor; a control means for controlling at least one of the moving body, a sensor of the moving body, the object, and a method for measuring a position or an attitude of the moving body based on the object information acquired by the material information acquisition means; An information processing device having the above configuration. (Configuration 2) the control means estimates a spatial shift amount of the feature of the object measured by the sensor, which has been displaced due to the object information, as an index of a degree of deterioration in measurement accuracy of the position or orientation of the moving body; Controlling at least one of a method for measuring the position or orientation of the moving body, a sensor of the moving body, the object, and the moving body based on the index. 2. The information processing device according to configuration 1. (Configuration 3) the control means determines a contribution rate of a feature of the feature point of the object based on the index, and controls a method for measuring the position or orientation of the moving body by preferentially using the feature having a high contribution rate. 3. The information processing device according to configuration 1 or 2. (Configuration 4) the control means controls at least one of a position and an attitude of the moving body, or a measurement range of a sensor of the moving body, based on the index so that the object does not enter a sensing range of the moving body. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) the control means controls an operating state of the object based on the index so that the moving body does not measure a portion where the measurement accuracy of the position or orientation of the moving body decreases. 5. The information processing device according to any one of configurations 1 to 4. (Configuration 6) The method further includes a notification means for notifying a result of at least one of the acquisition means, the material information acquisition means, and the control means. 5. The information processing device according to any one of configurations 1 to 4. (Method 1) An acquisition step of acquiring a position of a sensor disposed on a moving object; a material information acquiring step of acquiring object information including material information of objects around the moving object based on the position of the sensor; a control step of controlling at least one of the moving body, a sensor of the moving body, the object, and a method of measuring the position or attitude of the moving body based on the object information acquired in the material information acquisition step; The control method includes: (Program 1) A program for causing a computer to function as each of the means according to any one of configurations 1 to 6. [Explanation of symbols]

[0099] 201 Information Processing Systems 202 Information processing device 203 Sensor control unit 204 Position and Orientation Measurement Unit 205 Sensor position acquisition unit 206 Material Information Acquisition Department 207 Control Unit 208 Object information transmission unit 209 Object Control Unit

Claims

1. An acquisition means for acquiring a position of a sensor disposed on a moving object; a material information acquiring means for acquiring object information including material information of objects around the moving object based on a position of the sensor; a control means for controlling at least one of the moving body, a sensor of the moving body, the object, and a method for measuring a position or an attitude of the moving body based on the object information acquired by the material information acquisition means; An information processing device having the above configuration.

2. the control means estimates a spatial shift amount of the feature of the object measured by the sensor, which is displaced due to the object information, as an index of a degree of deterioration in the measurement accuracy of the position or orientation of the moving body, and controls at least one of the moving body, the sensor of the moving body, the object, and a measurement method of the position or orientation of the moving body based on the index.

2. The information processing apparatus according to claim 1,

3. the control means determines a contribution rate of a feature of the feature point of the object based on the index, and controls a method for measuring the position or orientation of the moving body by preferentially using the feature having a high contribution rate.

2. The information processing apparatus according to claim 1,

4. the control means controls at least one of a position and an attitude of the moving body, or a measurement range of a sensor of the moving body, based on the index so that the object does not enter a sensing range of the moving body.

2. The information processing apparatus according to claim 1,

5. the control means controls an operating state of the object based on the index so that the moving body does not measure a portion where the measurement accuracy of the position or orientation of the moving body decreases.

2. The information processing apparatus according to claim 1,

6. The method further includes a notification unit that notifies a result of at least one of the acquisition unit, the material information acquisition unit, and the control unit.

2. The information processing apparatus according to claim 1,

7. An acquisition step of acquiring a position of a sensor disposed on a moving object; a material information acquiring step of acquiring object information including material information of objects around the moving object based on the position of the sensor; a control step of controlling at least one of the moving body, a sensor of the moving body, the object, and a method of measuring a position or an attitude of the moving body based on the object information acquired in the material information acquisition step; The control method includes:

8. A program for causing a computer to function as each of the means according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Autonomous mobile robot

    JP2018185768A