Information processing device, information processing method, and program

Abstract

To stably move a mobile body.SOLUTION: An information processor for controlling a mobile body including measurement means for measuring the position of the mobile body from a picked-up image of a surrounding environment of the mobile body includes; detection means for detecting an occurrence of flicker in an environment in which the mobile body moves; estimation means for estimating first position information indicating a position at which measurement accuracy of the measurement means changes on the basis of a detection result of the detection means; and determination means for determining contents of the control on the basis of the first position information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Technologies have been proposed for automatically moving a mobile object such as a transport vehicle (e.g., an AGV; Automated Guided Vehicle) within an environment such as a factory or a logistics warehouse. When moving automatically, a map of the real space may be created and the vehicle's own position and orientation within the real space may be measured. A known method for this is the SLAM (Simultaneous Localization and Mapping) method. Non-Patent Document 1 describes a method for estimating the vehicle's own position and orientation (the camera's shooting position and orientation) by comparing a video frame captured by a camera with feature points on map information. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Jakob Engel,et.al.LSD-SLAM:Large-Scale Direct Monocular SLAM,Technical University Munich Summary of the Invention [Problem to be solved by the invention]

[0004] Indoor light sources such as fluorescent lights cause a flicker phenomenon in which the illumination light periodically flickers due to the influence of the power supply frequency. When capturing images under such a flickering light source, the brightness of the image fluctuates between frames. As described in Non-Patent Document 1, when position and orientation measurement is performed using images captured by a camera, the accuracy of position measurement can be reduced due to the brightness fluctuations between frames, which poses a problem in that it is not possible to stably drive a moving object.

[0005] SUMMARY OF THE INVENTION An object of the present invention is to move a moving body stably. [Means for solving the problem]

[0006] The present invention provides The captured images are frame images of a moving image, and feature points are extracted from one of the frame images and the other of the frame images captured at different times, and blinking feature points that are extracted in the one frame image but not in the other frame image are detected, and based on the blinking feature points, In the environment in which the moving object moves Luminance fluctuations due to flicker a detection means for detecting the occurrence of The space in which the moving object moves is divided into a plurality of regions, and the region in which the number of the flickering feature points detected by the detection means is equal to or greater than a threshold is determined as a region in which the flickering feature points are detected by the detection means due to luminance fluctuations caused by the flickering in the space in which the moving object moves. The measurement accuracy of the measuring means changes. As an area an estimation means for estimating; Areas where measurement accuracy changes Based on The speed of the moving object or the moving path of the moving object a determining means for determining When there are a plurality of regions in which the measurement accuracy changes in the space in which the moving body moves, the estimation means estimates an internal region of a polygon based on the plurality of regions in which the measurement accuracy changes as the region in which the measurement accuracy changes. It is characterized by: [Effects of the Invention]

[0007] According to the present invention, it is possible to move a moving body stably. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 10 is a diagram showing an image taken inside a logistics warehouse. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of each device. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 4] 10 is a flowchart illustrating a process executed by the information processing device. [Figure 5] FIG. 10 is a diagram illustrating an example of a GUI screen. [Figure 6] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 7] 10 is a flowchart illustrating a process executed by the information processing device. [Figure 8] FIG. 10 is a diagram illustrating an example of a GUI screen. [Figure 9] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] <Embodiment 1> In this embodiment, an example will be described in which the present invention is applied to the control of a mobile object that transports packages in a logistics warehouse. FIGS. 1(a) and 1(b) show images captured inside the logistics warehouse. The logistics warehouse is equipped with a light source 110 and an exterior window 111 as equipment for creating a lighting environment inside the logistics warehouse. The light source 110 is a lighting device that generates a flicker phenomenon, such as a fluorescent lamp. The exterior window 111 is a window that allows external light to enter. An imaging device 101 is mounted on a mobile object 100 that moves inside the logistics warehouse. In this embodiment, the mobile object 100 measures its own position and orientation based on feature points extracted from frame images of a video captured by the imaging device 101. In FIGS. 1(a) and 1(b), the positions of the extracted feature points are indicated by black circles.

[0011] Here, the luminance fluctuation between frame images that occurs due to the frequency at which the light intensity changes due to the flicker phenomenon of the light source 110 (hereinafter referred to as the flicker frequency) is called flicker-induced luminance fluctuation. Also, a feature point that is extracted in one frame image but not extracted in another frame image is called a flickering feature point. FIGS. 1(a) and (b) show feature points extracted from images captured at different times. In FIG. 1(a), feature points are extracted even around the light source 110, but in FIG. 1(b), it can be seen that the number of feature points around the light source 110 is reduced. In this way, the extracted feature points change depending on the capture time of the frame images. Feature points that are extracted in FIG. 1(a) but not extracted in FIG. 1(b) are flickering feature points.

[0012] The information processing device according to this embodiment detects the occurrence of flicker-induced luminance fluctuations based on the comparison results between multiple frame images in an environment in which a moving object 100 moves while measuring its own position and orientation. Based on this detection result, the device estimates an area where the position measurement accuracy will decrease, and performs control to adjust the speed of the moving object before it reaches a point or area where the position measurement accuracy will decrease. This allows the moving object to continue operating stably.

[0013] The mobile body system according to this embodiment is a system for automatically running (moving) a mobile body 100 in a logistics warehouse, and is configured with the mobile body 100 and an information processing device (FIG. 2(b)) wirelessly connected to the mobile body 100. Note that the information processing device may be mounted on the mobile body 100.

[0014] 2(a) shows an example of the configuration of a moving object 100. The moving object 100 is capable of transmitting and receiving data to and from an information processing device, and includes an imaging device 101, a position measurement device 102 that measures position information of the moving object 100 based on video captured by the imaging device 101, and a control device 103 that controls driving units such as motors of the moving object 100. The imaging device 101 captures video of the surrounding environment of the moving object 100. In this embodiment, the imaging device 101 is a stereo camera that is mounted on the moving object 100 and moves in accordance with the movement of the moving object 100. Note that the imaging device may be configured to be fixed in a position and orientation that allows it to measure the position and orientation of the moving object 100. The position measurement device 102 measures the self-position and orientation of the moving object 100 based on frame images (captured images) that make up a moving image captured by the imaging device 101. To measure the self-position and orientation, feature points extracted from the frame images are used. In this embodiment, the position measurement device 102 extracts feature points from images captured by the stereo camera, and therefore can acquire position information of the extracted feature points in three-dimensional coordinates based on the position of the stereo camera. The control device 103 controls the drive unit and various other devices provided in the moving body 100 in accordance with the control contents received from the information processing device.

[0015] FIG. 2(b) shows an example of the hardware configuration of an information processing device according to this embodiment. As shown in FIG. 2, the information processing device 200 includes a CPU 201, a ROM 202, a RAM 203, an external memory 204, an input unit 205, a display unit 206, and a network I / F 207. These units are connected to each other via a bus 208. The CPU 201 controls the entire information processing device 200. The CPU 201 loads a program stored in the external memory 204 into the RAM 203 and executes it to realize the processing of the flowcharts described below. The ROM 202 stores data such as various setting values ​​and parameters. The RAM 203 is a memory that temporarily stores data and control information, and serves as a work area used by the CPU 201 when executing various processes. The external memory 204 stores programs executed by the CPU 201.

[0016] The input unit 205 is a keyboard, mouse, or robot controller, and accepts input operations such as information. The display unit 206 outputs processing results under the control of the CPU 201. The display unit 206 may be of any type, such as a liquid crystal display device, a projector, or an LED indicator. The network I / F (interface) 207 is an interface for communicating with the outside world via a network. The information processing device 200 transmits and receives various data to and from the mobile object 100 via the network I / F 207. The CPU 201 receives, from the mobile object 100, video captured by the imaging device 101 and measurement results from the position measurement device 102, and transmits control details of the control device 103 to the mobile object 100 via the network I / F 207.

[0017] 3 shows an example of the functional configuration of an information processing device according to this embodiment. The information processing device 200 functions as an image acquisition unit 311, a flicker detection unit 312, a region estimation unit 313, a position acquisition unit 314, a control determination unit 315, and a presentation unit 316 by the CPU 201 executing a program stored in the external memory 204 or the like.

[0018] The image acquisition unit 311 acquires frame images that make up a moving image captured by the imaging device 101. The image acquisition unit 311 provides the acquired frame images to the flicker detection unit 312 in sequence. The flicker detection unit 312 detects the occurrence of flicker-induced luminance fluctuations based on multiple frame images. In this embodiment, the flicker detection unit 312 extracts feature points from frame images provided by the image acquisition unit 311 and compares the feature points between frame images captured at different times to detect flickering feature points. The flicker detection unit 312 also detects the location of flicker-induced luminance fluctuations based on the position coordinates of the flickering feature points. The flicker detection unit 312 also obtains the depth of the feature points by matching frame images captured by the stereo camera serving as the imaging device 101, and converts the position coordinates of the feature points into three-dimensional coordinates within the logistics warehouse where the mobile object 100 moves. The three-dimensional coordinates are expressed in a coordinate system that represents the location within the logistics warehouse, with the starting position of the mobile object as the origin. The flicker detection unit 312 provides the detection results to the area estimation unit 313.

[0019] The region estimation unit 313 estimates a region where the position measurement accuracy changes due to the flicker-induced luminance variation detected by the flicker detection unit 312. In this embodiment, the coordinate space in which the mobile object 100 moves is divided into multiple regions, and points or regions where the position measurement accuracy decreases are estimated based on the number of blinking feature points included in each region. The region estimation unit 313 provides position information of the estimated points or regions to the control determination unit 315. Note that the region estimation unit 313 may determine a certain range of the blinking feature points as a region affected by the flicker-induced luminance variation based on the position coordinates of the blinking feature points, and estimate a region where the position measurement accuracy decreases by calculating the logical sum of each region. Furthermore, when there are multiple points where the measurement accuracy of the position measurement device 102 decreases, the region estimation unit 313 may obtain a polygon that circumscribes these points and estimate the internal region of the polygon as a region where the measurement accuracy of the position measurement device 102 decreases. Furthermore, the area estimation unit 313 may identify points or areas where the measurement accuracy of the position measurement device 102 is reduced in a two-dimensional space or in a three-dimensional space.

[0020] The position acquisition unit 314 acquires the position information of the moving object 100 measured by the position measurement device 102 from the moving object 100 and provides it to the control determination unit 315 . The control determination unit 315 determines the control details for the moving body 100 and the peripheral devices of the moving body 100 based on the positional relationship between the position information provided by the area estimation unit 313 and the position information provided by the position acquisition unit 314. Here, the control determination unit 315 determines the control details for stably moving (traveling) the moving body 100 even if the measurement accuracy of the position measurement device 102 changes. The control determination unit 315 transmits the determined control details to the moving body 100 and the peripheral devices of the moving body 100.

[0021] In this embodiment, when the area estimated by the area estimation unit 313 is located within a portion of the moving path of the moving object 100, the control determination unit 315 determines to perform control to decelerate the moving object 100 at a predetermined acceleration. Specifically, immediately before the moving object 100 enters the area estimated by the area estimation unit 313, the control determination unit 315 transmits the determined control content to the moving object 100. When the moving object 100 receives the control content, the control device 103 reduces the rotation speed of the motor, thereby enabling the moving object 100 to move (travel) stably in an area where the measurement accuracy of the position measurement device 102 decreases. Note that when the moving object 100 is located within the area estimated by the area estimation unit 313, the control determination unit 315 may immediately transmit the determined control content to the moving object 100. The control determination unit 315 may also determine position information of a point on the moving path at which deceleration control is to be switched. The presentation unit 316 displays the control content determined by the control determination unit 315 on the display unit 206 .

[0022] Next, a process executed by the information processing device 200 according to this embodiment will be described. Fig. 4 is a flowchart showing the process executed by the information processing device 200 according to this embodiment. The flowchart in Fig. 4 is realized by the CPU 201 loading a program stored in the external memory 204 or the like into the RAM 203 and executing it. The flowchart in Fig. 4 starts when the information processing device 200 is started. Each process (step) in the flowchart will be described below with the prefix S (step) of each reference number.

[0023] As an initialization step, first in S401, the CPU 201 stores various setting values ​​such as the starting position of the moving body 100 and the driving speed of the moving body 100 in the ROM 202, etc. The CPU 201 also reads from the external memory 204, etc., the driving speed when decelerating the moving body 100 and predetermined parameters used to detect changes in position measurement accuracy from the blinking feature points in S403. The position measurement device 102 also acquires camera parameters necessary for self-position measurement.

[0024] When the moving object 100 starts moving, the CPU 201 starts receiving the video being captured by the imaging device 101 and the position and orientation being measured by the position measurement device 102, and repeatedly executes the processes of S402 to S405 at predetermined intervals. The intervals may be predetermined time intervals or predetermined distance intervals that the moving object 100 moves. Alternatively, the processes may be executed repeatedly at all times. In S402, the CPU 201 detects flicker in the moving environment of the moving object 100. In this embodiment, the CPU 201 first acquires frame images from data received from the moving object 100, and predicts feature points in the frame images after an infinitesimal time has elapsed based on the position and orientation at a given time and the amount of change in the position and orientation after an infinitesimal time has elapsed. The CPU 201 then uses the actual frame images to search for actual feature points around the predicted feature points, and if no actual feature points are found, designates the feature points as blinking feature points. In this manner, the CPU 201 extracts feature points from each of the frame images captured at different times, and selects feature points that are extracted in one frame image but not in the other frame image as blinking feature points.

[0025] Here, a specific method for selecting flickering feature points from among the feature points of frame images in this embodiment will be described. In the following description, it is assumed that even if some feature points in a frame image cannot be extracted due to flicker-induced luminance fluctuations, sufficient feature points have been extracted to perform position and orientation measurement in this step. First, the CPU 201 determines the position and orientation of the image capture device 101 measured from feature points in an arbitrary frame image and the relative position coordinates of the feature points in the frame image relative to the image capture device 101 as initial values ​​of the feature points. Next, because the position and orientation of the image capture device 101 will change after a short time due to movement of the moving object 100, the CPU 201 measures the position and orientation of the image capture device 101 based on the frame image after the short time and the initial values. Here, if feature points in a frame image after a short time have elapsed cannot be extracted due to flicker-induced luminance fluctuations, the number of feature points will be smaller than the number of feature points predicted from the change in position and orientation. In this embodiment, feature points that were predicted but could not be extracted are observed for a predetermined period of time, and if there is a difference from the repeated prediction, the feature points are selected as flickering feature points. In this embodiment, a spherical region of a certain radius for a feature point in a certain frame image is searched for, and if the feature point is not included in the spherical region, the feature point is determined to be a blinking feature point. Note that the above observation time is read in S401. On the other hand, if the number of feature points in a frame image after a short time period is greater than predicted, the feature points in the increased frame image are set as the initial values. In this embodiment, if one or more blinking feature points are included in a frame image, the CPU 201 detects the occurrence of a flicker-induced luminance fluctuation. The CPU 201 stores position information of the blinking feature points detected in the frame image in RAM 203.

[0026] In S403, the CPU 201 estimates an area where the position measurement accuracy will decrease due to flicker, based on the blinking feature points detected in S402. In this embodiment, the CPU 201 divides the three-dimensional coordinates in which the moving object 100 moves into small areas, and acquires the number of blinking feature points present in each small area based on the position information of the blinking feature points detected in S402. The CPU 201 then estimates an area where the position measurement accuracy will decrease, based on the number of blinking feature points present in each small area. Specifically, the CPU 201 sequentially targets each small area, and if the number of blinking feature points present in the target small area is equal to or greater than a threshold, the CPU 201 estimates the target small area as an area where the position measurement accuracy will decrease. On the other hand, if the number is less than the threshold, the CPU 201 determines that the position measurement accuracy will not change in the target small area.

[0027] In S404, the CPU 201 acquires, from the data received from the moving body 100, the position coordinates of the moving body 100 in the spatial coordinate system in which the moving body 100 moves. In S405, the CPU 201 calculates the distance between each point estimated in S403 and the position of the moving body 100, and if any of the distances is less than a predetermined threshold, it determines that the moving body 100 is approaching a point where the position measurement accuracy decreases, and determines to perform control to decelerate the moving body 100. On the other hand, if both distances are equal to or greater than the predetermined threshold, the moving body 100 is not decelerated. Note that the distance between each point estimated in S403 and the position of the moving body 100 may be the distance on the path along which the moving body 100 moves, or may be the distance in a spatial coordinate system along which the moving body 100 moves. The CPU 201 transmits the determined control content to the moving body 100 via the network I / F 207. Next, in S406, CPU 201 repeats the processes of S402 to S405 until it determines that an end instruction has been input from input unit 205, and if it determines that an end instruction has been input, it ends the series of processes shown in the flowchart of FIG.

[0028] In this embodiment, the CPU 201 displays a GUI (Graphical User Interface) screen on the display unit 206, reflecting the control content determined in S405. FIG. 5 shows an example of a GUI screen according to this embodiment. The user uses the GUI screen to check and edit the control content of the moving object 100. As shown in FIG. 5, the GUI screen displays a horizontal plane when the spatial coordinates in which the moving object 100 moves are viewed from above. Item 510 indicates the position and orientation of the moving object 100 on the horizontal plane. Also, hatched areas indicate areas on the horizontal plane that the moving object 100 cannot enter, while areas outside the hatched areas indicate areas that the moving object 100 can enter. Also, dashed lines indicate the path the user has planned for the moving object 100 to travel. The moving object 100 moves straight to the right from the left edge of the horizontal plane for a certain distance, then turns 90 degrees counterclockwise before continuing on the path.

[0029] Area 500 is an area where it is estimated that the measurement accuracy of the position measurement device 102 will decrease, and is displayed in gray on the map. In this embodiment, the area where the position measurement accuracy will decrease is estimated from the blinking characteristic points as described above. Icon 501 is located outside but near area 500 on the movement path of the moving object 100, and indicates a point where the control content determined by the control determination unit 315 will be changed before the moving object 100 enters area 500. In the example shown in FIG. 5, the moving object 100 enters from an area where there is no change in position measurement accuracy due to flicker into an area where the position measurement accuracy will decrease. In this embodiment, after passing the point indicated by icon 501, the moving object 100 moves (travels) at a speed slower than its normal movement speed.

[0030] A display field 502 showing the control content to be executed at the point of the icon 501 is provided at the bottom of the GUI screen. The display field 502 displays details of the control content determined in S405. Specifically, a message is displayed indicating that the moving object 100 will be driven at a reduced speed from the point of the icon 501 due to the occurrence of flicker, along with the driving speed. A color bar 503 is also provided at the upper left of the GUI screen. The color bar 503 indicates the correspondence between the background color on the map and whether or not the position measurement accuracy has decreased. In this embodiment, the background color of areas where the position measurement accuracy has decreased is gray, thereby distinguishing them from areas where the position measurement accuracy has not changed. Using the GUI screen described above, the user can check the areas where flicker occurs and the control content of the moving object 100 corresponding to the occurrence of flicker.

[0031] According to the above-described embodiment 1, in a moving body 100 having a self-position and orientation measurement function, even in an environment where the position measurement accuracy is reduced due to the occurrence of a flicker phenomenon, the moving body 100 can be made to move (run) stably while maintaining the operating efficiency of the moving body 100 as much as possible.

[0032] As a first modification of this embodiment, the method for detecting flicker in S402 of FIG. 4 is not limited to the method using feature points in frame images. For example, the CPU 201 may detect flicker based on the period of luminance changes in frame images and the frame rate of the video captured by the imaging device 101. The CPU 201 may also detect the occurrence of flicker based on a change in the number of feature points between multiple frames, even if the feature points are used, without using the position coordinates of the feature points. For example, flicker may be detected based on a decrease in the number of feature points in frame images after a short period of time. Furthermore, illumination information about the light source 110 (e.g., the presence or absence of a flicker phenomenon, the flicker frequency, and the illumination range in the spatial coordinates where the moving object 100 moves) may be stored in the ROM 202 or the like, and the CPU 201 may detect changes in the illumination information based on information acquired from another system. The CPU 201 may also detect the occurrence of flicker and estimate the area where the flicker occurs based on the measurement results of a flicker measurement device consisting of multiple illuminometers and oscilloscopes installed near the path of the moving object 100, and information about the measurement range of the illuminometers. In this case, the occurrence of flicker is detected using the measured cycle of illuminance change and the frame rate of the video captured by the imaging device 101. Another method is to use a camera (hereinafter referred to as a fixed camera) that has the same frame rate setting as the imaging device 101 and is installed in a position and orientation that allows it to capture the periphery of the moving object 100 and the environment in which it moves. In this case, the CPU 201 may perform threshold processing on the luminance change for each pixel of the frame image of the video captured by the fixed camera, and detect flicker and the illumination range of the light source 110 that causes the flicker. Note that a transformation matrix from the image coordinates of the fixed camera to the spatial coordinates in which the moving object 100 moves is stored in advance in the ROM 202, etc., and the CPU 201 uses this transformation matrix to transform the position coordinates of the illumination range and estimate the area in which flicker occurs.

[0033] As a second modification of this embodiment, the method for determining the position information of the blinking feature points in S402 of Fig. 4 is not limited to the method using a stereo camera. The image capture device 101 does not have to be a stereo camera. In this case, the three-dimensional coordinates of the blinking feature points may be determined from the frame images of the image capture device 101 as a monocular camera and the amount of movement of the moving object 100. The CPU 201 may also read a three-dimensional map of the environment in which the moving object 100 moves, and use the three-dimensional map to determine the three-dimensional coordinates of the feature points on the frame images.

[0034] As a third modification of this embodiment, the method for measuring the self-position and orientation is not limited to the method using the image captured by the image capturing device 101, as long as the method can measure the position information of the moving body 100 in the world coordinate system in the driving environment of the moving body 100. For example, a method may be used in which the position information of the moving body 100 is measured by image measurement or image recognition from an image captured by a camera not mounted on the moving body 100. Alternatively, a method may be used in which the edge arrangement and distance information obtained from a depth image are used as an index for position measurement.

[0035] As a fourth modification of the present embodiment, the control content determined by the CPU 201 in S405 of FIG. 4 is not limited to deceleration of the moving object 100. For example, the CPU 201 may control an audio output device (not shown) mounted on the moving object 100 to output a warning sound. In this case, the CPU 201 may control the output warning sound to be changed depending on the distance to the flicker occurrence area or the degree of deterioration in position measurement accuracy in the flicker occurrence area. The CPU 201 may also control changing of image processing parameters used by the position measurement device 102 when extracting feature points, so as to make it easier to extract feature points from images captured by the image capture device 101. The CPU 201 may also control changing of feature points on map information or map information when the position measurement device 102 measures the position and orientation. The CPU 201 may also control changing of the frame rate setting of the image capture device 101 so as to synchronize with the flicker frequency. Furthermore, the CPU 201 may perform control to change the route so as to bypass the point or area estimated in S404 of Fig. 4. Furthermore, the CPU 201 may perform control to accelerate the moving body 100 when the distance between each point estimated in S404 and the position of the moving body 100 becomes equal to or greater than a threshold.

[0036] <Embodiment 2> In the first embodiment, a method for detecting changes in the measurement accuracy of the position measurement device 102 based on blinking feature points included in frame images of the imaging device 101 mounted on the moving body 100 was described. In this embodiment, a method for dynamically changing the control content based on the degree of change in measurement accuracy at a point or area where the measurement accuracy changes will be described. Below, a description of the same parts as in the first embodiment will be omitted, and the description will focus on the differences.

[0037] FIG. 6 illustrates an example of the functional configuration of the information processing device 200 according to this embodiment. In the first embodiment, the region estimation unit 313 divides the coordinate space in which the moving object 100 moves and estimates a region in which measurement accuracy decreases by determining the number of blinking feature points included in the divided region based on a threshold. FIG. 6 differs from FIG. 3 in that the function of an influence estimation unit 600 is added between the region estimation unit 313 and the control determination unit 315. In this embodiment, the influence estimation unit 600 estimates the degree of influence on position measurement accuracy for the region estimated by the region estimation unit 313 based on the number of blinking feature points. In this embodiment, the control determination unit 315 changes the control content depending on the estimation result of the influence estimation unit 600. For example, the control determination unit 315 performs control to gradually reduce the movement speed of the moving object 100 depending on the degree of decrease in position measurement accuracy.

[0038] Next, a description will be given of the processing executed by the information processing device 200 according to this embodiment. Fig. 7 is a flowchart showing the processing executed by the information processing device 200 according to this embodiment. This flowchart differs from Fig. 4 in that S700 is added after S403. In S700, the CPU 201 estimates the degree of influence on the position measurement accuracy for the area estimated in S403. Specifically, the CPU 201 estimates the degree of degradation of the position measurement accuracy from the number of blinking feature points included in the area estimated in S403. In this embodiment, in S401, a data table is read that converts the data so that the greater the number of blinking feature points, the greater the degree of degradation of the position measurement accuracy. In this step, the number of blinking feature points is calculated for each small area divided in S403, and the degree of degradation of the position measurement accuracy in the target small area is estimated by referring to the data table in which the degree of degradation of the position measurement accuracy is uniquely calculated based on the number of blinking feature points.

[0039] In S405, the CPU 201 calculates a control parameter according to the degree of accuracy degradation estimated in S700. In this embodiment, the speed of the moving body 100 is controlled using the control parameter table read in S401. Specifically, when the moving body 100 is moving within an area where the position measurement accuracy is degraded and enters an area where the degree of degradation of the position measurement accuracy is even greater, the CPU 201 refers to the control parameter table and performs control to decelerate the moving body. On the other hand, when the moving body 100 enters an area where the degree of degradation of the position measurement accuracy is smaller, the CPU 201 also refers to the control parameter table and performs control to accelerate the moving body.

[0040] Fig. 8 shows an example of a GUI screen according to this embodiment. The following mainly describes the differences from Fig. 5. Area 800 is an area where it is estimated that the measurement accuracy of the position measurement device 102 will decrease, and is displayed in gray on the map. Area 800 is displayed in a lighter gray than area 500, and its display mode is different from that of area 500, thereby expressing that the control content is different from that of area 500. Area 800 is an area where the degree of decrease in position measurement accuracy is smaller than that of area 500.

[0041] Icon 801 is located outside but near area 800 on the movement path of moving body 100, and indicates a point where the control content determined by control determination unit 315 is changed before entering area 800. In addition, since area 800 differs from area 500 in the degree of decrease in position measurement accuracy, icon 801 executes control different from icon 501. In addition, a display field 802 showing the control content to be executed at the point of icon 801 is provided below the display field 502 showing the control content to be executed at the point of icon 501. In the display field 802, a message is displayed indicating that the moving object 100 will be driven at a low speed from the point of icon 801 due to the occurrence of flicker, along with the driving speed. Note that the control content in the region 500 and the region 800 varies depending on the degree of deterioration in the position measurement accuracy and the control parameter table read in S401. In this embodiment, since the degree of deterioration in the position measurement accuracy in the region 800 is smaller than in the region 500, the driving speed in the region 800 is faster than in the region 500 and slower than normal. Furthermore, a color bar 803 indicates the correspondence between the background color on the map and the degree of degradation in the position measurement accuracy of the position measurement device 102. In this embodiment, the degree of degradation in the position measurement accuracy in the environment in which the moving object 100 moves is indicated by the shade of the background color. In this embodiment, the darker the color of the color bar 803, the greater the degree of degradation in the position measurement accuracy.

[0042] According to the above-described second embodiment, in a moving body 100 having a self-position and orientation measurement function, even in an environment where the position measurement accuracy is reduced due to the occurrence of a flicker phenomenon, the moving body 100 can be made to move (run) stably while maintaining the operating efficiency of the moving body 100 as much as possible.

[0043] As a modified example of this embodiment, the method for estimating the degree of change in position measurement accuracy is not limited to the method using the number of blinking feature points for each small region divided in S403. For example, the CPU 201 may detect the flicker phenomenon of the light source 110 from the output value of the illuminance sensor and estimate the degree of deterioration in position measurement accuracy from the flicker frequency of the light source 110 and the frame rate of the video captured by the image capture device 101. The CPU 201 may also calculate the degree of deterioration in position measurement accuracy so that the larger the fluctuation range of the flicker-induced luminance fluctuation is, the greater the degree of deterioration in position measurement accuracy is. The CPU 201 may also calculate the degree of deterioration in position measurement accuracy so that the larger the amount of change in the luminance value of a pixel in a frame image of the image capture device 101 is, the greater the degree of deterioration in position measurement accuracy is. Note that when calculating the degree of deterioration in position measurement accuracy based on image information, images captured by a fixed camera installed in a position and orientation that can capture the surroundings and movement environment of the mobile object 100 may be used instead of images captured by the image capture device 101.

[0044] <Embodiment 3> In the first embodiment, a method for estimating an area where position measurement accuracy decreases from blinking feature points included in a frame image of an imaging device 101 mounted on a moving object 100 has been described. In the present embodiment, a method for sharing an area where position measurement accuracy decreases among multiple moving objects 100 while multiple moving objects 100 are operating in a logistics warehouse will be described. Below, a description of the same parts as in the first embodiment will be omitted, and the description will focus on the differences.

[0045] FIG. 9 illustrates an example of the functional configuration of the information processing device 200 according to this embodiment. FIG. 9 differs from FIG. 3 in that the function of a region retention unit 900 is added between the region estimation unit 313 and the control determination unit 315. The region retention unit 900 stores the estimation results of the region estimation unit 313 in the RAM 203. In this embodiment, because a plurality of mobile objects 100 are operating within the logistics warehouse, regions where the position measurement accuracy estimated from the image capture devices 101 mounted on each mobile object 100 decreases are integrated. In this case, the information processing device 200 integrates the regions where the position measurement accuracy decreases by logical summing. In this embodiment, flicker-induced luminance fluctuations are detected from images captured by the image capture devices 101 of multiple mobile objects 100, and therefore flicker-induced luminance fluctuations can be detected over a wider range in the environment in which the mobile object 100 moves than in the case of a single mobile object 100. Furthermore, for an area outside the range that can be captured by the imaging device 101 of a certain moving body 100, it may be possible to detect flicker-induced luminance fluctuations in an image captured by the imaging device 101 of another moving body 100. Therefore, flicker-induced luminance fluctuations can be detected earlier by integrating images captured by multiple moving bodies 100 rather than using images captured by only a single moving body 100. The control determination unit 315 determines the control details for each moving body 100 based on the result of integrating areas where the position measurement accuracy decreases, obtained from the multiple moving bodies 100, and the position information of each moving body 100.

[0046] According to the above-described third embodiment, in a moving body 100 having a self-position and orientation measurement function, even in an environment where the position measurement accuracy is reduced due to the occurrence of a flicker phenomenon, the moving body 100 can be made to move (run) stably while maintaining the operating efficiency of the moving body 100 as much as possible.

[0047] As a modification of this embodiment, the data shared among multiple moving bodies 100 is not limited to the estimation results of the region estimation unit 313. For example, the CPU 201 may store the position coordinates of blinking feature points and control contents. Furthermore, the CPU 201 may be configured to save and update region information as the estimation results of the region estimation unit 313 while the moving body 100 is moving, or may be configured to read region information of other moving bodies 100 stored in the ROM 202 in S401.

[0048] Although the present invention has been described above with reference to the embodiments, the above embodiments are merely illustrative of specific examples of how the present invention can be implemented, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0049] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0050] 100: moving body, 101: imaging device, 102: position measurement device, 103: control device, 200: information processing device

Claims

1. An information processing device for controlling a moving object, comprising: a measuring means for measuring a position of the moving object from a captured image of a surrounding environment of the moving object; a detection means for extracting feature points from each of the captured images, which are frame images of a moving image, one of the frame images captured at a different time, and another of the frame images, and detecting flicker feature points extracted in the one of the frame images but not extracted in the other of the frame images, and detecting the occurrence of luminance fluctuations due to flicker in the environment in which the moving object moves, based on the flicker feature points; an estimation means for dividing a space in which the moving object moves into a plurality of regions, and estimating the region in which the number of the flickering feature points detected by the detection means is equal to or greater than a threshold value as a region in the space in which the moving object moves, in which the measurement accuracy of the measurement means changes due to a luminance fluctuation caused by the flicker; a determining means for determining a speed or a moving path of the moving object based on the area where the measurement accuracy changes; and The information processing device is characterized in that, when there are multiple areas in the space in which the moving body moves where the measurement accuracy changes, the estimation means estimates the internal area of ​​a polygon based on the multiple areas where the measurement accuracy changes as the area where the measurement accuracy changes.

2. The method further comprises: a position acquisition means for acquiring position information indicating the position of the moving body measured by the measurement means based on feature points extracted from a captured image of the surrounding environment of the moving body; The information processing device according to claim 1, characterized in that the determination means determines the speed or the movement path of the moving body based on the positional relationship between the area where the measurement accuracy changes estimated by the estimation means and the position information acquired by the position acquisition means.

3. the estimation means further estimates a degree of change in the measurement accuracy based on the number of the blinking feature points detected by the detection means; 3. The information processing apparatus according to claim 1, wherein the determining means determines the speed of the moving object or the moving route of the moving object in accordance with the degree of change.

4. The information processing device according to claim 2, characterized in that the determination means controls the speed of the moving body to slow down when the distance from the position indicated by the position information acquired by the position acquisition means to the area where the measurement accuracy estimated by the estimation means changes is less than a predetermined threshold.

5. The method further includes a storage unit that controls the plurality of moving bodies and stores position information of an area where the measurement accuracy of the measurement unit changes, the position information being estimated for each of the plurality of moving bodies; 5. The information processing apparatus according to claim 1, wherein the determining means determines the speed of the moving object or the moving route of the moving object based on the stored position information.

6. 6. The information processing apparatus according to claim 1, further comprising a presentation unit that presents information indicating the speed of the moving object or the moving route of the moving object determined by the determination unit.

7. 7. The information processing apparatus according to claim 6, wherein the presenting means displays information indicating the speed of the mobile object or the moving route of the mobile object on a map on which the mobile object is moving.

8. The detection means a position and orientation of the moving object measured from a feature point in a frame image and a coordinate of a relative position and orientation of the feature point in the frame image of an imaging device that captured the frame image are set as initial values ​​of the position of the feature point; A feature point that could not be extracted during a predetermined time of observation at a position predicted in a frame image after a short time based on the initial value is defined as the blinking feature point; An information processing device according to any one of claims 1 to 7, characterized in that if the number of feature points in the frame image after the small amount of time becomes greater than the predicted position, the increased position is updated as an initial value of the position of the feature point.

9. 1. An information processing method for controlling a moving object, the method comprising: measuring means for measuring a position of the moving object from a captured image of a surrounding environment of the moving object; a detection step in which the captured images are frame images of a moving image, and feature points are extracted from one of the frame images and the other of the frame images captured at different times, and flicker feature points extracted in the one frame image but not in the other frame image are detected, and the occurrence of luminance fluctuations due to flicker in the environment in which the moving object moves is detected based on the flicker feature points; an estimation step of dividing a space in which the moving object moves into a plurality of regions, and estimating the region in which the number of the flickering feature points detected by the detection step is equal to or greater than a threshold as a region in the space in which the moving object moves, in which the measurement accuracy of the measurement means changes due to luminance fluctuations caused by the flicker; a determining step of determining a speed or a moving path of the moving object based on the region where the measurement accuracy changes; and An information processing method characterized in that, in the estimation step, if there are multiple areas in the space in which the moving body moves where the measurement accuracy changes, the internal area of ​​a polygon based on the multiple areas where the measurement accuracy changes is estimated as the area where the measurement accuracy changes.

10. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 8.

Images