Information processing device, mobile object control system, information processing method, and program

Abstract

To improve the operation efficiency of a moving object in the case where there are a plurality of obstacles on a movement path.SOLUTION: An information processing apparatus, for controlling the movement of a moving object, such as an unmanned carrier vehicle or an autonomously moving robot, comprises an image acquiring section 301 that acquires an image obtained by taking pictures of a moving object and a peripheral environment thereof, and a location-information acquiring section 303 that acquires information of a location / attitude of the moving object. An object analyzing section 302 analyzes a movement of the object existing in the peripheral environment of the moving object based on a group of images acquired in a predetermined period of time by the image acquiring section 301. A control-content determining section 304 determines a control content (moving direction, moving velocity, moving location or the like) related to a movement of the moving object based on the location information of the moving object acquired by the location-information acquiring section 303 and an analysis result of the object analyzing section 302.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technology for controlling the movement of a moving object. [Background technology]

[0002] Mobile objects used in factories, logistics warehouses, etc. are devices that transport cargo such as products and parts to a predetermined location. Examples include automated guided vehicles (AGVs) and autonomous mobile robots (AMRs).

[0003] Patent Document 1 discloses a method for more reliably avoiding collisions with obstacles when controlling the movement of an autonomous moving body. After detecting an obstacle on a movement path, a movement path is generated to avoid interference between the moving body and the obstacle, and movement control is performed along the movement path. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-288930 Summary of the Invention [Problem to be solved by the invention]

[0005] However, obstacles on the path of a moving object are not limited to a single obstacle. For example, consider a case where a large number of pedestrians are present on the path. In this case, conventional movement control of a moving object may result in a decrease in the efficiency of route planning to the destination and the operation of the moving object. An object of the present invention is to improve the travel efficiency of a mobile object when a plurality of obstacles exist on the travel route. [Means for solving the problem]

[0006] An apparatus according to an embodiment of the present invention is an information processing apparatus for controlling the movement of a moving body, and includes: an image acquisition means for acquiring captured images of the moving body and its surrounding environment; a position information acquisition means for acquiring information on the position and orientation of the moving body; an analysis means for analyzing the movements of a plurality of objects present in the surrounding environment of the moving body using a plurality of images acquired at different times by the image acquisition means; and the position and orientation information of the moving body acquired by the position information acquisition means. a determining means for determining a control content relating to the movement of the moving body based on the statistical quantity relating to the object acquired by the analyzing means; And, it is equipped with. The determination means determines the amount of change in the movement position of the moving body to be a first change amount when the density of the distribution of the object is a first value, and determines the amount of change in the movement position of the moving body to be a second change amount smaller than the first change amount when the density of the distribution of the object is a second value greater than the first value. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the operation efficiency of a mobile object when a plurality of obstacles exist on the travel route. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating the overall system configuration of an information processing device. [Figure 2] FIG. 1 is a block diagram showing a configuration of an information processing device. [Figure 3] FIG. 2 is a block diagram showing a functional configuration of the information processing device. [Figure 4] 4 is a flowchart showing the processing of the information processing device according to the first embodiment. [Figure 5] 4 is a flowchart showing the object analysis process of the first embodiment. [Figure 6] 4 is a flowchart showing a process for determining control content in the first embodiment. [Figure 7] 10 is a flowchart showing a process for determining control content according to a second embodiment. [Figure 8] FIG. 11 is a schematic diagram showing an example of display information in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In this embodiment, an example of a control system for an automated guided vehicle, which is a moving body, is shown. Note that the configuration shown in the following example is merely an example, and the present invention is not limited to the configuration shown in the drawings. Identical components are designated by the same reference numerals to avoid duplication of explanation.

[0010] [First Example] In this embodiment, an application example of the present invention is shown for a case where multiple obstacles exist on the movement path of a moving object. A method for determining the control of a moving object toward a destination when multiple pedestrians exist on the movement path is described. In this specification, the direction and speed of the flow of the entire group of pedestrians are defined as the direction and speed of movement of people. A process for detecting the movement direction, speed, density, and position of people is performed, and movement control is performed so that the movement direction, movement position, and movement speed of the moving object move similarly to the flow of the pedestrian group.

[0011] The overall configuration of the system will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing the overall configuration of the system of this embodiment. The system of this embodiment is composed of a computer 101, an imaging device 102, a mobile object 103, and a network 104. The computer 101 is connected to the imaging device 102 and the mobile object 103 via the network 104. The network 104 can be realized by a wired connection such as Ethernet (registered trademark) or a wireless connection such as a wireless LAN (Local Area Network). There are no particular limitations on the network 104 as long as it is configured to enable mutual communication of information between connected devices.

[0012] The imaging device 102 is a network camera or the like. For example, if the imaging area of ​​the imaging device 102 is within the premises of a factory, the imaging device 102 is installed so as to be able to capture images overlooking the entire premises. Based on information received from the computer 101, the imaging device 102 captures images overlooking the entire premises and transmits signals of the captured images to the computer 101.

[0013] The mobile unit 103 is an unmanned transport vehicle, and calculates its position and orientation, sets its destination, and sets its objectives based on information received from the computer 101. earth In the following description, the moving body 103 will be referred to as an AGV 103.

[0014] The computer 101 receives captured image signals from the imaging device 102 and performs processing to analyze the movement of objects in the captured images. The computer 101 also acquires position and orientation information of the AGV 103. Position and orientation information is information that indicates the position and orientation of an object in real space. In the following explanation, position and orientation information will be referred to as "position information," but orientation information is also included. The computer 101 performs processing to set the movement route of the AGV 103 and processing to determine the control content based on the acquired position information of the AGV 103 and the image analysis results of the captured images. Details of the software functions of the computer 101 will be described later.

[0015] The hardware configuration of the computer 101 will be described with reference to Figure 2. Figure 2 is a block diagram showing the configuration of the computer 101. A CPU (Central Processing Unit) 201 controls various devices connected to a system bus 208. A ROM (Read Only Memory) 202 stores a BIOS (Basic Input / Output System) program and a boot program. A RAM (Random Access Memory) 203 is used as the main storage device of the CPU 201. An external memory 204 stores programs processed by the computer 101.

[0016] The input unit 205 is a keyboard, pointing device, robot controller, etc., and processes input of information from the user. The display unit 206 has a display device such as a liquid crystal display or projector, and outputs the calculation results of the computer 101 to the display device and displays them on the screen in accordance with commands from the CPU 201. The communication interface (I / F) unit 207 communicates information with the imaging device 102 and the AGV 103.

[0017] The functional configuration of the information processing device of this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the functional configuration of the information processing device 300, and the imaging device 102 and AGV 103. The information processing device 300 corresponds to the software functional configuration of the computer 101, and is composed of an image acquisition unit 301, an object analysis unit 302, a position information acquisition unit 303, and a control content determination unit 304. The image acquisition unit 301 receives signals of images captured by the imaging device 102 over a predetermined period of time, and acquires image data. The image data and various information are transmitted and received via the network 104.

[0018] The object analysis unit 302 performs object analysis on the captured image acquired by the image acquisition unit 301. In the object analysis, statistics such as the moving direction and moving speed of the object, the density of the object (distribution density or density), and the position of the object are calculated. The object analysis unit 302 outputs the calculated statistics to the control content determination unit 304.

[0019] The position information acquisition unit 303 acquires, as position information, information on the position and orientation of the AGV 103 calculated by a position information calculation unit 321 (described later). The control content determination unit 304 determines the control content of the AGV 103 based on the position information of the AGV 103 acquired by the position information acquisition unit 303 and statistics that are the analysis results of the object analysis unit 302. Details of the processing performed by the control content determination unit 304 will be described later.

[0020] The imaging device 102 includes an imaging unit 311 and a drive control unit 312. The imaging unit 311 captures an image of a subject and records the image data. The drive control unit 312 controls the drive of the imaging direction and imaging angle of view. Specifically, PTZ control of the imaging unit 311 is performed. P, T, and Z stand for panning, tilting, and zooming, respectively.

[0021] The AGV 103 includes a position information calculation unit 321 and a mobile object control unit 322. The position information calculation unit 321 receives a captured image signal acquired by the imaging unit 311 and detects the AGV 103 from the captured image. The position information calculation unit 321 calculates the detected coordinate values ​​as position information. The mobile object control unit 322 controls the motor, which is an actuator equipped in the AGV 103, and performs steering control to change the direction of the wheels.

[0022] With reference to Figure 4, the control of adjusting the movement speed and movement position of the AGV 103 in accordance with the flow of multiple moving objects (other AGVs, AMRs, etc.) and people in this embodiment will be described. Figure 4 is a flowchart explaining the processing of the information processing device 300, and each process is realized by the CPU 201 executing a control program. Assuming a situation in which multiple pedestrians are present on the movement route of the AGV 103, a method for determining the content of movement control of the AGV 103 toward the destination will be described.

[0023] In S401, a system initialization process is executed. A process of reading a program from the external memory 204 is executed, and the information processing device 300 is ready to operate. Also, in S401, the information processing device 300 sets the destination of the AGV 103. The destination is a predetermined location to which the user wants the AGV 103 to move, and is expressed by the coordinates of the real space (X WT ,Y WT ), the coordinate values ​​are specified. In addition, a process is executed to read a coordinate conversion table that shows the correspondence between the coordinate values ​​of the captured image and the coordinate values ​​of the real space. T ,Y T ) is used. By using a coordinate conversion table, it is possible to convert the coordinate values ​​of the captured image to the corresponding positions in real space. Specifically, the data in the coordinate conversion table consists of a rotation matrix that converts the coordinate values ​​of the captured image to the corresponding positions in real space, a translation vector, and the camera's internal parameters such as the focal length and resolution of the camera. This coordinate conversion table allows the coordinates (X T ,Y T ) to real-space coordinates (X WT ,YWT ) can be calculated. In the three-dimensional coordinate system, the coordinate (X T ,Y T ,Z T ) and coordinates (X WT ,Y WT ,Z WT ), but below we will explain movement control in a two-dimensional coordinate system.

[0024] In S402, the image acquisition unit 301 executes a process for acquiring data of the captured image captured by the imaging device 102. The captured image is, for example, an overhead image capturing the entire site. As image information for a predetermined period, a plurality of image data items are acquired from T seconds ago to the present. However, any value can be set for the predetermined time, T seconds.

[0025] In S403, an object analysis process is executed by the object analysis unit 302. For example, a process is executed to calculate statistics such as the direction and speed of people's movement, the density of people's distribution, and people's positions based on the data group of the captured images acquired in S402. Details of the process will be described later with reference to FIG.

[0026] In S404, the position information acquisition unit 303 executes a process of acquiring the position information of the AGV 103. The information acquired here is information on the coordinates and direction in the real space. The coordinates of the AGV 103 in the real space are (X WC ,Y WC ) is written as

[0027] In S405, the control content determination unit 304 executes a process of determining control content related to the movement of the AGV 103 based on the statistics calculated in S403 and the position information acquired in S404. The control content determines the movement direction, movement speed, movement position, etc. of the AGV 103. Details of the process will be described later with reference to FIG. 6.

[0028] In S406, as an end determination process, the information processing device 300 determines whether the AGV 103 has arrived at the destination. If it is determined that the AGV 103 has arrived at the destination, the process shown in Fig. 4 is ended, and if it is determined that the AGV 103 has not arrived at the destination, the process proceeds to S402 and continues.

[0029] Next, the processing performed by the object analysis unit 302 in S403 of Fig. 4 will be described in detail with reference to Fig. 5. Fig. 5 is a flowchart illustrating the object analysis and evaluation value calculation processing. First, in S501, a process of dividing the captured image into a plurality of small regions is performed. For example, the captured image is divided into five regions each in the vertical direction and horizontal direction, resulting in a process of dividing the captured image into 25 rectangular small regions. The processes from S502 to S508 are then executed for each divided small region.

[0030] Next, in S502, the object analysis unit 302 detects the head of a person in the captured image. A model for detecting the head of a person is prepared in advance, and the person is detected by applying a template matching method. The object analysis unit 302 estimates the coordinate position of the person's feet in the captured image from the detected head of the person. Furthermore, the object analysis unit 302 uses a coordinate conversion table to determine the coordinates of the person's feet estimated in the captured image in real space.

[0031] In S503, the object analysis unit 302 tracks the movement of the person detected in S502. Specifically, a process is performed to calculate the amount of movement of the person detected in S502 from a group of consecutive captured images (captured image frames). The coordinates of the target person at any time (t) are (X W1(t) ,Y W1(t) ) and the coordinates of the target person at time (t-1) one unit time before time (t) are (X W1(t-1) ,Y W1(t-1) ) is written as

[0032] As shown in the following equation 1, the coordinates (X W1(t) ,Y W1(t) ) and coordinates (X W1(t-1) ,Y W1(t-1)The movement amount (ΔX1, ΔY1) is calculated from the difference between the

number

[0033] In S504, the object analysis unit 302 calculates a value corresponding to the person's movement direction. The value of the person's movement direction indicates the overall direction of the flow of people. The object analysis unit 302 adds up the movement vectors calculated in S503 for the number of people tracked. By determining the direction of the movement vector after addition, a value corresponding to the person's movement direction is obtained.

[0034] In S505, the object analysis unit 302 calculates the movement speed of the people. The movement vector after addition obtained in S504 is divided by the number of people tracked and the differential time (the difference between time (t) and time (t-1)). This makes it possible to calculate the average movement speed.

[0035] In S506, the object analysis unit 302 calculates the density of people. The density of people (distribution density) can be calculated by dividing the number of people included in each small area divided in S501 by the area of ​​the real space corresponding to one small area.

[0036] In S507, the object analysis unit 302 calculates the position of the person. The position of the person is a value indicating which area within the small area the detected person is in. The object analysis unit 302 averages the coordinates of the person detected in S502, and performs calculation processing to determine which coordinates in real space the calculation result corresponds to.

[0037] In S508, the object analysis unit 302 determines whether the calculation of statistics has been completed for all small regions. If it is determined that the processes from S502 to S507 have been completed for all small regions, the process ends. If not, the process returns to object detection in S502 and continues to process the next small region.

[0038] Based on the statistics obtained by the above process and predetermined conditions, a process is executed to determine the control contents of the AGV 103, i.e., the movement direction, movement speed, and movement position. Fig. 6 is a flowchart explaining the process performed by the control content determination unit 304 in S405 of Fig. 4.

[0039] First, in S601, the control content determination unit 304 acquires the statistics calculated in S403. Next, in S602, the control content determination unit 304 evaluates the density value. The density value is compared with a predetermined value (threshold value). If it is determined that the density value is less than the predetermined value, the process proceeds to S604. On the other hand, if it is determined that the density evaluation value is equal to or greater than the predetermined value, the process proceeds to S603.

[0040] In S603, the control content determination unit 304 evaluates the value of the movement direction. A comparison process is performed between the value indicating the person's movement direction and the value indicating the orientation of the AGV 103 acquired in S404. Specifically, an angular difference is calculated between the orientation of the movement vector corresponding to the value of the person's movement direction and the orientation of the AGV 103. The calculated angular difference is compared with a predetermined value (threshold value). If it is determined that the angular difference is less than the predetermined value, the process proceeds to S604. If it is determined that the calculated angular difference is equal to or greater than the predetermined value, the process proceeds to S605. Note that if the calculated angular difference is less than the threshold value, it indicates that the movement direction of the AGV 103 is close to the movement direction of the person. Furthermore, if the calculated angular difference is equal to or greater than the threshold value, it indicates that the movement direction of the AGV 103 is significantly different from the movement direction of the person.

[0041] In S604, the control content determination unit 304 determines the movement direction of the AGV 103 based on the determination results of S602 and S603. Here, it is determined that the movement direction of the AGV 103 will not be changed, and the process proceeds to S606. Also, in S605, the control content determination unit 304 determines, based on the determination results of S602 and S603, that the control content of the AGV 103 will be to wait for a certain period of time. The AGV 103 will wait for the certain period of time, and then each process is performed in order from S601 to determine the next control content again.

[0042] In S606, the control content determination unit 304 evaluates the position. The value of the person's position is compared with the position information of the AGV 103. As explained in S507, the person's position is a value indicating which area within the small area the person is located in. As in S507, the control content determination unit 304 performs a calculation process based on the position information of the AGV 103 to determine which area within the small area the relevant position is included in. Specifically, a value corresponding to an area identified by a two-dimensional position index (left-right and front-back directions, etc.) within the small area is calculated. If it is determined that the calculated area in which the AGV 103 is located matches the area indicating the person's position, proceed to S608; if it is determined that they do not match, proceed to S607.

[0043] In S607, the control content determination unit 304 calculates the minimum distance to the center position of the two-dimensional area within the small area representing the person's position from the position information of the AGV 103. The coordinates of the center position are converted from the coordinates on the captured image to coordinates in real space, and the minimum distance in real space is calculated. After S607, the process proceeds to S608.

[0044] In S608, the control content determination unit 304 determines the control content for the movement position of the AGV 103 based on the results of S606 and S607. If it is determined that the area where the AGV 103 is located matches the area representing the person's location, the control content determination unit 304 determines not to change the movement position. On the other hand, if it is determined that the area where the AGV 103 is located does not match the area representing the person's location, the control content determination unit 304 determines to control the movement position in the direction of the area representing the person's location, in accordance with the minimum distance calculated in S607.

[0045] In S609, the control content determination unit 304 evaluates the movement speed. The movement speed of the person is compared with the movement speed of the AGV 103. The control content determination unit 304 calculates the absolute value of the difference between the movement speed of the person and the movement speed of the AGV 103, and compares it with a predetermined value (threshold value). If it is determined that the calculated value is less than the predetermined value, the process proceeds to S611, and if it is determined that the calculated value is equal to or greater than the predetermined value, the process proceeds to S610.

[0046] In S610, the control content determination unit 304 calculates the speed difference between the moving speed of the person and the moving speed of the AGV 103. That is, the speed difference is calculated by subtracting the moving speed of the AGV 103 from the moving speed of the person. Next, the process proceeds to S611. In S611, the control content determination unit 304 determines the control content for the moving speed of the AGV 103 based on the results of S609 and S610. If the value calculated in S609 is less than the threshold, the control content determination unit 304 does not change the moving speed of the AGV 103, but leaves it at the current moving speed. On the other hand, if the value calculated in S609 is equal to or greater than the threshold, the control content determination unit 304 adds the speed difference calculated in S610 to the current moving speed as a correction amount. That is, the moving speed is adjusted to approximate the flow of people (flow of pedestrians).

[0047] As described above, the movement direction, movement speed, and movement position of the AGV 103 are controlled according to the flow of people, and the AGV 103 proceeds toward the destination. A signal corresponding to the control content determined by the control content determination unit 304 is transmitted to the mobile object control unit 322, and the movement of the AGV 103 toward the destination is controlled based on the control content.

[0048] In S406 (end determination) of FIG. 4, the coordinates of the destination (X WT ,Y WT ) and the position information of AGV103 acquired in S404 (X WC ,Y WC) to determine whether the AGV 103 has arrived at the destination. Specifically, the difference between the X and Y coordinates is calculated as shown in the following equation 2, and if the absolute value of the difference is equal to or less than a predetermined threshold, it is determined that the AGV 103 has arrived at the destination. If the difference between the X and Y coordinates is greater than the predetermined threshold, it is determined that the AGV 103 has not arrived at the destination, and the process returns to S402 to continue.

number

[0049] According to this embodiment, it is possible to accurately determine the control details related to the movement of a moving object using the statistics calculated in S403 of Fig. 4, and to control the movement of a moving object while taking into account the flow of people. The moving object does not stagnate even in crowded situations, and proceeds toward its destination in line with the flow of people, thereby improving the operation efficiency of the moving object.

[0050] [Modification of the first embodiment] A modification of the first embodiment will now be described. <Variation 1-1> In the first embodiment, an example was shown in which the position information of the AGV 103 was calculated from a captured image, but in this modified example, another example of a method for calculating the position information of the AGV 103 will be described. The AGV 103 is equipped with a sensor (GPS, RFID, beacon, etc.) that measures position information, and the position information of the AGV 103 is calculated based on the sensor output. GPS is an abbreviation for "Global Positioning System," and RFID is an abbreviation for "Radio Frequency Identifier."

[0051] The AGV 103 is equipped with a camera or a sensor, and position information of the AGV 103 is calculated by applying SLAM technology from images captured by the equipped camera or sensor (LiDAR: Light Detection And Ranging) information. SLAM is an abbreviation for "Simultaneous Localization And Mapping." In the first embodiment, the position information calculation unit 321 that calculates the position information of the AGV 103 is shown as a function of the AGV 103, but this configuration is not limiting. In a modified example, this function is configured as one of the functions of the information processing device 300. Alternatively, the imaging device 102 or another device may be configured to include the position information calculation unit 321.

[0052] <Variation 1-2> In the first embodiment, an example was shown in which object analysis was performed on a current captured image captured by the imaging device 102. Object analysis can be applied to previously captured images. In this modified example, signals of multiple captured images captured by the imaging device 102 over a predetermined time period are received to acquire the previously captured images. The predetermined time period involves processing to acquire multiple image data groups over a period of several seconds, several minutes, several hours, or several days. Then, object analysis is performed on the previously captured images, and processing to calculate statistics (people's movement direction, movement speed, density, and position) is performed.

[0053] Furthermore, in this modification, the calculated statistics are aggregated for each time period, thereby executing a process for identifying busy times and the flow of people during each time period. Specifically, there are places that become congested depending on the time period, such as building entrances and exits. During busy times, the route of the AGV 103 is set to avoid such places as impassable areas. By using a well-known route search algorithm such as Dijkstra's algorithm to assign costs to impassable areas and find an appropriate route, it is possible to set a route that avoids congested places. On the other hand, during less congested times, the cost can be reduced as a possible area and set it as the driving route of the AGV 103. In this way, in this modification, the operation efficiency of the mobile object can be improved by storing data on the statistics for each time period and setting possible and impassable areas according to the statistics.

[0054] <Variation 1-3> In this modification, the control content of the AGV 103 is determined according to the statistical quantity of people flow, rather than based on a comparison between the statistical quantity calculated by object analysis and a predetermined threshold. In other words, instead of making a binary judgment by comparing the evaluation value of people flow with a threshold, the control content of the AGV 103 is determined using an analog value corresponding to the degree of disadvantage to the progress of the AGV 103.

[0055] As an example, the control content of the movement speed of the AGV 103 is determined according to the density of people. The lower the density of people, the faster the movement speed of the AGV 103 is controlled, and the higher the density of people, the slower the movement speed of the AGV 103 is controlled. Also, the control content of the movement position of the AGV 103 is determined according to the density of people. Specifically, the higher the density of people, the smaller the amount of change in the movement position of the AGV 103 is controlled. In this way, in this modified example, the control content of the AGV 103 is determined continuously or stepwise according to the density of people, thereby improving operation efficiency.

[0056] <Variation 1-4> In this modification, the object analysis target is not limited to people, but can be any moving object. For example, other AGVs, forklifts, flying objects, bicycles, automobiles, etc. present around (in the nearby environment of) the AGV 103 can be analyzed. Specifically, in the object analysis of a forklift, the template matching model used in the object detection method is changed. By performing object detection using a forklift model, it is possible to analyze the forklift, and it is possible to change the object to be detected and evaluate each object. When multiple types of detection targets are mixed, it is possible to analyze the objects using the above-described method using multiple models. According to this modification, it is possible to improve the operation efficiency of moving objects by taking into account not only people but also any object.

[0057] <Variation 1-5> In the first embodiment, a captured image is divided into multiple small regions, and the control content of the AGV 103 is determined based on statistics calculated for each small region. In this modified example, the small regions are further subdivided, and the control content of the AGV 103 is determined based on statistics calculated for each smaller region. Specifically, in order for the AGV 103 to follow the flow of people in its direction of travel, a process of calculating statistics for the flow of people in the direction of travel is performed. The control content of the AGV 103 is determined based on the calculated statistics. Furthermore, when the AGV 103 guides people, a process of calculating statistics for the flow of people in the opposite direction to the direction of travel, i.e., behind the AGV 103, is performed, and the control content of the AGV 103 is determined based on the calculated statistics. This modified example enables movement control such as following or guiding a moving object, thereby improving the operation efficiency of the moving object.

[0058] [Second Example] Next, a second embodiment of the present invention will be described. In this embodiment, an example of setting a detour route when the movement of the AGV 103 is difficult due to the flow of people will be shown. Note that explanations of matters similar to those in the first embodiment will be omitted, and differences will be mainly explained. This method of omitting explanations will be the same in the embodiments and modified examples described later.

[0059] In the first embodiment, an example was shown in which the movement speed was adjusted to move closer to the flow of people when the movement direction of the people and the movement direction of the AGV 103 are similar to each other on the set movement route. In this embodiment, a method for determining the control content, including changing the route of the AGV 103, will be explained when the movement direction of the people and the movement direction of the AGV 103 differ more significantly.

[0060] In this embodiment, the movement direction, movement speed, density, and position of people flow are detected, and the movement direction of the people is compared with the movement direction of the AGV 103. If the movement direction of the people and the movement direction of the AGV 103 differ significantly, processing is performed to change the movement direction of the AGV 103. Following the change in the movement direction of the AGV 103, processing is performed to change the movement path. Then, again, the movement direction, movement position, and movement speed of the AGV 103 are controlled based on the statistics of the people flow.

[0061] 7 is a flowchart illustrating the control content determination process for the AGV 103 in this embodiment. The overall process is the same as in the first embodiment. The processes from S701 to S704 and S706 to S711 are the same as the processes from S601 to S604 and S606 to S611 in FIG. 6, so their description will be omitted.

[0062] The process proceeds from S703 to S712, where the control content determination unit 304 changes the movement direction of the AGV 103 based on the determination results of S702 and S703. This step is reached when the density result of S702 is equal to or greater than a predetermined value, and the angular difference between the movement direction of the AGV 103 and the movement direction of the person is equal to or greater than a threshold value in S703. The control content determination unit 304 changes the movement direction of the AGV 103 so as to approach the statistics of the movement direction of the person acquired in S701.

[0063] In S713, the control content determination unit 304 resets the movement route to the destination. The movement route to the destination is reset in the movement direction of the AGV 103 changed in S712. By changing the movement direction of the AGV 103 and resetting the movement route to the destination in this way, the AGV 103 can move along with the flow of people without stagnation.

[0064] In this embodiment, movement control that takes people flow into consideration is possible even if the direction of people flow differs from the movement direction of the AGV 103. By determining the control content of the AGV 103 according to the density, movement speed, and position statistics, the AGV 103 can proceed toward the destination, and its operation efficiency can be improved.

[0065] [Third Example] A third embodiment of the present invention will be described with reference to Fig. 8. In the first and second embodiments, examples were shown in which the movement direction, movement speed, and movement position were determined as control details for the AGV 103 based on each statistical quantity calculated by object analysis. In this embodiment, an example is shown in which each statistical quantity calculated by object analysis is visualized and displayed, and used to set the movement route of the AGV 103. The functional configuration of the information processing device 300 in this embodiment is a configuration in which display means (display unit 206 and display control program) is added to the functional configuration of Fig. 3 described in the first embodiment.

[0066] FIG. 8 is a schematic diagram showing an example of a UI (user interface) screen output to the display unit 206 in this embodiment. A captured image 800 captured by the imaging device 102 is displayed on the display screen. Statistics calculated by the object analysis unit 302, such as people density, movement direction, and movement speed, are superimposed on the captured image 800. The position of the AGV 103 in the captured image 800 is indicated by a black dot. A display area 802 visualizing the density evaluation value is presented to the user. In this embodiment, the distribution density of moving people or objects is displayed using shades of color. For example, darker areas indicate high density and congested areas. Multiple arrows 801 visualize and display the direction and speed of people's movement. The direction of the arrows 801 indicates the direction of people's movement, and the length of the arrows 801 indicates the speed of people's movement.

[0067] In this embodiment, each statistical amount calculated by the object analysis unit 302 is visualized and displayed superimposed on the captured image, so that the surrounding environment of the AGV 103, the congestion state, etc. can be easily grasped.

[0068] [Modification of the third embodiment] A modification of the third embodiment will now be described. <Variation 3-1> In this modification, the statistical information calculated by the object analysis unit 302 is displayed superimposed on the map information instead of the captured image. The information processing device 300 acquires the map information in advance, and performs processing to output an image in which the statistical information is superimposed on the map information to the display unit.

[0069] <Variation 3-2> In the third embodiment, an example was shown in which object analysis is performed on a current captured image captured by the imaging device 102. In this modification, statistical information calculated by the object analysis unit 302 for captured images previously acquired by the imaging device 102 is displayed superimposed on the captured image. Furthermore, the information processing device 300 can tally the calculated statistical information for each time period and display the tally results superimposed on the captured image or a map. In this way, busy time periods and the flow of people for each time period can be visualized and presented to the person monitoring the AGV 103.

[0070] <Variation 3-3> In this modification, in addition to the statistical information calculated by the object analysis unit 302, information such as the direction and speed of movement of the AGV 103, the movement route, and the movement position are superimposed on the captured image or the map, thereby allowing the person monitoring the AGV 103 to visually grasp the operation efficiency.

[0071] [Modification of the above embodiment] The AGV 103 in the above embodiment is an example of an autonomous moving object, but in a modified example, the movement of the moving object is controlled by a route guidance system, for example, using a magnetic tape as a guide. Specifically, when the direction of people flow is close to the direction of movement of the AGV 103, the AGV 103 adjusts its movement speed to the movement speed of the people and moves according to the magnetic tape. In addition, the movement position of the AGV 103 can be adjusted by installing magnetic tapes at appropriate intervals so that the movement position can be changed.

[0072] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0073] 301 Image Acquisition Unit 302 Object Analysis Department 303 Location information acquisition unit 304 Control Content Decision Unit

Claims

1. An information processing device that controls movement of a moving object, image acquisition means for acquiring an image of the moving object and its surrounding environment; a position information acquisition means for acquiring information on the position and orientation of the moving body; an analysis means for analyzing the movements of a plurality of objects present in the surrounding environment of the moving object using a plurality of images acquired at different times by the image acquisition means; a determination means for determining control details relating to the movement of the moving object based on the information on the position and orientation of the moving object acquired by the position information acquisition means and the statistics relating to the object acquired by the analysis means, The determining means determines the amount of change in the movement position of the moving body to be a first change amount when the density of the distribution of the objects is a first value, and determines the amount of change in the movement position of the moving body to be a second change amount when the density of the distribution of the objects is a second value greater than the first value.

1. An information processing device comprising:

2. The analysis means calculates statistics of the moving direction or moving speed of the object, the distribution density of the object, or the position of the object from data of a group of images acquired by the image acquisition means during a predetermined period.

2. The information processing apparatus according to claim 1, wherein:

3. The determining means determines the movement of the moving object from one or more of the statistics of the moving direction or moving speed of the object, the distribution density of the object, or the position of the object, which are analyzed by the analyzing means using the captured images acquired for each time period.

3. The information processing device according to claim 1 or 2.

4. The determining means determines the moving direction of the moving body when the density of the distribution of the object is less than a threshold value, or when the density of the distribution of the object is equal to or greater than a threshold value and the difference between the value indicating the moving direction of the object, which is the statistical quantity, and the value indicating the orientation of the moving body is less than a threshold value.

4. The information processing apparatus according to claim 3,

5. The determining means determines a position to which the moving object should be moved by evaluating a relationship between an area in the captured image where the moving object is located and an area representing the position of the object, which is the statistical quantity.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. The determining means performs a process of changing the moving position of the moving body to a position determined from the area representing the object position, which is the statistical quantity, when the area where the moving body is located is different from the area representing the object position, which is the statistical quantity.

6. The information processing apparatus according to claim 5,

7. The determining means determines the moving speed of the moving body by evaluating the moving speed of the object and the moving speed of the moving body, which are the statistical quantities.

7. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

8. The determining means calculates a speed difference between the moving speed of the object, which is the statistical quantity, and the moving speed of the moving body, and performs a process of changing the moving speed of the moving body by correcting the moving speed of the moving body in accordance with the speed difference.

8. The information processing apparatus according to claim 7,

9. The determining means changes the moving direction of the moving body and resets the moving path when the density of the distribution of the object is equal to or greater than a threshold and the difference between the value indicating the moving direction of the object, which is the statistical quantity, and the value indicating the orientation of the moving body is equal to or greater than a threshold.

9. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

10. The determining means determines the moving speed of the moving body to be a first speed when the density of the distribution of the objects is a first value, and determines the moving speed of the moving body to be a second speed greater than the first speed when the density of the distribution of the objects is a second value smaller than the first value.

10. The information processing device according to claim 1, wherein the information processing device is a computer.

11. The determining means determines a region where the moving object can proceed or a region where the moving object cannot proceed based on statistics related to the object for each time period.

11. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

12. a display means for displaying statistics of the moving direction or moving speed of the object, the distribution density of the object, or the position of the object, which are the analysis results of the analysis means; 12. The information processing device according to claim 1, wherein the information processing device is a computer.

13. An information processing device according to any one of claims 1 to 12; and an imaging means for imaging the moving object and its surrounding environment. A control system for a moving object.

14. An information processing method executed by an information processing device that controls movement of a moving object, a first acquisition step of acquiring a captured image of the moving object and its surrounding environment; a second acquisition step of acquiring information on the position and orientation of the moving object; an analysis step of analyzing the movements of a plurality of objects present in the surrounding environment of the moving object using a plurality of images acquired at different times by the first acquisition step; a determination step of determining control details relating to the movement of the moving object based on the information on the position and orientation of the moving object acquired in the second acquisition step and the statistics relating to the object acquired in the analysis step, In the determining step, when the density of the distribution of the objects is a first value, a process is performed in which the amount of change in the movement position of the moving object is determined to be a first amount of change, and when the density of the distribution of the objects is a second value greater than the first value, the process is performed in which the amount of change in the movement position of the moving object is determined to be a second amount of change smaller than the first amount of change.

1. An information processing method comprising:

15. A program that causes a computer to execute the steps according to claim 14.

Images