Control system for moving object

Abstract

To provide a control system for a moving object, which can efficiently move the moving object.SOLUTION: A control system for a moving object includes: a moving object capable of autonomous travel; area imaging means including a person detection camera disposed to capture a person detection area along a travel path from a current location to a destination of the moving object; and a server device that performs processes of: setting a person identification area within the person detection area to detect a person; calculating a number of detected persons within the person identification area based on image data captured by the area imaging means; and increasing a passage cost set for the travel path when the number of detected persons is equal to or greater than a threshold value by comparing the number of detected persons with the threshold value set for the person identification area, wherein the server device calculates the travel path of the moving object such that the passage cost is minimized.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control system for a mobile object that can move freely within a factory. [Background technology]

[0002] Conventionally, there have been autonomous mobile robots (hereinafter simply referred to as robots) that are mobile bodies configured to be able to move freely within a vast factory area without using a moving guide such as a magnetic tape. The autonomous mobile robot transports objects to their destination along a free route without interfering with the work of workers collaborating with it in the factory. The autonomous mobile robot has a sensing function for detecting obstacles around the robot, and this sensing function allows it to move to its destination without coming into contact with obstacles.

[0003] For example, the autonomous mobile robot described in Patent Document 1 is configured with a sensor device having a recognition sensor such as a laser sensor or a camera. With this configuration, the autonomous mobile robot described in Patent Document 1 can move to a destination without coming into contact with obstacles. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-156243 Summary of the Invention [Problem to be solved by the invention]

[0005] The autonomous mobile robot described in Patent Document 1 can move freely within a vast factory without coming into contact with three-dimensional objects (people or structures) by detecting obstacles using a sensing function equipped in the robot. However, the sensing function using a sensor device equipped in advance in the autonomous mobile robot can only detect three-dimensional objects around the robot, and if the route the robot is trying to take is congested with workers and makes it difficult to pass, the robot will only select a different route once it reaches the congested point and finds that it is impassable, or it will only resume moving once the congested situation at that point has been resolved, which could prevent the autonomous mobile robot from moving optimally.

[0006] The present invention has been made in view of the above-mentioned problems, and has an object to provide a control system for a mobile object that enables the mobile object to move efficiently. [Means for solving the problem]

[0007] A first aspect of the present invention includes a mobile body capable of autonomous travel, an area capture means including a person detection camera capable of capturing images of a person detection area set on a travel route from the current location of the mobile body to a destination, and a server device that sets a person identification area for detecting people within the person detection area, calculates the number of people detected in the person identification area from the image data captured by the area capture means, compares the number of people detected in the person identification area with a threshold value set in the person identification area, and, if the number of people detected is equal to or greater than the threshold value, increases the passing cost set on the travel route, wherein the server device calculates the travel route of the mobile body so that the passing cost is minimized.

[0008] According to a second aspect of the present invention, the area photographing means includes a signal light detection camera, and the signal light detection camera is arranged to be able to photograph the lighting patterns of signal lights set within a signal light detection area set on the movement route, and the server device sets a lighting identification area within the signal light detection area for detecting the lighting patterns of the signal lights, detects the lighting patterns of the signal lights within the lighting identification area from the photographed data photographed by the signal light detection camera, and when it is determined that the lighting pattern of the signal light set in the lighting identification area is lit in a lighting pattern that is different from normal, performs processing to increase the passing cost set for the movement route that passes through the signal light detection area, and calculates the movement route of the moving body so that the passing cost is minimized.

[0009] According to a third aspect of the present invention, the server device controls the operation of the autonomous mobile robot passing through the person detection area in accordance with the number of people detected in the person identification area when the autonomous mobile robot enters the person detection area, or controls the operation of the autonomous mobile robot passing through the person detection area in accordance with the lighting pattern of the signal light detected in the lighting identification area when the autonomous mobile robot enters the signal light detection area. [Effects of the Invention]

[0010] According to a first aspect of the present invention, there is provided a system including: a mobile body capable of autonomous driving; an area photographing means including a person detection camera arranged to photograph a person detection area set on a travel route from the current location of the mobile body to a destination; and a server device that sets a person identification area for detecting people within the person detection area, calculates the number of people detected within the person identification area from the photographing data photographed by the area photographing means, compares the number of people detected within the person identification area with a threshold value set in the person identification area, and, if the number of people detected is equal to or greater than the threshold value, increases the passage cost set on the travel route; the server device is configured to calculate the travel route of the mobile body so that the passage cost is minimized, thereby enabling the autonomously driven mobile body to travel to the destination efficiently.

[0011] According to a second aspect of the present invention, the area photographing means includes a signal light detection camera, and the signal light detection camera is arranged to be able to photograph the lighting patterns of signal lights set within a signal light detection area set on the movement route, and the server device sets a lighting identification area within the signal light detection area for detecting the lighting patterns of the signal lights, and detects the lighting patterns of the signal lights within the lighting identification area from the photographed data photographed by the signal light detection camera, and when it is determined that the lighting pattern of the signal light set in the lighting identification area is lit in a lighting pattern that is different from normal, performs processing to increase the passing cost set for the movement route that passes through the signal light detection area, and calculates the movement route of the moving body so that the passing cost is minimized, thereby allowing the movement of the autonomously traveling moving body to always travel along an optimal movement route without being obstructed, and to move to the destination efficiently.

[0012] According to a third aspect of the present invention, the server device controls the operation of the autonomous mobile robot passing through the person detection area in accordance with the number of people detected in the person identification area at the time the autonomous mobile robot enters the person detection area, or controls the operation of the autonomous mobile robot passing through the person detection area in accordance with the lighting pattern of the signal light detected in the lighting identification area at the time the autonomous mobile robot enters the signal light detection area, thereby optimizing the movement of the mobile object within the person detection area or signal light detection area and allowing the mobile object to move efficiently to its destination. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing a schematic configuration of a control system for a moving body according to an embodiment of the present invention; [Figure 2] 1 is a diagram showing the internal configuration of an autonomous mobile robot according to an embodiment of the present invention; [Figure 3] FIG. 2 is a diagram showing the internal configuration of a three-dimensional object detection means according to one embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating an internal configuration of a server device according to an embodiment of the present invention. [Figure 5] 1 is a diagram showing a movement path of a moving body according to an embodiment of the present invention; [Figure 6] FIG. 2 is a diagram showing a human detection area and an identification area according to an embodiment of the present invention. [Figure 7] 1 is a diagram showing a signal light detection area and an identification area according to an embodiment of the present invention. FIG. [Figure 8] FIG. 2 is a diagram showing a control flow of a control system for a moving body according to an embodiment of the present invention. [Figure 9] FIG. 10 is a diagram illustrating a method for calculating a passage cost in a mobile object control system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] The present invention relates to a control system M for a mobile body that can travel autonomously from a current location to a destination, and that, by devising information provided to the mobile body, makes the movement of the mobile body to the destination efficient while reducing the risk of contact between the mobile body and a person on the travel route as much as possible. Note that in this embodiment, the mobile body will be described as an autonomous traveling robot 1.

[0015] FIG. 1 is a diagram showing a schematic configuration of a mobile object control system M. FIG. 2 is a diagram showing the internal configuration of an autonomous mobile robot 1. FIG. 3 is a diagram showing the internal configuration of an area photographing means 2. FIG. 4 is a diagram showing the internal configuration of a server device 3. FIG. 5 is a diagram showing the photographing areas (person detection area 40, traffic light detection area 50) of the person detection camera 21 and the traffic light detection camera 22 that constitute the area photographing means 2, and the movement route of the autonomous mobile robot 1. FIG. 6 is a diagram showing the photographing area R and the identification area D of the person detection camera 21. FIG. 7 is a diagram showing the photographing area R and the identification area D of the traffic light detection camera 22. FIG. 8 is a diagram showing the control flow of the mobile object control system M. FIG. 9 is a diagram showing a method for calculating the passage cost of a movement route that passes through a person detection area.

[0016] As shown in FIG. 1 , the mobile object control system M includes an autonomous mobile robot 1 as a mobile object capable of autonomous travel; an area photographing unit 2 that photographs people within a person detection area 40 set on the movement path of the autonomous mobile robot 1 and photographs the lighting patterns of signal lights 51 within a signal light detection area 50 set on the movement path of the autonomous mobile robot 1; and a server device 3 that controls the movement of the autonomous mobile robot 1 based on the number of people within the person detection area 40 photographed by the area photographing unit 2. The autonomous mobile robot 1 and the server device 3, as well as the area photographing unit 2 and the server device 3, are connected to each other via a network environment W. The network environment W is configured to include all or part of the Internet, a wireless LAN (Local Area Network), and short-range wireless communication. The wireless LAN may be, for example, compliant with Wi-Fi (registered trademark). The short-range wireless communication line may be, for example, compliant with Bluetooth (registered trademark).

[0017] The autonomous mobile robot 1 includes a robot body 1a, and its operation is controlled by a robot control unit 11 housed in or fixed to the robot body 1a. The robot body 1a may be formed in any shape suitable for movement. The autonomous mobile robot 1 is, for example, a robot for cleaning, security, guidance, carrying luggage, or transporting people, and in this embodiment, a robot for carrying luggage will be described as an example.

[0018] As shown in FIG. 2, the autonomous mobile robot 1 includes a robot control unit 11, a memory 12, a communication processing unit 13, a surrounding information detection unit 14, an actuator unit 15, a position processing unit 16, and an interface unit 17.

[0019] The robot control unit 11 has an arithmetic processing unit 11a including a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The memory 12 includes a non-volatile memory such as a ROM (Read Only Memory) or a flash memory, and a volatile memory such as a RAM (Random Access Memory). The memory 12 may also be a memory built into the robot control unit 11. The memory 12 stores building floor plan data, landmark data, and the like. The robot control unit 11 executes programs stored in the memory 12 using the arithmetic processing unit 11a, thereby implementing various functions of the robot control unit 11. The communication processing unit 13 functions as a communication means for implementing two-way communication with the server device 3 via the network environment W.

[0020] The surrounding information detection unit 14 simultaneously estimates the self-position of the autonomous mobile robot 1 and creates an environmental map of the inside of the building, and is equipped with a sensor camera 14a that acquires surrounding information of the autonomous mobile robot 1 and a distance measurement sensor 14b that measures the distance between the autonomous mobile robot 1 and surrounding objects. The autonomous mobile robot 1 acquires surrounding information at a predetermined cycle using the sensor camera 14a and the distance measurement sensor 14b, and the acquired surrounding information is transmitted to the robot control unit 11.

[0021] The sensor camera 14a has a shooting area based on the robot main body 1a. The sensor camera 14a grasps the surrounding environment based on shooting data obtained by shooting the shooting area and extracts feature points of objects. The sensor camera 14a is configured to estimate its own position and create an environmental map by detecting changes in feature points in the shooting data. The shooting area of ​​the sensor camera 14a includes at least the area in front of the robot main body 1a. The shooting area of ​​the sensor camera 14a may also be configured to include left and right areas and a rear area of ​​the robot main body 1a. Note that the front-rear and left-right directions of the autonomous mobile robot 1 are determined in relation to the structure of the robot main body 1a, with the moving direction of the autonomous mobile robot 1 considered to be the front, and both sides of the traveling direction being considered the left and right directions. Note that the sensor camera 14a in this embodiment is provided on the front of the robot main body 1a.

[0022] The distance measurement sensor 14b is mounted on the robot body 1a and is used to measure distance information between the robot body 1a and surrounding three-dimensional objects. The distance measurement sensor 14b transmits the measured distance information relating to the surrounding three-dimensional objects to the robot control unit 11. The distance measurement sensor 14b is configured by a LiDAR (Light Detection and Ranging) that measures distance using laser light, or a radar that measures distance using radio waves. The distance measurement sensor 14b may also be configured by combining LiDAR and radar. In this embodiment, the distance measurement sensor 14b is provided near the lower end of the front surface of the robot body 1a.

[0023] The actuator unit 15 is equipped with various actuators for driving the robot body 1a. The various actuators are controlled by the robot control unit 11. Driving the robot body 1a includes moving and turning the robot body 1a. Moving a part of the robot body 1a also corresponds to driving the robot body 1a. In this embodiment, the autonomous mobile robot 1 has wheels for driving at the left and right lower ends of the robot body 1a, and is configured so that the rotation speed of the left and right wheels can be controlled independently. This allows the autonomous mobile robot 1 to freely change the orientation of the robot body 1a and move not only forward and backward but also left and right.

[0024] The position processing unit 16 acquires the position information of the autonomous mobile robot 1 using a technology such as SLAM (Simultaneous Localization and Mapping) that simultaneously performs self-position estimation and environmental mapping. The position processing unit 16 compares the surrounding information of the autonomous mobile robot 1 acquired by the sensor camera 14a and distance measurement sensor 14b of the surrounding information detection unit 14 with building floor plan data, landmark data, etc. stored in advance in the memory 12 to estimate the self-position of the autonomous mobile robot 1. Here, "self-position estimation" refers to recognizing the autonomous mobile robot's current location and the direction of movement, and "environmental mapping" refers to understanding the surrounding environment of the autonomous mobile robot 1 and creating a map. The position information of the autonomous mobile robot 1 acquired by the position processing unit 16 is transmitted to the server device 3 via the communication processing unit 13.

[0025] The interface unit 17 is a man-machine interface between the person using the autonomous mobile robot 1 and the autonomous mobile robot 1. The interface unit 17 includes a microphone, a speaker, a display screen, etc., and various operations of the autonomous mobile robot 1 can be set via the interface unit 17.

[0026] The area photographing means 2 controls the movement of the autonomous mobile robot 1 by detecting three-dimensional objects such as people and the lighting patterns of signal lights 51 on the movement path P of the autonomous mobile robot 1. As shown in Fig. 3 , the area photographing means 2 has a person detection camera 21 that detects the number of people in a person detection area 40 set on the movement path P, and a signal light detection camera 22 that photographs the lighting patterns of the signal lights 51 set in the signal light detection area 50.

[0027] The person detection camera 21 has an imaging unit 21a that captures an image of a part of a person detection area 40 set on the movement path P of the autonomous mobile robot 1 as a capturing area R, a memory 21b that temporarily stores the image data captured by the imaging unit 21a, and a communication unit 21c that transmits the image data stored in the memory 21b to the server device 3. The image data captured by the person detection camera 21 is transmitted to the server device 3 via a network environment W. The person detection camera 21 can be installed in any position on the movement path P, such as on the ceiling or wall, as long as it is installed so that a part of the person detection area 40 becomes the capturing area R. Multiple person detection cameras 21 are installed for the same person detection area 40, and in this embodiment, three person detection cameras 21 are installed for the same person detection area 40.

[0028] The signal light detection camera 22 has the same configuration as the person detection camera 21, and includes an imaging unit 22a that captures an image of a part of a signal light detection area 50 set on the movement path P of the autonomous mobile robot 1 as a shooting region R, a memory 22b that temporarily stores the image data captured by the imaging unit 22a, and a communication unit 22c that transmits the image data stored in the memory 22b to the server device 3. The signal light detection camera 22 is installed so that a part of the signal light detection area 50 set on the movement path P of the autonomous mobile robot 1 is set as the shooting region R, and can be installed in any position such as on the ceiling or wall of the movement path P as long as it can detect the signal lights 51 set in the signal light detection area 50. A plurality of signal light detection cameras 22 are installed for the same signal light detection area 50, and in this embodiment, three signal light detection cameras 22 are installed for the same signal light detection area 50.

[0029] As shown in Fig. 4, the server device 3 has a server control unit 31, memory 32, a communication processing unit 33, an obstacle determination unit 34, and a path calculation unit 39. The server device 3 functions as a robot management device for controlling the operation of the autonomous mobile robot 1 together with the surrounding information detection unit 14. The server device 3 is composed of one or more computers connected to a network environment W. The server device 3 may be formed using cloud computing, or may be built into the autonomous mobile robot 1. When the server device 3 is built into the autonomous mobile robot 1, the autonomous mobile robot 1 and the area photographing means 2 are connected via the network environment W.

[0030] The server control unit 31 has an arithmetic processing unit 31a including a CPU, a GPU, etc. The memory 32 includes non-volatile memory such as ROM or flash memory, and volatile memory such as RAM. The memory 32 may also be memory built into the server control unit 31. The server control unit 31 executes programs stored in the memory 32 using the arithmetic processing unit 31a, thereby implementing each function of the server control unit 31. The communication processing unit 33 realizes two-way communication with a counterpart device via the network environment W. The counterpart device (the counterpart device connected via the communication processing unit 33) for the server device 3 includes the autonomous mobile robot 1 and the area photographing means 2.

[0031] The obstacle determination unit 34 is a part that determines the number of people detected within the person detection area 40 and the lighting pattern of the signal lights 51 installed within the signal light detection area 50 based on the video data acquired from the area shooting means 2 via the communication processing unit 33, and varies the passage cost that has been set in advance for the movement route P of the autonomous mobile robot 1 based on each determination result. The obstacle determination unit 34 has an image recognition unit 35, an image data collection unit 36, a model learning unit 37, and a passage cost processing unit 38.

[0032] The image recognition unit 35 is a part that determines the number of people detected in the photographic data captured by the person detection camera 21 of the area photographing means 2 via the communication processing unit 33, and the lighting pattern of the signal light 51 in the photographic data obtained from the signal light detection camera 22.

[0033] The determination of the number of people detected by the image recognition unit 35 in the image capture data of the person detection cameras 21 will be described with reference to FIG. 6. The image recognition unit 35 determines the number of people detected based on image capture data captured by at least two or more person detection cameras 21. In this embodiment, the determination is made using three person detection cameras 21. As shown in FIG. 6, each person detection camera 21 has an image capture area R1, R2, or R3 that captures a portion of the person detection area 40. Each image capture area R1, R2, or R3 has a person identification area D1, D2, or D3 that is used to determine the number of people detected by the person detection cameras 21. The person identification areas D1, D2, or D3 are set arbitrarily in the image capture areas R1, R2, or R3. These person identification areas D1, D2, or D3 are set arbitrarily in the image capture areas R1, R2, or R3. These person identification areas D1, D2, or D3 are set at arbitrary positions on the movement route P of the autonomous mobile robot 1. For example, suppose a person detection area 40 is set at an intersection on the path of the autonomous mobile robot 1, and the image capture areas R1, R2, and R3 of the person detection cameras 21 are set within the person detection area 40. The image capture areas R1, R2, and R3 are set so that the entire intersection can be detected. Among these, a person identification area D (D1, D2, and D3) is set to include the center of the intersection where a person is expected to pass. The person identification area D (D1, D2, and D3) can be set arbitrarily for each person detection camera 21. For example, in FIG. 6 , the person detection camera 21A sets the person identification area D1 at the center of the image capture area R1 because the center of the intersection is located at the center of the image capture area R1. The person detection camera 21B sets the person identification area D2 near the left edge of the image capture area R2 because the center of the intersection is located near the left edge of the image capture area R2. The person detection camera 21C sets the person identification area D3 near the right edge of the image capture area R3 because the center of the intersection is located near the right edge of the image capture area R3. In this way, by setting the part of the set shooting areas R1, R2, R3 that is most likely to be passed through by people as person identification area D (D1, D2, D3), it is possible to prevent erroneous detection of the number of people detected in shooting area R. Furthermore, by setting person identification areas D1, D2, D3 in person detection cameras 21A, 21B, 21C installed at different angles, it is possible to prevent erroneous detection of the number of people in person identification area D (D1, D2, D3).The data on the number of people detected within the person identification area D (D1, D2, D3) determined by the image recognition unit 35 is transmitted to the image data collection unit .

[0034] Next, the determination of the lighting pattern of the signal light 51 by the image recognition unit 35 in the image capture data of the signal light detection cameras 22 will be described with reference to FIG. 7. The image recognition unit 35 determines the lighting pattern of the signal light 51 based on data captured by at least two or more signal light detection cameras 22. In this embodiment, the determination is made using three signal light detection cameras 22. As shown in FIG. 7, each signal light detection camera 22 has an image capture area R4, R5, or R6 that captures a portion of the signal light detection area 50. Each image capture area R4, R5, or R6 has an image capture identification area H (H1, H2, or H3) for detecting the lighting pattern of the signal light 51 detected by the signal light detection camera 22. The image capture identification area H (H1, H2, or H3) is installed in a position where the signal light 51 can be confirmed within the image capture area R4, R5, or R6. By installing the signal light detection camera 22 in this way so that it captures the lighting patterns of the signal lights 51 installed in the signal light detection area 50 from different angles, it is possible to prevent the lighting patterns of the signal lights 51 from being detected incorrectly.

[0035] The image data collection unit 36 ​​is a unit that temporarily stores the person detection number data and data determining the lighting pattern of the signal light 51 used by the image recognition unit 35. In other words, the image data collection unit 36 ​​stores, as training data, the image data captured by the person detection camera 21 of the area imaging means 2 in which it has been determined by the image recognition unit 35 that a person is present in the person identification region D, and the image data captured by the signal light detection camera 22 in which it has been determined that a person is lit in an unusual lighting pattern in the lighting identification region H. The photographed data stored in the image data collection unit 36 ​​is transmitted to the model learning unit 37.

[0036] The model learning unit 37 separates the training data stored in the image data collection unit 36 ​​into two types of model data, person detection model data 37a and signal light recognition model data 37b, and extracts features from each of the stored model data. The model learning unit 37 extracts features for identifying a person from the acquired model data and feeds back the extracted features to the image recognition unit 35. Similarly, the model learning unit 37 extracts features for recognizing the lighting patterns of signal lights from training data for identifying lighting patterns and feeds back the extracted features to the image recognition unit 35. In this way, the model learning unit 37 extracts features for determining a person from training data for identifying a person and extracts feature data from training data for identifying the lighting patterns of signal lights 51 and feeds back the accumulated feature data to the image recognition unit 35, thereby improving the accuracy of person identification and the accuracy of identifying the lighting patterns of signal lights 51 in the image recognition unit 35. In other words, by transmitting each feature amount extracted by the model learning unit 37 to the image recognition unit 35, it is possible to reduce false detections in the image recognition unit 35 and make the control signal transmitted from the server device 3 to the autonomous mobile robot 1 appropriate. Information related to the number of people detected determined by the image recognition unit 35 is transmitted to the passage cost processing unit 38.

[0037] The passing cost processing unit 38 calculates a passing cost set for the movement route P of the autonomous mobile robot 1 based on the number of people detected in the person identification region D of the person detection area 40 determined by the image recognition unit 35 and the lighting pattern of the signal lights 51 in the signal light detection area 50 detected by the image recognition unit 35. The passing cost processing unit 38 performs processing to increase the set passing cost when it is determined that there is a person in the person identification region D of the person detection area 40 because the number of people detected by the image recognition unit 35 is equal to or greater than a threshold set for each person detection area 40. Alternatively, the passing cost processing unit 38 performs processing to increase the passing cost set for the movement route P when the image recognition unit 35 of the obstacle determination unit 34 detects that the signal lights 51 in the signal light detection area 50 are lit with an unusual lighting pattern. In this way, the passing cost information increased by the passing cost processing unit 38 is transmitted to the route calculation unit 39 of the server device 3. Details of the passing cost processing by the passing cost processing unit 38 will be described later.

[0038] The route calculation unit 39 is a part that sets a plurality of travel routes P from the current location to the destination using an interface provided in the server device 3, and determines the shortest route from the current location to the destination among the set travel routes P. The route calculation unit 39 determines the shortest route to the destination using, for example, the Dijkstra algorithm.

[0039] The route calculation unit 39 sets a plurality of possible travel routes P to the goal using the interface of the server device 3, sets nodes N (waypoints) at arbitrary intervals on each route, and sets the straight-line distance between each node N on edges E connecting the nodes N with straight lines as a passing cost (ease of travel). Of the multiple travel routes P set on routes connecting the current location to the destination, the route calculation unit 39 sets the route that has the smallest sum of the passing costs set on each edge E from the current location to the destination as the initial travel route P. In other words, the initial travel route P set by the route calculation unit 39 before the autonomous mobile robot 1 starts traveling is selected as the shortest route from the current location to the destination. In this embodiment, a person detection area 40 is set on the travel route P, and the passing cost of the edge E set to pass through the person detection area 40 is set to vary depending on the number of people detected within the person detection area 40. In this way, by setting a variable passing cost for each edge E on the travel route P, the easiest travel route P to pass through can be set by referring to the number of people detected in the person detection area 40, and the autonomous mobile robot 1 can be moved efficiently to its destination.

[0040] The fault determination unit 34 is configured to have the image recognition unit 35 receive feedback of information related to the feature amounts of people extracted by the model learning unit 37 and information related to the feature amounts of the lighting pattern of the signal light 51, thereby improving the accuracy of determining that a person is present in the person identification area D of the person detection camera 21 received from the area photographing means 2. In other words, when the image recognition unit 35 determines that a person is present, it is possible to reduce erroneous detections such as determining that a person is present when it is not a person, or determining that a person is not present when it is a person. Furthermore, when the image recognition unit 35 determines the lighting pattern of the signal light 51, it is possible to determine which lighting pattern the signal light 51 is lit in even if part of the signal light 51 is hidden by some kind of three-dimensional object, thereby improving the accuracy of determining the lighting pattern.

[0041] Furthermore, the obstacle determination unit 34 calculates, in the image recognition unit 35, the average value (or median) of the number of detected three-dimensional objects (people, objects) (in this embodiment, the number of detected people) detected within the person identification area D set by the multiple person detection cameras 21 (cameras A to C). If the calculated average value (or median) is equal to or greater than a threshold value set in advance for each person detection area 40, it is determined that the three-dimensional object (person, object) is obstructing the movement of the autonomous mobile robot 1, and information is transmitted from the image recognition unit 35 to the passing cost processing unit 38 to instruct the image recognition unit 35 to perform processing to increase the passing cost set for the edge E of the set movement route P that passes through that person detection area 40. The obstacle determination unit 34 transmits the passing costs set for each edge E of the movement route P from the passing cost processing unit 38 to the route calculation unit 39. When the route calculation unit 39 receives information that the passing cost set for each edge E of the movement route P has changed, it sets the movement route P to the route that has the smallest passing cost from the current location to the destination, from among the set movement routes P. The set movement route P is transmitted to the autonomous mobile robot 1. The autonomous mobile robot 1 moves along the received travel route P to the destination.

[0042] Referring to FIG. 9 , a process for increasing the passing cost set for an edge E passing through a person detection area 40 when the number of detected people exceeds a threshold set in the person identification region D of the person detection area 40 will be described. In FIG. 9 , the symbol D indicates an ellipse D inscribed in the person detection area 40 set on the movement path P of the autonomous mobile robot 1. The symbol L indicates the shortest distance between the edge E passing through the person detection area 40 and the center C of the ellipse D, and this distance is referred to as "weight W1." The weight W1 is set to be maximum when the edge E passes through the center C, and its influence decreases as the edge E moves away from the center C. The weight W1 is set to be inversely proportional to the shortest distance L.

[0043] Furthermore, the symbol r indicates the relative angle between edge E, which connects nodes NA and NB, located at the front and rear positions of the movement path P passing through the person detection area 40, and the line connecting node NA, located on the upstream side of the movement path P, and the center C of ellipse D; this value is referred to as "weight W2." Weight W2 increases as relative angle r approaches 0 degrees, and the passing cost becomes 0 when relative angle r reaches 180 degrees. In other words, when relative angle r is 0 degrees, edge E passes through center C, and when relative angle r is 180 degrees, center C is located on the extension line of edge E, and edge E moves in a direction away from center C. Note that weights W1 and W2 are both set as percentages. The passing cost of edge E passing through person detection area 40 is calculated using weights W1 and W2 according to the following formula: Passing cost = Straight-line distance between nodes NA and NB + a * (weight 1) + b * (weight 2) + c * number of people detected + d * (elapsed time t) (Equation 1) The symbols a, b, c, and d in Equation 1 represent coefficients that can be set arbitrarily. The elapsed time t in Equation 1 represents the time that has elapsed since the number of people detected in person identification area D was detected to be equal to or greater than the threshold value.

[0044] In this way, the passing cost set for edge E connecting nodes NA and NB passing through person detection area 40 is increased by the shortest distance L (weight 1) connecting the center C of ellipse D and the edge, the relative angle r (weight 2) between the line connecting node NA located on the entry side of person detection area 40 and the ellipse center C and the line connecting nodes NA and NB, the number of people detected in person identification area D of person detection area 40, and the elapsed time from the time (detection time) when it is detected that the number of people detected in person identification area D of person detection area 40 is greater than or equal to the threshold set in person identification area D of person detection area 40 is in person identification area D to the current time.

[0045] In this way, by configuring the path calculation unit 39 to be able to recalculate the movement path of the autonomous mobile robot 1 based on the average (or median) number of people detected within the person identification area D set by multiple person detection cameras 21, the autonomous mobile robot 1 can be moved efficiently to its destination.

[0046] As shown in Fig. 7, the lighting pattern of the signal light 51 is determined by the fault determination unit 34 from the image capture data of the lighting identification area H (H1, H2, H3) set within each of the image capture areas R (R4, R5, R6) of the three signal light detection cameras 22 (22A, 22B, 22C) installed in the signal light detection area 50. The fault determination unit 34 determines the lighting pattern of the signal light 51 within the lighting identification area H (H1, H2, H3) for each piece of image capture data it receives. The fault determination unit 34 determines the most common lighting pattern in the image capture data as the lighting pattern of the signal light 51. For example, of the lighting identification areas H (H1, H2, H3) of the three signal light detection cameras 22, in lighting identification areas H1 and H2 it is determined that the signal light 51 is lit in a lighting pattern that stops the autonomous mobile robot 1, and in lighting identification area H3, if a three-dimensional object (such as luggage or a person) placed on the movement path prevents the entire image of the signal light 51 from being seen and its lighting pattern cannot be confirmed, it is determined that the signal light 51 is lit in a lighting pattern that stops the autonomous mobile robot 1. In other words, when determining the lighting pattern of the signal light 51 in the signal light detection area 50, the lighting pattern of the signal light 51 that is determined to have the greatest number of lighting patterns among the lighting patterns of the signal light 51 detected by the signal light detection cameras 22 set in the signal light detection area 50 is determined to be the lighting pattern of that signal light 51.

[0047] Here, the signal light 51 is a stacked signal light in which LEDs of multiple colors are stacked. In this embodiment, the signal light 51 is a signal light in which green, yellow, and red LEDs are stacked from bottom to top. Because the signal light 51 has a configuration with multiple colors, the autonomous mobile robot 1 can be controlled in a variety of patterns by combining each color with a lighting pattern.

[0048] A method for resetting (rerouting) the movement path P of the autonomous mobile robot 1 by the path calculation unit 39 will be described using FIG. 5. In FIG. 5, symbols P1 and P2 indicate possible movement paths from the current location to the destination. Symbols 50A and 50B indicate signal light detection areas. Symbols 40A, 40B, and 40C indicate person detection areas. Symbol S indicates the start point (current location) of the movement of the autonomous mobile robot 1, and symbol G indicates the destination (destination) of the autonomous mobile robot 1. Small white circles shown on the movement paths of symbols P1 and P2 indicate node N, which is a waypoint in the Dijkstra algorithm. Symbol E indicates an edge connecting nearby nodes. A passing cost is set for each edge E according to the straight-line distance.

[0049] 5, when the image recognition unit 35 of the obstacle determination unit 34 detects a number of detected people equal to or greater than a threshold value in the person identification region D of the person detection area 40A, the server device 3 calculates the passing cost of an edge E1 passing through the person detection area 40A in a movement route P1 that passes through the person detection area 40A using a passing cost processing unit 38. The recalculated passing cost information for the edge E1 is transmitted to a route calculation unit 39 of the server device 3, which then recalculates a movement route from the current location to the destination. The autonomous mobile robot 1 selects a movement route with the smallest passing cost from the movement routes recalculated by the route calculation unit 39 and moves along that movement route. In other words, when the autonomous mobile robot 1 determines that the number of detected people in the person identification region D set in the person detection area 40 is equal to or greater than the threshold value set for the person identification region D, it changes the passing cost set for the edge E of the movement route P, calculates the passing cost of each changed movement route, and moves along the movement route with the smallest passing cost. In this way, by setting a threshold value for the person identification area D and calculating the optimal route when a number of people equal to or greater than the threshold value is detected, the autonomous mobile robot 1 can always move along the optimal route.

[0050] In this way, the control system M of the autonomous mobile robot 1 described in this embodiment is configured to vary the passage cost of the movement route P of the autonomous mobile robot 1 from its current location to its destination depending on the number of people detected within the person identification region D of the person detection area 40, which is set as the movement route P, thereby enabling the autonomous mobile robot 1 to move efficiently to its destination.

[0051] In this embodiment, a configuration is described in which the server device 3 is provided separately from the autonomous mobile robot 1, but the present invention is not limited to this, and the server device 3 may be provided within the autonomous mobile robot 1. In this way, by providing the server device 3 within the autonomous mobile robot 1, the autonomous mobile robot 1 can be controlled without having to install a new server device 3 in the factory.

[0052] <<Control flow of control system M of autonomous mobile robot 1>> The control system M of the autonomous mobile robot 1 is configured as described above, and the control flow for the autonomous mobile robot 1 to automatically move from its current location to its destination will be described with reference to Fig. 8. The movement of the autonomous mobile robot 1 is controlled roughly in two steps: a transport preparation step T1 and a transport step T2.

[0053] <Transportation preparation process T1> The following describes the transfer preparation step T1 for the autonomous mobile robot 1 to transfer an article. The transfer preparation step T1 is made up of 10 steps, from step S1 to step S10.

[0054] The worker places the transported object in the robot body 1a (step S1).

[0055] The autonomous mobile robot 1 carrying the transported item compares the information acquired by the sensor camera 14a and the distance measurement sensor 14b of the surrounding information detection unit 14 with the floor plan data of the building stored in advance in the memory 12 to estimate its own position and determine its current position (step S2).

[0056] The current position information of the autonomous mobile robot 1 whose self-position has been estimated is transmitted together with the floor plan data of the building to the server device 3 via the network environment W (step S3).

[0057] Current position information of the autonomous mobile robot 1 and destination information of the autonomous mobile robot 1 are input to the route calculation unit 39 of the server device 3, and multiple travel routes P from the current location to the destination are set using the interface of the server device 3 (step S4). Note that the travel route P set in step S4 can be set manually to any route.

[0058] Nodes N are set as waypoints on each of the multiple travel routes P set by the route calculation unit 39 (step S5). When nodes N are set on each travel route, the edges between the nodes and the passing costs of the edges are automatically set. Note that the passing cost is the distance between adjacent nodes N, and is therefore determined by referring to the plan view data stored in advance in the memory 12.

[0059] The travel route information set by the route calculation unit 39 of the server device 3 is transmitted to the autonomous mobile robot 1 (step S6). The server device 3 sets a person detection area 40 on the set travel route (step S7). The person detection area 40 is set at an appropriate position by an operator using the interface of the server device 3. The person detection area 40 is set near an area where blind spots are likely to occur when the autonomous mobile robot 1 moves, such as near a crossroads, T-junction, or Y-junction. The person detection area 40 may also be set at a crossroads, T-junction, or Y-junction.

[0060] Three person detection cameras 21 installed near the set person detection area 40 are selected, and a person identification area D is set in the shooting area R of each person detection camera 21 (step S8). The person identification area D is set at an appropriate position by an operator using the interface of the server device 3. The person identification area D set for each person detection camera 21 is saved in the memory 32 of the server device 3.

[0061] Furthermore, signal light detection areas 50 (50A, 50B) are set separately from the person detection areas 40 (40A, 40B, 40C) set by the server device 3 (step S9). The person detection areas 40 and the signal light detection areas 50 can also be set at the same location on the movement route P.

[0062] Three signal light detection cameras 22 installed near the set signal light detection area 50 are selected, and a light-on identification area H is set in the shooting area R of each signal light detection camera 22 (step S10). The light-on identification area H for each signal light detection camera 22 is set by the image recognition unit 35 comparing data that stores feature amounts for recognizing a signal light 51, which is stored in signal light model data that has been learned in advance by the model learning unit 37 of the server device 3 from the shooting area R of each signal light detection camera 22, with the image data of the signal light 51 photographed by the signal light detection camera 22. However, the light-on identification area H for each signal light detection camera 22 can also be set manually using the interface of the server device 3. The light-on identification area D set for each signal light detection camera 22 is stored in the memory 32 of the server device 3.

[0063] The transport preparation step T1 for the autonomous mobile robot 1 is configured in this way, and after the processing up to step S10 is completed, the autonomous mobile robot 1 starts moving to the destination.

[0064] <Transportation process> The transport process T2 is made up of eight steps, steps S11 to S18, as shown in Fig. 8. When the autonomous mobile robot 1 completes the process of moving to the destination in the transport process T2, it waits at the nearest waiting location from its current location until control information is input again.

[0065] After the lighting identification area H is set in advance in the signal light detection camera 22 installed in the signal light detection area 50 in step S10, the actuator unit 15 is driven to move the autonomous mobile robot 1 along the movement route set in step S4 that has the smallest passing cost (step S11).

[0066] When the autonomous mobile robot 1 starts moving, the server device 3 periodically determines whether the number of people detected within the person identification region D set in the multiple person detection areas 40 of the multiple person detection cameras 21 constituting the area photographing means 2, which are provided on the movement route, is equal to or greater than a preset threshold, or whether the lighting pattern of the signal lights 51 within the lighting identification region H set in the signal light detection area 50 is other than a normal lighting pattern (step S12). As described above, the server device 3 determines the number of people detected within the person identification region D of the three person detection cameras 21 installed to photograph the set person detection area 40 based on the average (or median (hereinafter also referred to as the determination value)). Furthermore, as described above, the server device 3 determines the lighting pattern of the signal lights 51 based on the most common lighting pattern among the lighting patterns detected within the lighting identification region H of the three signal light detection cameras 22 installed to photograph the set signal light detection area 50 as the lighting pattern of the signal light 51.

[0067] If the judgment value of the person detection area 40 is equal to or greater than a threshold value set in advance for each person detection area 40, or if it is judged that the signal light detection area 50 does not have a normal lighting pattern, the passage cost processing unit 38 of the obstacle judgment unit 34 calculates and resets the passage cost of the movement route set in step S5 according to the calculation formula of Equation 1 (step S13). Note that the judgment of the number of people detected in the person detection area 40 and the judgment of the lighting pattern in the signal light detection area 50 are judged at a predetermined cycle, and even if the two judgments occur simultaneously in step S12, the process proceeds to step S13.

[0068] After resetting the passage costs associated with the nodes N set for each movement route P, the path calculation unit 39 uses the reset passage costs to calculate the movement route P with the minimum passage cost (step S14). The autonomous mobile robot 1 moves along the set movement route P (step S15).

[0069] Thereafter, the server device 3 compares the self-position estimation of the autonomous mobile robot 1 with the floor plan data of the building to determine whether or not the autonomous mobile robot 1 has arrived at the destination (step S16). In step S16, when the autonomous mobile robot 1 has arrived at the destination, the control of the autonomous mobile robot 1 in the transport process T2 ends. If the autonomous mobile robot 1 has not arrived at the destination, the server device 3 determines whether or not the autonomous mobile robot 1 has entered the person detection area 40 or the traffic light detection area 50 (step S17).

[0070] If it is determined in step S17 that the autonomous mobile robot 1 is present in the person detection area 40, the obstacle determination unit 34 of the server device 3 determines the number of people detected in the person identification region D of the person detection area 40 (step S18). The movement of the autonomous mobile robot 1 within the person detection area 40 is controlled according to the number of people within the person identification region D determined in step S18 (step S19). Controlling the movement of the autonomous mobile robot 1 in step S19 means that if the number of people detected within the person identification region D of the person detection area 40 is zero, the autonomous mobile robot 1 travels through the person detection area 40 without changing its speed, and if the number of people detected is one or more, the autonomous mobile robot 1 travels through the person detection area 40 at a reduced speed, or temporarily stops upon entering the person detection area 40, or issues an alert to notify the autonomous mobile robot 1 that it is about to pass through. The autonomous mobile robot 1 repeats the processes of steps S17 to S19 until it passes through the person detection area 40. In addition, the temporary suspension of the autonomous mobile robot 1 and the alarm control in step S19 are only performed when the autonomous mobile robot 1 first enters the person detection area 40, and only the speed control of the autonomous mobile robot 1 is performed while the autonomous mobile robot 1 is moving within the person detection area 40.

[0071] Furthermore, if the autonomous mobile robot 1 enters the signal light detection area 50 in step S17, the lighting pattern of the signal lights 51 in the lighting identification area H is determined by the fault determination unit 34 of the server device 3 (step S18). The movement of the autonomous mobile robot 1 is controlled according to the lighting pattern of the signal lights 51 in the lighting identification area H determined in step S18 (step S19). Controlling the movement of the autonomous mobile robot 1 in step S19 means that if the lighting pattern of the signal lights 51 in the signal light detection area 50 is a normal lighting pattern, the autonomous mobile robot 1 travels through the signal light detection area 50 without changing its moving speed, and if the lighting pattern of the signal lights 51 is other than the normal lighting pattern, the autonomous mobile robot 1 travels through the signal light detection area 50 at a reduced moving speed, or temporarily stops or issues an alert upon entering the signal light detection area 50 and then passes through. In addition, the temporary suspension of the autonomous mobile robot 1 and the alarm control in step S19 are performed only when the autonomous mobile robot 1 first enters the traffic light detection area 50, and only the speed control of the autonomous mobile robot 1 is performed while the autonomous mobile robot 1 is moving within the traffic light detection area 50.

[0072] In step S12, the determination of the number of people detected within the person detection area 40 and the determination of the lighting pattern of the signal lights 51 within the signal light detection area 50 are made independently. For example, if the lighting pattern of the signal lights 51 is changed while the number of people detected is being determined, the obstacle determination unit 34 performs a process to change the passage cost, and the route calculation unit 39 calculates a movement route from the current location to the destination taking into account the change information of the passage cost. Conversely, if the number of people detected within the person detection area 40 exceeds the threshold value while detecting a change in the lighting pattern of the signal lights 51, the obstacle determination unit 34 performs a process to change the passage cost, and the route calculation unit 39 calculates a movement route from the current location to the destination taking into account the change information of the passage cost. In this way, the processing from step S12 to step S19 is executed at a predetermined interval while the autonomous mobile robot 1 is traveling from its current location to its destination. When the number of people detected in the person detection area 40 exceeds the threshold value set therein, or when the signal light 51 lights up in an abnormal pattern, the cost of passing through the travel route is calculated, and a decision is made as to whether the travel route to the destination should be changed.

[0073] The autonomous mobile robot 1 repeats the processes from step S12 to step S19 until it arrives at the destination. When the autonomous mobile robot 1 estimates its own position and determines that it has arrived at the destination, it ends the movement process of the transport step T2. When the autonomous mobile robot 1 confirms that the worker at the destination has received the transported item, it waits at the waiting location closest to its current location until control information is input again.

[0074] As the present invention is configured as described above, the passing cost of the movement route P is changed in accordance with the number of people detected in the person identification area D set in the person detection area 40 and / or the lighting pattern of the signal lights 51 in the lighting identification area H set in the signal light detection area 50, and the movement route is made changeable based on the changed passing cost, while the movement of the autonomous mobile robot 1 in the person detection area 40 and the signal light detection area 50 is controlled in accordance with the number of people detected in the person identification area D and / or the lighting pattern of the signal lights 51 in the lighting identification area H, thereby enabling the autonomous mobile robot 1 to move efficiently to its destination.

[0075] It should be noted that the present invention is not limited to the above-described embodiments, but also includes configurations in which the components disclosed in the above-described embodiments are mutually substituted or the combinations are changed, known inventions, and configurations in which the components disclosed in the above-described embodiments are mutually substituted, etc. Furthermore, the technical scope of the present invention is not limited to the above-described embodiments, but extends to the matters set forth in the claims and their equivalents. [Explanation of symbols]

[0076] 1. Autonomous robot 2 Area photography method 3. Server equipment 21 Person detection camera 22 Traffic light detection camera 34 Disability Assessment Department 35 Image Recognition Unit 36 Image Data Collection Department 37 Model Learning Section 38 Passing cost processing unit 39 Route calculation unit D Person identification area H Lighting identification area M Control System R Shooting area

Claims

1. A mobile body capable of autonomous driving; an area photographing means including a person detection camera that is provided so as to be able to photograph a person detection area set on a movement route from the current location of the moving body to the destination; a server device that sets a person identification area for detecting people within the person detection area, calculates the number of people detected within the person identification area from image data captured by the area image capturing means, compares the number of people detected within the person identification area with a threshold value set for the person identification area, and, if the number of people detected is equal to or greater than the threshold value, increases a passing cost set for the movement route; Including, The control system for a mobile object, wherein the server device calculates the movement route of the mobile object so that the passing cost is minimized.

2. the area photographing means includes a traffic light detection camera, the signal light detection camera is provided so as to be able to photograph the lighting patterns of signal lights provided within a signal light detection area set on the movement route, The server device 2. The control system for a moving body according to claim 1, further comprising: setting a lighting identification area within the signal light detection area for detecting the lighting pattern of the signal light; detecting the lighting pattern of the signal light within the lighting identification area from the photographic data captured by the signal light detection camera; and, if it is determined that the lighting pattern of the signal light set in the lighting identification area is a lighting pattern that is different from normal, performing a process of increasing a passing cost set for the movement route that passes through the signal light detection area, while calculating the movement route of the moving body so that the passing cost is minimized.

3. The server device controlling the operation of the autonomous mobile robot passing through the person detection area in accordance with the number of people detected in the person identification region at the time when the autonomous mobile robot enters the person detection area; or When the autonomous mobile robot enters the signal light detection area, the operation of the autonomous mobile robot passing through the person detection area is controlled in accordance with the lighting pattern of the signal light detected in the lighting identification region.

3. The control system for a moving body according to claim 2.

Images