CONTROL SYSTEM, CONTROL METHOD, AND PROGRAM

Abstract

To reduce consumption power by adaptively changing a processing load in response to a situation, in controlling a system that is installed in a facility and includes multiple cameras.SOLUTION: A control system executes system control for controlling a system that is installed in a facility and includes multiple cameras, recognizes a characteristic of a person imaged by a camera, and executes group categorizing processing for grouping into a preset first or second group on the basis of the characteristic. The system control selects a first operation mode and a second operation mode, respectively, in a case where there is a person belonging to the first group and where there is no person belonging to the first group. In the system control, in the case of selecting the second operation mode, the group categorization processing is executed so as to reduce one of the number of cameras to be put to work, of the multiple cameras, and that of cameras to be used as an information source, of the multiple cameras, in the categorization of the group categorization processing, in comparison with the case of selecting the first operation mode.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a control system, a control method, and a program. [Background technology]

[0002] Patent Document 1 discloses a control system for controlling a mobile robot that moves within a facility. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-86199 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, surveillance systems for monitoring people have been introduced in various facilities including hospitals, but when used within a hospital facility, there are situations where there are people from outside the facility, such as patients or visitors, and situations where there are no such people and only people from within the facility, such as hospital staff, are present. However, if such a surveillance system performs surveillance without distinguishing between these situations (situations), a certain processing load will always be imposed.

[0005] Therefore, it is desirable to flexibly change the processing load according to the situation and reduce power consumption. However, such a problem cannot be solved by the technology described in Patent Document 1.

[0006] The present disclosure has been made to solve such problems, and provides a control system, control method, and program that are capable of flexibly changing the processing load depending on the situation and reducing power consumption when controlling a system including multiple cameras installed within a facility. [Means for solving the problem]

[0007] A control system according to a first aspect of the present disclosure executes system control for controlling a system including a plurality of cameras installed in a facility, recognizes characteristics of a person captured by the cameras, and executes a group classification process for classifying the person into a first group or a second group based on the characteristics, and the system control selects a first operation mode when the person belonging to the first group is present, and selects a second operation mode different from the first operation mode when the person belonging to the first group is not present, and executes the group classification process so as to reduce either the number of cameras to be operated among the plurality of cameras or the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process when the second operation mode is selected, compared to when the first operation mode is selected. This makes it possible to flexibly change the processing load according to the situation and reduce power consumption when controlling a system including a plurality of cameras installed in a facility.

[0008] The system control may control the plurality of cameras so that, when the first operation mode is selected, the plurality of cameras are operated, and, when the second operation mode is selected, the plurality of cameras are stopped from operating, except for the first camera. This makes it possible to reduce the processing load in the second operation mode while keeping at least the first camera necessary for monitoring in operation.

[0009] The first camera may include a camera installed at a position monitoring a security gate in the facility. This makes it possible to reduce the processing load in the second operation mode while at least the first camera installed in a portion requiring monitoring for security reasons is in operation.

[0010] The system control may include control of a mobile robot that autonomously moves in a preset area in the facility, the camera being provided at a position spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the moving mobile robot, and the system control may control the plurality of cameras to change the camera that functions as the first camera according to the running position of the mobile robot. This makes it possible to monitor the mobile robot and reduce the processing load in the second operation mode while at least the first camera provided around the mobile robot is in operation.

[0011] The system control may include control of a mobile robot that autonomously moves within a preset area within the facility, and the camera may be provided at a position spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the running mobile robot, thereby enabling monitoring of the mobile robot.

[0012] The group classification process may classify the persons according to the characteristics of the clothes worn by the persons or whether or not the persons are carrying a predetermined belongings, thereby making it easy to classify the persons.

[0013] A control method according to a second aspect of the present disclosure includes executing system control for controlling a system including a plurality of cameras installed in a facility, recognizing characteristics of a person captured by the cameras, and executing a group classification process for classifying the person into a first group or a second group based on the characteristics, the system control selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present, and executing the group classification process so as to reduce either the number of cameras to be operated among the plurality of cameras or the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process when the second operation mode is selected, compared to when the first operation mode is selected. This makes it possible to flexibly change the processing load according to the situation and reduce power consumption when controlling a system including a plurality of cameras installed in a facility.

[0014] The system control may control the plurality of cameras so that, when the first operation mode is selected, the plurality of cameras are operated, and, when the second operation mode is selected, the plurality of cameras are stopped from operating, except for the first camera. This makes it possible to reduce the processing load in the second operation mode while keeping at least the first camera necessary for monitoring in operation.

[0015] The first camera may include a camera installed at a position monitoring a security gate in the facility. This makes it possible to reduce the processing load in the second operation mode while at least the first camera installed in a portion requiring monitoring for security reasons is in operation.

[0016] The system control may include control of a mobile robot that autonomously moves in a preset area in the facility, the camera being provided at a position spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the moving mobile robot, and the system control may control the plurality of cameras to change the camera that functions as the first camera according to the running position of the mobile robot. This makes it possible to monitor the mobile robot and reduce the processing load in the second operation mode while at least the first camera provided around the mobile robot is in operation.

[0017] The system control may include control of a mobile robot that autonomously moves within a preset area within the facility, and the camera may be provided at a position spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the running mobile robot, thereby enabling monitoring of the mobile robot.

[0018] The group classification process may classify the persons according to the characteristics of the clothes worn by the persons or whether or not the persons are carrying a predetermined belongings, thereby making it easy to classify the persons.

[0019] A program according to a third aspect of the present disclosure is a program for causing a computer to execute a control method, the control method executing a system control for controlling a system including a plurality of cameras installed in a facility, recognizing characteristics of a person captured by the cameras, and executing a group classification process for classifying the person into a first group or a second group based on the characteristics, the system control selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present, and executing the group classification process so as to reduce either the number of cameras to be operated among the plurality of cameras or the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process when the second operation mode is selected, compared to when the first operation mode is selected. This makes it possible to flexibly change the processing load according to the situation and reduce power consumption when controlling a system including a plurality of cameras installed in a facility.

[0020] The system control may control the plurality of cameras so that, when the first operation mode is selected, the plurality of cameras are operated, and, when the second operation mode is selected, the plurality of cameras are stopped from operating, except for the first camera. This makes it possible to reduce the processing load in the second operation mode while keeping at least the first camera necessary for monitoring in operation.

[0021] The first camera may include a camera installed at a position monitoring a security gate in the facility. This makes it possible to reduce the processing load in the second operation mode while at least the first camera installed in a portion requiring monitoring for security reasons is in operation.

[0022] The system control may include control of a mobile robot that autonomously moves in a preset area in the facility, the camera being provided at a position spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the moving mobile robot, and the system control may control the plurality of cameras to change the camera that functions as the first camera according to the running position of the mobile robot. This makes it possible to monitor the mobile robot and reduce the processing load in the second operation mode while at least the first camera provided around the mobile robot is in operation.

[0023] The system control may include control of a mobile robot that autonomously moves within a preset area within the facility, and the camera may be provided at a position spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the running mobile robot, thereby enabling monitoring of the mobile robot.

[0024] The group classification process may classify the persons according to the characteristics of the clothes worn by the persons or whether or not the persons are carrying a predetermined belongings, thereby making it easy to classify the persons. Effect of the Invention

[0025] According to the present disclosure, it is possible to provide a control system, a control method, and a program that can flexibly change the processing load depending on the situation and reduce power consumption when controlling a system including multiple cameras installed within a facility. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a conceptual diagram for explaining an example of the overall configuration of a transport system using a mobile robot into which a control system according to an embodiment can be incorporated. [Diagram 2] FIG. 2 is a control block diagram of the transport system in FIG. [Diagram 3] FIG. 1 is a schematic diagram illustrating an example of a mobile robot. [Figure 4] FIG. 2 is a control block diagram showing a control system for mode control. [Diagram 5] 11 is a table for explaining an example of staff information. [Figure 6] 11 is a table for explaining an example of mode information. [Figure 7] 4 is a flowchart illustrating an example of a control method according to the embodiment. [Figure 8] FIG. 13 is a diagram for explaining an embodiment of mode control. [Figure 9] FIG. 13 is a diagram for explaining an embodiment of mode control. [Figure 10] 10 is a flowchart showing another example of a control method according to the embodiment. [Figure 11] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential means for solving the problems.

[0028] (Schematic configuration) The control system according to the present embodiment is a system capable of monitoring people by executing system control for controlling a system including a plurality of cameras installed in a facility and executing group classification processing for classifying people. The system control can be performed by a system control unit provided in the control system, and the group classification processing can be performed by a group classification unit provided in the control system.

[0029] First, an example of a transport system using a mobile robot into which the control system according to the present embodiment can be incorporated will be described. FIG. 1 is a conceptual diagram for explaining an example of the overall configuration of a transport system 1 using a mobile robot 20 into which the control system according to the present embodiment can be incorporated. For example, the mobile robot 20 is a transport robot that performs the task of transporting an object to be transported. The mobile robot 20 travels autonomously to transport the object to be transported in a medical and welfare facility such as a hospital, rehabilitation center, nursing home, or elderly care facility. The transport system according to the present embodiment can also be used in commercial facilities such as shopping malls.

[0030] A user U1 places an item in the mobile robot 20 and requests that it be transported. The mobile robot 20 autonomously moves to a set destination and transports the item. That is, the mobile robot 20 executes a luggage transport task (hereinafter simply referred to as a task). In the following description, the location where the item is loaded is referred to as the origin, and the location where the item is delivered is referred to as the destination.

[0031] For example, the mobile robot 20 moves within a general hospital with multiple medical departments. The mobile robot 20 transports supplies, consumables, medical equipment, etc. between the multiple medical departments. For example, the mobile robot 20 delivers the items from a nurse's station in one medical department to a nurse's station in another medical department. Alternatively, the mobile robot 20 delivers the items from a storage room for supplies and medical equipment to the nurse's station in the medical department. The mobile robot 20 also delivers medicines dispensed in a pharmacy department to the medical department or patient who plans to use them.

[0032] Examples of the goods to be transported include consumables such as medicines and packets, specimens, test equipment, medical equipment, hospital food, stationery, and other supplies. Examples of medical equipment include blood pressure monitors, blood transfusion pumps, syringe pumps, foot pumps, nurse call, bed exit sensors, foot pumps, low pressure continuous inhalers, electrocardiogram monitors, drug injection controllers, enteral nutrition pumps, ventilators, cuff pressure gauges, touch sensors, aspirators, nebulizers, pulse oximeters, blood pressure monitors, artificial resuscitators, sterilization devices, and echo devices. Food such as hospital food and test food may also be transported. Furthermore, the mobile robot 20 may transport used equipment, used tableware, and the like. When the destination is on a different floor, the mobile robot 20 may move using an elevator or the like.

[0033] The transport system 1 includes a mobile robot 20, a host management device 10, a network 600, a communication unit 610, and a user terminal 400. A user U1 or a user U2 can use the user terminal 400 to make a transport request for an object. For example, the user terminal 400 is a tablet computer or a smartphone. The user terminal 400 may be any information processing device capable of wireless or wired communication.

[0034] In this embodiment, the mobile robot 20 and the user terminal 400 are connected to the host management device 10 via a network 600. The mobile robot 20 and the user terminal 400 are connected to the network 600 via a communication unit 610. The network 600 is a wired or wireless LAN (Local Area Network) or WAN (Wide Area Network). Furthermore, the host management device 10 is connected to the network 600 by wire or wireless. The communication unit 610 is, for example, a wireless LAN unit installed in each environment. The communication unit 610 may be, for example, a general-purpose communication device such as a WiFi (registered trademark) router.

[0035] Various signals transmitted from the user terminals 400 of the users U1 and U2 are first sent to the host management device 10 via the network 600, and then transferred from the host management device 10 to the target mobile robot 20. Similarly, various signals transmitted from the mobile robot 20 are first sent to the host management device 10 via the network 600, and then transferred from the host management device 10 to the target user terminal 400. The host management device 10 is a server connected to each device and collects data from each device. In addition, the host management device 10 is not limited to a single physical device, and may include multiple devices that perform distributed processing. In addition, the host management device 10 may be distributed and arranged in edge devices such as the mobile robot 20. For example, a part or all of the transport system 1 may be mounted on the mobile robot 20.

[0036] The user terminal 400 and the mobile robot 20 may transmit and receive signals without going through the higher level management device 10. For example, the user terminal 400 and the mobile robot 20 may transmit and receive signals directly by wireless communication. Alternatively, the user terminal 400 and the mobile robot 20 may transmit and receive signals via the communication unit 610.

[0037] User U1 or user U2 requests the transport of an item using a user terminal 400. In the following description, it is assumed that user U1 is the transport requester at the transport origin, and user U2 is the intended recipient at the transport destination (destination). Of course, user U2 at the transport destination can also make the transport request. Also, a user at a location other than the transport origin or destination may make the transport request.

[0038] When user U1 makes a transport request, he / she uses the user terminal 400 to input the contents of the transported item, the recipient of the transported item (hereinafter also referred to as the transport origin), the destination of the transported item (hereinafter also referred to as the transport destination), the scheduled arrival time at the transport origin (time of receipt of the transported item), the scheduled arrival time at the transport destination (transport deadline), etc. Hereinafter, this information is also referred to as transport request information. User U1 can input the transport request information by operating the touch panel of the user terminal 400. The transport origin may be the location where user U1 is located, or may be the storage location of the transported item, etc. The transport destination is the location where user U2 who is scheduled to use the item or the patient is located.

[0039] The user terminal 400 transmits the transport request information input by the user U1 to the host management device 10. The host management device 10 is a management system that manages a plurality of mobile robots 20. The host management device 10 transmits operation commands to the mobile robots 20 to execute the transport task. For each transport request, the host management device 10 determines the mobile robot 20 that will execute the transport task. Then, the host management device 10 transmits a control signal including the operation command to the mobile robot 20. The mobile robot 20 moves from the transport source to the transport destination in accordance with the operation command.

[0040] For example, the host management device 10 assigns a transport task to a mobile robot 20 at or near the transport origin. Alternatively, the host management device 10 assigns a transport task to a mobile robot 20 heading toward the transport origin or its vicinity. The mobile robot 20 to which the task is assigned goes to the transport origin to pick up the transported object. The transport origin is, for example, a location where the user U1 who requested the task is located.

[0041] When the mobile robot 20 arrives at the destination, user U1 or another staff member loads the transported item onto the mobile robot 20. The mobile robot 20 carrying the transported item moves autonomously to the destination. The upper management device 10 transmits a signal to the user terminal 400 of user U2 at the destination. This allows user U2 to know that the transported item is being transported and the estimated arrival time. When the mobile robot 20 arrives at the set destination, user U2 can receive the transported item stored in the mobile robot 20. In this manner, the mobile robot 20 executes the transport task.

[0042] In such an overall configuration, the robot control system for controlling the mobile robot 20 in the transport system 1 can be constructed by distributing each element among the mobile robot 20, the user terminal 400, and the host management device 10. Also, the elements of the robot control system, i.e., the substantial elements for realizing the transport of an object by the mobile robot 20, can be gathered in one device. The host management device 10, which can function as this device, will control one or more mobile robots 20.

[0043] The mobile robot 20 used in the transport system 1 can be a mobile robot that moves autonomously by referring to a map. The robot control system that controls the mobile robot 20 acquires distance information indicating the distance to a person measured using a distance sensor, for example. The robot control system estimates a movement vector indicating the moving speed and moving direction of the person according to a change in the distance to the person. The robot control system assigns a cost for restricting the movement of the mobile robot 20 on the map. The robot control system controls the robot to move according to the cost updated according to the measurement result of the distance sensor. The robot control system may be mounted on the mobile robot 20, or a part or all of the robot control system may be mounted on the upper management device 10. However, in this embodiment, the control method of the mobile robot 20 in the robot control system does not matter.

[0044] The control system according to this embodiment can be incorporated into the transport system 1 together with such a robot control system or as an independent system. The elements of such a control system, i.e., the elements of a control system for monitoring people, can be constructed in a distributed manner in the mobile robot 20, the user terminal 400, and the host management device 10, or can be constructed in a single device. In the following example, an example will be described in which the host management device 10, as an example of this device, is equipped with the elements of the control system, i.e., the host management device 10 executes system control and group classification processing, i.e., is equipped with a system control unit and a group classification unit.

[0045] The system control unit controls a system including a plurality of cameras (hereinafter, exemplified as environmental cameras) installed in the facility. The group classification unit recognizes the characteristics of people captured by the environmental cameras, and classifies the people into a first group or a second group that is set in advance based on the characteristics. In this embodiment, an example will be described in which the first group is non-staff and the second group is staff, but classification of people can also be performed based on various other attributes. Furthermore, the mobile robot 20 can also be a monitoring target, in which case the mobile robot 20 can be classified as staff.

[0046] The group classification unit can perform classification by matching pre-stored images of staff members (e.g., facial images) with images of people captured by an environmental camera. Any method for recognizing (detecting or extracting) the characteristics of people for classification can be used. The group classification unit can also perform classification using, for example, a learning model based on machine learning. The group classification unit may classify people according to, for example, the characteristics of the people's clothing, or whether or not the people are carrying certain belongings. This makes it easy to classify people.

[0047] We will now explain non-staff and staff. Facility users include staff who work at the facility and other non-staff. If the facility is a hospital, non-staff include patients, inpatients, visitors, outpatients, and attendants. Staff include doctors, nurses, pharmacists, office staff, occupational therapists, and various other employees. Staff may also include people who deliver various items, maintenance workers, cleaners, and the like. Staff are not limited to direct employees or sub-employees of the hospital, and may also include related workers.

[0048] It is desirable for the control system to reduce power consumption in an environment where staff and non-staff are monitored together, such as a hospital. Therefore, the host management device 10 needs to monitor the facility while appropriately reducing power consumption according to the situation of the facility.

[0049] Specifically, the system control unit of the upper management device 10 selects a first operation mode when a person belonging to the non-staff category is present, and selects a second operation mode different from the first operation mode when a person belonging to the non-staff category is not present. In the following description, the first operation mode is referred to as the non-staff mode and the second operation mode is referred to as the staff mode, but classification can also be made based on other attributes as explained in the classification of the first group / second group.

[0050] For example, if the system control unit recognizes that no non-staff person is present when the first operation mode is selected, it can switch to the second operation mode, and conversely, if the system control unit recognizes that no non-staff person is present when the second operation mode is selected, it can switch to the first operation mode.

[0051] When the second operation mode is selected, the system control unit executes a group classification process so as to reduce either the number of cameras to be operated among the multiple environmental cameras provided in the facility or the number of environmental cameras to be used as information sources among the multiple environmental cameras in classification by the group classification unit, compared to when the first operation mode is selected. In the former case, the control target in the system control unit for such group classification process can be the environmental cameras (switching control of operating / non-operating the environmental cameras), and in the latter case, can be the group classification unit or an image data acquisition unit that acquires image data to be used for classification, etc.

[0052] In the second operation mode, the environmental camera or image data used may be a predetermined environmental camera, but is not limited to this, and may be changed depending on the time of day, or may be determined depending on the location where a non-staff member was last confirmed, etc. In the first operation mode, the environmental camera or image data used may be a predetermined environmental camera, but is not limited to this, and may be changed depending on the time of day, etc.

[0053] In this way, the system control unit allows the host management device 10 to switch the monitoring mode depending on whether the person being monitored is a staff member who does not need to be monitored very much, or a non-staff member who needs to be monitored as much as possible. Therefore, the control system according to this embodiment can take measures to reduce the monitoring load depending on the presence or absence of non-staff members, that is, when controlling a system including multiple environmental cameras installed for monitoring within a facility, it is possible to flexibly change the processing load depending on the situation (scene) and reduce power consumption.

[0054] (Control block diagram) Fig. 2 is a control block diagram showing a control system of the transport system 1. As shown in Fig. 2, the transport system 1 includes a host management device 10, a mobile robot 20, and the above-mentioned environmental camera 300.

[0055] The transport system 1 efficiently controls a plurality of mobile robots 20 while autonomously moving the mobile robots 20 within a given facility. For this purpose, a plurality of environmental cameras 300 are installed within the facility. For example, the environmental cameras 300 are installed in passages, halls, elevators, entrances, and the like within the facility. The environmental cameras 300 are used not only for controlling the mobile robots 20, but also for monitoring people as described above. However, in this embodiment, the environmental cameras 300 may not be used for controlling the mobile robots 20, and may be used only for monitoring people, for example, or a camera for monitoring people and a camera for monitoring the mobile robots 20 may be provided separately.

[0056] The environmental camera 300 captures images of the range in which people and the mobile robot 20 move. In the transport system 1, the images captured by the environmental camera 300 and information based on them are collected by the host management device 10. As for images used to control the mobile robot 20, the images captured by the environmental camera 300 may be directly transmitted to the mobile robot. The environmental camera 300 may be installed as a surveillance camera in the passages and entrances of a facility. The environmental camera 300 may be used to obtain the distribution of congestion in the facility.

[0057] Regarding transportation, in the transportation system 1, the host management device 10 plans a route based on transportation request information. Based on the route planning information created by the host management device 10, the host management device 10 instructs each mobile robot 20 on its destination. Then, the mobile robot 20 moves autonomously toward the destination specified by the host management device 10. The mobile robot 20 moves autonomously toward the destination (destination) using sensors, a floor map, position information, etc. provided on the mobile robot.

[0058] For example, the mobile robot 20 runs so as not to come into contact with surrounding equipment, objects, walls, and people (hereinafter collectively referred to as surrounding objects). Specifically, the mobile robot 20 detects the distance to the surrounding objects, and runs while being at a certain distance (referred to as the distance threshold) or more away from the surrounding objects. When the distance to the surrounding object becomes equal to or less than the distance threshold, the mobile robot 20 slows down or stops. In this way, the mobile robot 20 can run without coming into contact with surrounding objects. As contact can be avoided, safe and efficient transportation becomes possible.

[0059] The upper management device 10 includes a calculation processing unit 11, a storage unit 12, a buffer memory 13, and a communication unit 14. The calculation processing unit 11 performs calculations for monitoring people and the mobile robot 20, and calculations for controlling and managing the mobile robot 20. The calculation processing unit 11 can be implemented as a device capable of executing a program, such as a central processing unit (CPU) of a computer. Various functions can also be realized by a program. In FIG. 2, only the robot control unit 111, the route planning unit 115, the transported goods information acquisition unit 116, and the mode control unit 117, which are characteristic of the calculation processing unit 11, are shown, but other processing blocks can also be included.

[0060] The robot control unit 111 performs calculations for remotely controlling the mobile robot 20 and generates a control signal. The robot control unit 111 generates the control signal based on route planning information 125, which will be described later, and the like. Furthermore, the control signal is generated based on various information obtained from the environmental camera 300 and the mobile robot 20. The control signal may include update information such as a floor map 121, robot information 123, and robot control parameters 122, which will be described later. In other words, when various information is updated, the robot control unit 111 generates a control signal according to the update information.

[0061] The transported item information acquisition unit 116 acquires information about the transported item. The transported item information acquisition unit 116 acquires information about the contents (type) of the transported item being transported by the mobile robot 20. The transported item information acquisition unit 116 acquires transported item information about the transported item being transported by the mobile robot 20 in which an error has occurred.

[0062] The route planning unit 115 plans routes for each mobile robot 20. When a transport task is input, the route planning unit 115 plans a route for transporting the transported object to a transport destination (destination) based on transport request information. Specifically, the route planning unit 115 refers to route planning information 125 and robot information 123 already stored in the storage unit 12, and determines the mobile robot 20 that will execute the new transport task. The starting point is the current location of the mobile robot 20, the destination of the previous transport task, the recipient of the transported object, etc. The destination is the destination of the transported object, a waiting location, a charging location, etc.

[0063] Here, the route planning unit 115 sets pass points from the starting point to the destination of the mobile robot 20. The route planning unit 115 sets the order of passing the pass points for each mobile robot 20. The pass points are set, for example, at branching points, intersections, lobbies in front of elevators, and the surrounding areas. In addition, in narrow passages, it may be difficult for the mobile robots 20 to pass each other. In such cases, a pass point may be set just before the narrow passage. Candidates for pass points may be registered in the floor map 121 in advance.

[0064] The route planning unit 115 determines which mobile robot 20 will perform each transport task from among the multiple mobile robots 20 so that the tasks can be executed efficiently as a whole system. The route planning unit 115 preferentially assigns transport tasks to mobile robots 20 that are on standby or that are close to the transport source.

[0065] The route planning unit 115 sets pass points including a starting point and a destination point for the mobile robot 20 to which a transport task is assigned. For example, when there are two or more moving routes from the transport origin to the transport destination, the pass points are set so that the robot can travel in a shorter time. For this reason, the upper level management device 10 updates information indicating the congestion status of the passageway based on camera images, etc. Specifically, places where other mobile robots 20 are passing through and places with many people are highly crowded. Therefore, the route planning unit 115 sets pass points so as to avoid places with high congestion.

[0066] There are cases where the mobile robot 20 can move to the destination via either a left-handed or a right-handed movement route. In such cases, the route planning unit 115 sets a passing point so that the mobile robot 20 passes through the less congested movement route. By the route planning unit 115 setting one or more passing points on the way to the destination, the mobile robot 20 can move along a less congested movement route. For example, when the passageways are divided at a branch point or an intersection, the route planning unit 115 appropriately sets passing points at the branch point, intersection, corner, and their surroundings. This can improve the efficiency of transportation.

[0067] The route planning unit 115 may set the passing points in consideration of the congestion status of the elevators, the moving distance, and the like. Furthermore, the upper management device 10 may estimate the number of mobile robots 20 and the number of people at the scheduled time when the mobile robots 20 pass a certain location. Then, the route planning unit 115 may set the passing points according to the estimated congestion status. Furthermore, the route planning unit 115 may dynamically change the passing points according to the change in the congestion status. The route planning unit 115 sets the passing points in order for the mobile robots 20 to which the transportation task is assigned. The passing points may include the transportation origin and the transportation destination. As will be described later, the mobile robot 20 moves autonomously so as to pass through the passing points set by the route planning unit 115 in order.

[0068] The mode control unit 117 includes the group classification unit and corresponds to a part or all of the system control unit. The mode control unit 117 performs control for switching between a first operation mode (exemplified by non-staff mode) and a second operation mode (exemplified by staff mode) according to the situation of the facility. By switching the mode according to the situation of the facility, the processing load can be reduced and power consumption can be reduced. The control of the mode control unit 117 will be described later.

[0069] The memory unit 12 is a memory unit that stores information necessary for managing and controlling the robot, including monitoring people and the robot. In the example of Fig. 2, a floor map 121, robot information 123, robot control parameters 122, route plan information 125, transported item information 126, staff information 128, and mode information 129 are shown, but other information may be stored in the memory unit 12. The calculation processing unit 11 performs calculations using the information stored in the memory unit 12 when performing various processes. In addition, the various information stored in the memory unit 12 can be updated to the latest information.

[0070] The floor map 121 is map information of the facility in which the mobile robot 20 moves. This floor map 121 may be created in advance, may be generated from information obtained from the mobile robot 20, or may be a basic map created in advance to which map correction information generated from information obtained from the mobile robot 20 has been added.

[0071] For example, the floor map 121 stores the positions and information of walls, gates, doors, stairs, elevators, fixed shelves, etc. of the facility. The floor map 121 may be expressed as a two-dimensional grid map. In this case, information on walls, doors, etc. is added to each grid in the floor map 121.

[0072] The robot information 123 describes the ID, model number, specifications, etc. of the mobile robot 20 managed by the upper management device 10. The robot information 123 may include position information indicating the current position of the mobile robot 20. The robot information 123 may include information on whether the mobile robot 20 is executing a task or is on standby. The robot information 123 may also include information on whether the mobile robot 20 is in operation or out of order. The robot information 123 may also include information on transportable and untransportable goods.

[0073] The robot control parameters 122 describe control parameters such as a threshold distance between the mobile robot 20 managed by the upper management device 10 and a surrounding object. The threshold distance is a margin distance for avoiding contact with a surrounding object including a person. Furthermore, the robot control parameters 122 may include information on the strength of the operation of the mobile robot 20, such as an upper limit of the moving speed of the mobile robot 20.

[0074] The robot control parameters 122 may be updated according to the situation. The robot control parameters 122 may include information indicating the availability and usage of the storage space in the storehouse 291. The robot control parameters 122 may include information on transportable and untransportable goods. The robot control parameters 122 associate the above various information with each mobile robot 20.

[0075] The route planning information 125 includes route planning information planned by the route planning unit 115. The route planning information 125 includes, for example, information indicating a transportation task. The route planning information 125 may include information such as an ID of the mobile robot 20 to which the task is assigned, a starting point, the contents of the transported object, a transport destination, a transport origin, an expected arrival time at the transport destination, an expected arrival time at the transport origin, and an arrival deadline. In the route planning information 125, the above-mentioned various information may be associated with each transportation task. The route planning information 125 may include at least a part of the transportation request information input by the user U1.

[0076] Furthermore, the route planning information 125 may include information on pass points for each mobile robot 20 and each transport task. For example, the route planning information 125 includes information indicating the order of passing through pass points for each mobile robot 20. The route planning information 125 may include the coordinates of each pass point on the floor map 121 and information on whether or not the pass point has been passed.

[0077] The transported item information 126 is information about the item for which a transport request has been made. For example, it includes information such as the contents (type) of the item, the origin of the item, and the destination of the item. The transported item information 126 may include the ID of the mobile robot 20 in charge of the transport. Furthermore, the transported item information may include information indicating the status such as being transported, before transport (before loading), and already transported. The transported item information 126 associates these pieces of information with each transported item. The transported item information 126 will be described later.

[0078] Staff information 128 is information for classifying whether a user of the facility is a staff member or not. In other words, staff information 128 includes information for classifying a person included in image data as a non-staff member or a staff member. For example, staff information 128 includes information related to staff members registered in advance. Mode information 129 includes information for controlling each mode based on the classification result. Details of staff information 128 and mode information 129 will be described later.

[0079] The route planning unit 115 formulates a route plan by referring to various information stored in the storage unit 12. For example, the mobile robot 20 that executes the task is determined based on the floor map 121, the robot information 123, the robot control parameters 122, and the route planning information 125. The route planning unit 115 then sets pass points to the destination and the order of passing through them by referring to the floor map 121 and the like. Candidates for pass points are registered in the floor map 121 in advance. The route planning unit 115 then sets the pass points according to the congestion situation and the like. In addition, when tasks are processed continuously, the route planning unit 115 may set the source and destination as pass points.

[0080] Note that two or more mobile robots 20 may be assigned to one transport task. For example, if the transported object is larger than the transportable capacity of the mobile robot 20, one transported object is divided into two and loaded onto two mobile robots 20. Alternatively, if the transported object is heavier than the transportable weight of the mobile robot 20, one transported object is divided into two and loaded onto two mobile robots 20. In this way, one transport task can be shared and executed by two or more mobile robots 20. Of course, when controlling mobile robots 20 of different sizes, route planning may be performed so that the mobile robot 20 capable of transporting the object receives the object.

[0081] Furthermore, one mobile robot 20 may perform two or more transport tasks in parallel. For example, one mobile robot 20 may simultaneously carry two or more transported objects and transport them sequentially to different destinations. Alternatively, one mobile robot 20 may carry one transported object while carrying another transported object. Furthermore, the destinations of the transported objects carried at different locations may be the same or different. In this way, tasks can be executed efficiently.

[0082] In such a case, the accommodation information indicating the usage status or availability of the accommodation space of the mobile robot 20 may be updated. That is, the upper management device 10 may manage the accommodation information indicating the availability and control the mobile robot 20. For example, when the loading or receiving of the transported object is completed, the accommodation information is updated. When the transport task is input, the upper management device 10 refers to the accommodation information and directs the mobile robot 20 that has the availability to load the transported object to receive it. In this way, it becomes possible for one mobile robot 20 to simultaneously execute multiple transport tasks, or for two or more mobile robots 20 to share and execute the transport task. For example, a sensor may be installed in the accommodation space of the mobile robot 20 to detect the availability. In addition, the capacity and weight of each transported object may be registered in advance.

[0083] The buffer memory 13 is a memory that accumulates intermediate information generated in the processing in the arithmetic processing unit 11. The communication unit 14 is a communication interface for communicating with a plurality of environmental cameras 300 and at least one mobile robot 20 provided in a facility where the transport system 1 is used. The communication unit 14 can perform both wired and wireless communication. For example, the communication unit 14 transmits, to each mobile robot 20, a control signal required for controlling the mobile robot 20. The communication unit 14 also receives information collected by the mobile robot 20 and the environmental camera 300. The communication unit 14 also transmits information for remotely controlling the operation / non-operation of the environmental camera 300 to the environmental camera 300 to be controlled.

[0084] The mobile robot 20 includes a processor 21, a memory unit 22, a communication unit 23, a proximity sensor (e.g., a distance sensor group 24), a camera 25, a drive unit 26, a display unit 27, and an operation reception unit 28. Note that, although Fig. 2 shows only representative processing blocks included in the mobile robot 20, the mobile robot 20 also includes many other processing blocks that are not shown.

[0085] The communication unit 23 is a communication interface for communicating with the communication unit 14 of the higher-level management device 10. The communication unit 23 communicates with the communication unit 14 using, for example, a wireless signal. The distance sensor group 24 is, for example, a proximity sensor, and outputs nearby object distance information indicating the distance to an object or person present around the mobile robot 20. The distance sensor group 24 includes a distance measuring sensor such as a lidar. By manipulating the emission direction of the optical signal, the distance to the surrounding object can be measured. In addition, the surrounding object may be recognized from point cloud data detected by the distance measuring sensor or the like. The camera 25, for example, takes an image for grasping the surrounding situation of the mobile robot 20. In addition, the camera 25 can also take an image of a position marker provided on, for example, the ceiling of the facility. The mobile robot 20 may grasp its own position using the position marker.

[0086] The camera 25 can also function as one of the environmental cameras 300. In this case, since the camera 25 is necessary for the movement of the mobile robot 20 while it is moving, the targets to be reduced in staff mode do not include the camera 25 of the mobile robot 20 while it is moving (the camera 25 is not taken out of operation), but can include images from the camera 25 as an information source (images from the camera 25 are not used).

[0087] The driving unit 26 drives the driving wheels provided on the mobile robot 20. The driving unit 26 may include an encoder that detects the number of rotations of the driving wheels or the driving motor. The mobile robot's own position (current position) may be estimated according to the output of the encoder. The mobile robot 20 detects its own current position and transmits it to the upper management device 10. The mobile robot 20 estimates its own position on the floor map 121 by odometry or the like.

[0088] The display unit 27 and the operation reception unit 28 are realized by a touch panel display. The display unit 27 displays a user interface screen that serves as the operation reception unit 28. The display unit 27 may also display information indicating the destination of the mobile robot 20 and the state of the mobile robot 20. The operation reception unit 28 receives operations from the user. The operation reception unit 28 includes the user interface screen displayed on the display unit 27, as well as various switches provided on the mobile robot 20.

[0089] The arithmetic processing unit 21 performs calculations used to control the mobile robot 20. The arithmetic processing unit 21 can be implemented as a device capable of executing programs, such as a central processing unit (CPU) of a computer. Various functions can also be realized by programs. The arithmetic processing unit 21 includes a movement command extraction unit 211 and a drive control unit 212. Note that while FIG. 2 shows only representative processing blocks included in the arithmetic processing unit 21, processing blocks not shown in the figure are also included. The arithmetic processing unit 21 may search for a route between passing points.

[0090] The movement command extraction unit 211 extracts a movement command from a control signal provided by the upper management device 10. For example, the movement command includes information on the next passing point. For example, the control signal may include coordinates of the passing point and information on the order of passing through the passing point. Then, the movement command extraction unit 211 extracts this information as a movement command.

[0091] Furthermore, the movement command may include information indicating that movement to the next passing point is now possible. If the passage width is narrow, the mobile robots 20 may not be able to pass each other. Also, there may be cases where the passage cannot be passed temporarily. In such cases, the control signal includes a command to stop the mobile robot 20 at a passing point just before the place where it should stop. Then, after another mobile robot 20 has passed or is able to pass, the upper management device 10 outputs a control signal to notify the mobile robot 20 that it is now able to move. This causes the mobile robot 20, which had been temporarily stopped, to resume movement.

[0092] The drive control unit 212 controls the drive unit 26 to move the mobile robot 20 based on the movement command given from the movement command extraction unit 211. For example, the drive unit 26 has a drive wheel that rotates according to a control command value from the drive control unit 212. The movement command extraction unit 211 extracts a movement command so that the mobile robot 20 moves toward the passing point received from the upper management device 10. Then, the drive unit 26 rotates and drives the drive wheel. The mobile robot 20 moves autonomously toward the next passing point. In this way, the mobile robot 20 passes through the passing points in order and arrives at the transport destination. In addition, the mobile robot 20 may estimate its own position and transmit a signal indicating that it has passed the passing point to the upper management device 10. This allows the upper management device 10 to manage the current position and transport status of each mobile robot 20.

[0093] The storage unit 22 stores a floor map 221, robot control parameters 222, and transported item information 226. FIG. 2 shows only a part of the information stored in the storage unit 22, and the information stored in the storage unit 22 includes information other than the floor map 221, robot control parameters 222, and transported item information 226 shown in FIG. 2. The floor map 221 is map information of a facility in which the mobile robot 20 is to be moved. This floor map 221 is, for example, a downloaded floor map 121 of the upper management device 10. The floor map 221 may be one that has been created in advance. The floor map 221 may not be map information of the entire facility, but may be map information that partially includes an area to be moved.

[0094] The robot control parameters 222 are parameters for operating the mobile robot 20. The robot control parameters 222 include, for example, a distance threshold to a surrounding object. Furthermore, the robot control parameters 222 include an upper limit of the speed of the mobile robot 20.

[0095] The transported item information 226 includes information on the transported item, similar to the transported item information 126. It includes information on the contents (type) of the transported item, the origin, the destination, etc. The transported item information may include information indicating the status, such as being transported, before transport (before loading), and already transported. The transported item information 226 associates these pieces of information with each transported item. The transported item information 226 only needs to include information on the transported item transported by the mobile robot 20. Therefore, the transported item information 226 becomes a part of the transported item information 126. In other words, the transported item information 226 does not need to include information on transported items by other mobile robots 20.

[0096] The drive control unit 212 refers to the robot control parameters 222 and stops or decelerates the operation in response to the distance indicated by the distance information obtained from the distance sensor group 24 falling below the distance threshold. The drive control unit 212 controls the drive unit 26 so that the mobile robot 20 travels at a speed equal to or lower than the upper speed limit. The drive control unit 212 limits the rotation speed of the drive wheels so that the mobile robot 20 does not travel at a speed equal to or higher than the upper speed limit.

[0097] (Configuration of the mobile robot 20) Here, the external appearance of the mobile robot 20 will be described. Fig. 3 shows a schematic diagram of the mobile robot 20. The mobile robot 20 shown in Fig. 3 is one aspect of the mobile robot 20, and other aspects may also be used. In Fig. 3, the x direction is the forward and backward direction of the mobile robot 20, the y direction is the left-right direction of the mobile robot 20, and the z direction is the height direction of the mobile robot 20.

[0098] The mobile robot 20 includes a main body 290 and a cart 260. The main body 290 is mounted on the cart 260. The main body 290 and the cart 260 each include a rectangular parallelepiped housing, and each component is mounted inside this housing. For example, the drive unit 26 is housed inside the cart 260.

[0099] The main body 290 is provided with a storage 291 serving as a storage space and a door 292 for sealing the storage 291. The storage 291 is provided with a plurality of shelves, and the availability status is managed for each shelf. For example, the availability status can be updated by arranging various sensors such as weight sensors on each shelf. The mobile robot 20 autonomously transports the transported goods stored in the storage 291 to a destination instructed by the upper management device 10. The main body 290 may be equipped with a control box or the like (not shown) in the housing. The door 292 may be lockable with an electronic key or the like. When the transport destination is reached, the user U2 unlocks the door 292 with the electronic key. Alternatively, the door 292 may be unlocked automatically when the transport destination is reached.

[0100] 3, the exterior of the mobile robot 20 is provided with a front-rear distance sensor 241 and a left-right distance sensor 242 as the distance sensor group 24. The mobile robot 20 measures the distance to surrounding objects in the front-rear direction of the mobile robot 20 using the front-rear distance sensor 241. The mobile robot 20 also measures the distance to surrounding objects in the left-right direction of the mobile robot 20 using the left-right distance sensor 242.

[0101] For example, the front / rear distance sensor 241 is disposed on the front and rear surfaces, respectively, of the housing of the main body 290. The left / right distance sensor 242 is disposed on the left and right surfaces, respectively, of the housing of the main body 290. The front / rear distance sensor 241 and the left / right distance sensor 242 are, for example, ultrasonic distance sensors or laser range finders. They detect the distance to a surrounding object. When the distance to a surrounding object detected by the front / rear distance sensor 241 or the left / right distance sensor 242 becomes equal to or less than a distance threshold, the mobile robot 20 decelerates or stops.

[0102] The driving unit 26 is provided with a driving wheel 261 and a caster 262. The driving wheel 261 is a wheel for moving the mobile robot 20 forward, backward, left and right. The caster 262 is a driven wheel that is not given a driving force and rolls following the driving wheel 261. The driving unit 26 is provided with a driving motor (not shown) and drives the driving wheel 261.

[0103] For example, the driving unit 26 supports, within the housing, two driving wheels 261 and two casters 262, each of which comes into contact with the running surface. The two driving wheels 261 are arranged so that their rotational axes coincide with each other. Each driving wheel 261 is independently driven to rotate by a motor (not shown). The driving wheels 261 rotate according to a control command value from the driving control unit 212 in FIG. 2. The casters 262 are driven wheels, and are provided so that a pivot axis extending vertically from the driving unit 26 is separated from the rotation axis of the wheels to support the wheels, and follow the moving direction of the driving unit 26.

[0104] For example, if the two drive wheels 261 are rotated in the same direction at the same rotation speed, the mobile robot 20 moves straight, and if they are rotated in opposite directions at the same rotation speed, the mobile robot 20 turns around a vertical axis passing through approximately the center of the two drive wheels 261. In addition, by rotating the two drive wheels 261 in the same direction at different rotation speeds, the mobile robot 20 can move while turning left and right. For example, by making the rotation speed of the left drive wheel 261 higher than that of the right drive wheel 261, the mobile robot 20 can turn right. Conversely, by making the rotation speed of the right drive wheel 261 higher than that of the left drive wheel 261, the mobile robot 20 can turn left and right, etc. in any direction by controlling the rotation directions and rotation speeds of the two drive wheels 261, respectively.

[0105] Furthermore, in the mobile robot 20, a display unit 27 and an operation interface 281 are provided on the upper surface of the main body unit 290. The operation interface 281 is displayed on the display unit 27. When the user touches the operation interface 281 displayed on the display unit 27, the operation reception unit 28 can receive an instruction input from the user. Furthermore, an emergency stop button 282 is provided on the upper surface of the display unit 27. The emergency stop button 282 and the operation interface 281 function as the operation reception unit 28.

[0106] The display unit 27 is, for example, a liquid crystal panel, and displays a character's face as an illustration, and presents information about the mobile robot 20 as text or icons. If a character's face is displayed on the display unit 27, it is possible to give the impression to observers in the vicinity that the display unit 27 is a pseudo face. The display unit 27 mounted on the mobile robot 20 can also be used as the user terminal 400.

[0107] A camera 25 is installed on the front of the main body 290. Here, the two cameras 25 function as a stereo camera. That is, the two cameras 25 having the same angle of view are arranged horizontally spaced apart from each other. Images captured by each camera 25 are output as image data. Based on the image data of the two cameras 25, it is possible to calculate the distance to the subject and the size of the subject. The arithmetic processing unit 21 can detect people, obstacles, etc. ahead in the moving direction by analyzing the image of the camera 25. If there are people, obstacles, etc. ahead in the moving direction, the mobile robot 20 moves along the route while avoiding them. In addition, the image data of the camera 25 is transmitted to the upper management device 10.

[0108] The mobile robot 20 recognizes surrounding objects and identifies its own position by analyzing image data output by the camera 25 and detection signals output by the front-rear distance sensor 241 and the left-right distance sensor 242. The camera 25 captures images of the area ahead of the mobile robot 20 in the direction of travel. As shown in the figure, the side of the mobile robot 20 on which the camera 25 is installed is considered to be the front of the mobile robot 20. In other words, during normal travel, the direction ahead of the mobile robot 20 is the direction of travel, as indicated by the arrow.

[0109] (Mode control processing) Next, the mode control process will be described with reference to Fig. 4. Here, the process for mode control will be described as being performed by the upper management device 10. Therefore, Fig. 4 is mainly a block diagram showing the control system of the mode control unit 117. Of course, the mobile robot 20 may be provided with a mode control unit and perform at least a part of the process of the mode control unit 117. Alternatively, the environmental camera 300 may perform at least a part of the process for mode control.

[0110] The mode control unit 117 includes an image data acquisition unit 1170, a feature extraction unit 1171, a classifier 1172, a selection unit 1173, and a switching unit 1174. The environmental camera 300 includes an image sensor and a calculation processing unit, not shown. The image sensor captures images to monitor the inside of the facility. The calculation processing unit of the environmental camera 300 includes a GPU that performs image processing on the images captured by the image sensor, and is configured to be able to respond to external operation / non-operation control and stop (or start) the power supply (or low power mode).

[0111] The image data acquisition unit 1170 acquires image data of an image captured by the environmental camera 300. Here, the image data may be the image data itself captured by the environmental camera 300, or may be data obtained by processing the image data. For example, the image data may be data of features extracted from the image data. Information such as the image capture time and the image capture location may be added to the image data. The image data acquisition unit 1170 may acquire image data from the camera 25 of the mobile robot 20, not limited to the image data from the environmental camera 300. In other words, the image data acquisition unit 1170 may acquire image data based on an image captured by the camera 25 provided on the mobile robot 20. The image data acquisition unit 1170 may acquire image data from a plurality of environmental cameras 300.

[0112] The feature extraction unit 1171 corresponds to a part of the group classification unit described above, and extracts features of people in a captured image. More specifically, the feature extraction unit 1171 detects people included in the image data by performing image processing on the image data. Then, the feature extraction unit 1171 extracts features of people included in the image data. In addition, the arithmetic processing unit 311 provided in the environmental camera 300 may perform at least a part of the processing for feature extraction. Note that various techniques are known to those skilled in the art as a means for detecting that a person is included in image data, such as, for example, Histograms of Oriented Gradients (HOG) features and machine learning including convolution processing. Therefore, detailed description will be omitted here.

[0113] The feature extraction unit 1171 detects the color of the clothing of the detected person. More specifically, for example, the feature extraction unit 1171 calculates the ratio of an area of ​​a specific color to the clothing of the detected person. Alternatively, the feature extraction unit 1171 detects the color of clothing in a specific part of the clothing of the detected person. In this way, the feature detection unit 511 extracts characteristic parts of the clothing of the staff member.

[0114] In addition, the characteristic shape of the staff's clothing, characteristic accessories, and other belongings may be extracted as features. Furthermore, the feature extraction unit 1171 may extract features of a face image. In other words, the feature extraction unit 1171 may extract features for face recognition. The feature extraction unit 1171 supplies the extracted feature information to the classifier 1172.

[0115] The classifier 1172 corresponds to a part of the group classification unit described above, and classifies people into a first group or a second group set in advance based on the feature extraction result. For example, the classifier 1172 classifies people based on the feature information received from the feature extraction unit 1171 and the staff information 128 stored in the storage unit 12. The classifier 1172 supplies the classification result to the selection unit 1173. The classifier 1172 classifies staff into the second group and classifies people other than staff into the first group. The classifier 1172 supplies the classification result to the selection unit 1173.

[0116] Based on the classification result, the selection unit 1173 selects non-staff mode if a person belonging to non-staff is present, and selects staff mode if a person belonging to non-staff is not present, and supplies the selection result to the switching unit 1174.

[0117] The switching unit 1174 switches between the staff mode and the non-staff mode based on the selection result by the selection unit 1173. For example, if the switching unit 1174 recognizes that no person belonging to the non-staff category exists when the non-staff mode is selected, it can switch to the staff mode, and conversely, if the switching unit 1174 recognizes that no person belonging to the non-staff category exists when the staff mode is selected, it can switch to the non-staff mode.

[0118] When the staff mode is selected, compared to when the non-staff mode is selected, a group classification process is executed so as to reduce either the number of cameras to be operated among the multiple environmental cameras 300 provided in the facility, or the number of environmental cameras to be used as information sources among the multiple environmental cameras in classification by the classifier 1172. The control target of the switching unit 1174 for such group classification process can be the environmental cameras 300 (operation / non-operation switching control of the environmental cameras 300) in the former case, and can be the image data acquisition unit 1170 in the latter case, but can also be the feature extraction unit 1171 or the classifier 1172.

[0119] When controlling the operation / non-operation of the environmental camera 300, the switching unit 1174 controls the power supply of the environmental camera 30 to be controlled. When the camera 25 of a certain mobile robot 20 is functioning as the environmental camera 300, the switching unit 1174 can also instruct the mobile robot 20 to stop the camera 25. When controlling the use / non-use as an information source, the switching unit 1174 controls the image data acquisition unit 1170 to acquire image data of the environmental camera 300 required as an information source, controls the feature extraction unit 1171 to extract features only from the image data, or controls the classifier 1172 to perform classification only from the features.

[0120] The switching unit 1174 may control the multiple cameras 300 so that, when the non-staff mode is selected, the multiple environmental cameras 300 are operated, and when the staff mode is selected, the operation of the cameras other than the first camera among the multiple environmental cameras 300 is stopped. There is no particular method for controlling the operation / non-operation (stop) of the environmental cameras 300, and existing remote power control technology may be applied. This makes it possible to reduce the processing load in the staff mode while keeping at least the first camera required for monitoring in operation.

[0121] Alternatively, the switching unit 1174 can control the multiple environmental cameras 300, etc., so that when the non-staff mode is selected, the multiple environmental cameras 300 are operated and used as information sources, and when the staff mode is selected, a first camera among the multiple environmental cameras 300 is operated and used as an information source.

[0122] The first camera may include a camera installed at a position for monitoring a security gate in the facility. The security gate may be the entrance itself or a facility installed at the entrance, but may also be a key point (monitoring point) in the facility or a facility installed at the key point. This makes it possible to reduce the processing load in the staff mode while at least the first camera installed in the part that needs to be monitored for security reasons is in operation. The first camera may be a preset parent camera among the multiple environmental cameras 300, and the other cameras may be child cameras.

[0123] Also, as in the transport system 1 exemplified here, a system control unit (arithmetic processing unit 11 in the example of FIG. 2) of the host management device 10 controls the mobile robot 20 that moves autonomously in a preset area in a facility, and the environmental camera 300 can be provided at a position separated from the running surface of the mobile robot 20 so as to capture images of the periphery of the running mobile robot 20. This allows the mobile robot 20 to be monitored by the environmental camera 300, which is also used for monitoring people.

[0124] Furthermore, the system control unit (the calculation processing unit 11 in the example of FIG. 2) of the higher-level management device 10 may control the plurality of environmental cameras 300 to change (add or switch) the camera functioning as the first camera according to the traveling position of the mobile robot 20. This allows the mobile robot to be monitored, and in the staff mode, the processing load can be reduced while at least the first cameras installed around the mobile robot 20 are in operation.

[0125] Next, an example of staff information 128 is shown in FIG. 5. FIG. 5 is a table showing an example of staff information 128. Staff information 128 is information for classifying staff and non-staff into groups corresponding to their type. The left column shows the "category" of staff. From the top, the items in the staff category are "non-staff," "pharmacist," and "nurse." Of course, items other than the items shown in the example may be included. To the right of the staff category, the columns of "clothing color" and "group classification" are shown, respectively.

[0126] The clothing color (tone) corresponding to each staff category item will be explained below. The clothing color corresponding to "non-staff" is "unspecified." In other words, when the feature extraction unit 1171 detects a person from image data, if the color of the detected person's clothing is not included in the preset colors, the feature extraction unit 1171 determines the detected person as "non-staff." Also, according to the staff information 128, the group classification corresponding to "non-staff" is the first group.

[0127] The color of the clothes is associated with the category. For example, the color of the uniform of the staff is determined for each category. In this case, the color of the uniform is different for each category. Therefore, the classifier 1172 can identify the category from the color of the clothes. Of course, the staff of one category may wear uniforms of different colors. For example, a nurse may wear a white (white coat) or pink uniform. Alternatively, the staff of multiple categories may wear uniforms of a common color. For example, a nurse and a pharmacist may wear a white uniform. Furthermore, the characteristics are not limited to the color of the clothes, and the shape of the clothes, hats, etc. may be used. Then, the classifier 1172 identifies a category that matches the characteristics of the person in the image. Of course, if two or more people are included in the image, the classifier 1172 identifies the category of each person.

[0128] The classifier 1172 can easily and appropriately determine whether a person is a staff member based on the color of the person's clothing. For example, even if a new staff member is added, it can be determined whether the person is a staff member without using information about the new staff member. Alternatively, the classifier 1172 may classify a person as a non-staff member or a staff member based on the presence or absence of a predetermined possession such as a name tag, an ID card, or an entry card. For example, the classifier 1172 classifies a person who has a name tag attached to a predetermined part of their clothing as a staff member. Alternatively, the classifier 1172 classifies a person who has an ID card or an entry card hanging from their neck by a card holder or the like as a staff member.

[0129] Furthermore, the classifier 1172 may perform classification based on the features of the face image. For example, the staff information 128 may store in advance the face image of the staff or its features. If the facial features of a person included in an image captured by the environment camera 300 can be extracted, it is possible to determine whether or not the person is a staff member by comparing the features of the face image. Furthermore, if a staff category is registered in advance, it is possible to identify the staff member from the features of the face image. Of course, the classifier 1172 can perform classification by combining multiple features.

[0130] In this way, the classifier 1172 determines whether or not a person in the image is a staff member. The classifier 1172 classifies staff members into the second group. The classifier 1172 classifies non-staff members into the first group. That is, the classifier 1172 classifies people other than staff members into the first group. In other words, the classifier 1172 classifies people who cannot be identified as staff members into the first group. Note that while it is preferable that staff members are registered in advance, new staff members may be classified by the color of their clothing.

[0131] The classifier 1172 may be a machine learning model generated by machine learning. In this case, machine learning can be performed using images captured for each staff category as training data. In other words, a machine learning model with high classification accuracy can be constructed by performing supervised learning using image data to which the staff category is labeled as a correct answer as training data. In other words, captured images of staff wearing a specified uniform can also be used as training data.

[0132] The machine learning model may perform feature extraction in the feature extraction unit 1171 and classification processing in the classifier 1172. In this case, an image including a person is input to the machine learning model, and the machine learning model outputs a classification result. A machine learning model according to the feature to be classified may be used. For example, a machine learning model that classifies based on the color of clothing and a machine learning model that classifies based on the feature amount of a face image may be used independently. Then, if the person is recognized as a staff member by any one of the machine learning models, the classifier 1172 determines that the person belongs to the second group. If the person cannot be identified as a staff member, the classifier 1172 determines that the person belongs to the first group.

[0133] (Mode Information) Fig. 6 is a table showing an example of the mode information 129. Fig. 6 shows the difference between the processing in the non-stuff mode and the stuff mode. Fig. 6 shows the items of the environment camera as the items subject to mode control. The switching unit 1174 can switch the items shown in Fig. 6 according to the mode indicated by the selection result by the selection unit 1173.

[0134] As shown in the item of the environment camera, the switching unit 1174 can use all the environment cameras 300 (or use them as information sources) in the non-stuff mode, and can use only the first camera (or use them as information sources) in the stuff mode. The switching unit 1174 can switch the power or sleep of the environment camera 300 on / off. In the stuff mode, the environment camera 300 is powered off or in a sleep state. In the non-stuff mode, the environment camera 300 operates without sleeping. In other words, the switching unit 1174 outputs a control signal for switching the power or sleep of the environment camera 300 on / off depending on the mode. In the stuff mode, the environment camera 300 is powered off or in a sleep state, thereby reducing the processing load and power consumption.

[0135] Also, a camera pixel item can be added to the mode information 129, and the switching unit 1174 can switch the number of pixels of the environment camera 300. In the stuff mode, the environment camera 300 outputs a captured image with a low number of pixels, or the image data acquisition unit 1170, the feature extraction unit 1171, or the classifier 1172 performs thinning processing so that the number of pixels of the captured image used as an information source is reduced. In the non-stuff mode, the environment camera 300 outputs a captured image with a high number of pixels, or the image data acquisition unit 1170, the feature extraction unit 1171, or the classifier 1172 performs processing using the captured image used as an information source as is.

[0136] Also, in addition to or instead of the item of camera pixels, a frame rate item can be provided, and the switching unit 1174 can switch the frame rate of the environment camera 300 depending on the mode. In the stuff mode, the environment camera 300 captures images at a low frame rate. In the non-stuff mode, the environment camera 300 captures images at a high frame rate. That is, the switching unit 1174 outputs a control signal for switching the frame rate of the captured images of the environment camera 300 depending on the mode. Since the captured images are captured at a high frame rate, the processing load on the processor and the like is higher than when the frame rate is low. Also, items other than the camera pixels and frame rate can be appropriately added to reduce the processing load and thereby reduce power consumption.

[0137] (Example of control method) 7 is a flowchart showing an example of a control method according to this embodiment. First, the image data acquisition unit 1170 acquires image data from the environmental camera 300 (S101). That is, when the environmental camera 300 captures an image of a monitoring area, the captured image is transmitted to the upper management device 10. The image data may be a moving image or a still image. Furthermore, the image data may be data obtained by performing various processes on the captured image.

[0138] Next, the feature extraction unit 1171 extracts features of people in the captured image (S102). Here, the feature extraction unit 1171 detects people included in the captured image and extracts features for each person. For example, the feature extraction unit 1171 extracts the color of the person's clothes as a feature. Of course, the feature extraction unit 1171 may extract not only the color of the clothes, but also a feature amount for face recognition or the shape of the clothes. The feature extraction unit 1171 may extract the presence or absence of a nurse's cap, the presence or absence of a name tag, the presence or absence of an ID card, etc. as features.

[0139] The classifier 1172 classifies people included in the captured image into a first group or a second group based on the characteristics of the people (S103). The classifier 1172 refers to the staff information and determines whether or not the person belongs to the second group (staff) based on the characteristics of each person. Specifically, the classifier 1172 determines that a person belongs to the second group when the color of the clothes matches the color of a uniform set in advance. In this way, all people included in the captured image are classified into the first group or the second group. Of course, the classifier 1172 can perform classification using other characteristics, not limited to the color of the clothes.

[0140] Then, the classifier 1172 judges whether or not the first group of people is present in the monitoring area (S104). If the first group of people (non-staff) is present (YES in S104), the selection unit 1173 selects the non-staff mode, which is a high load, and the switching unit 1174 controls to use all the environmental cameras 300 (or a number of environmental cameras 300 greater than the number of environmental cameras 300 in the staff mode) according to the selection result (S105).

[0141] If there is no person in the first group (NO in S104), the selection unit 1173 selects the low-load staff mode, and the switching unit 1174 performs control in accordance with the selection result so that only the first camera is used (S106).

[0142] 8 and 9 are diagrams for explaining a specific example of mode switching. FIG. 8 and FIG. 9 are schematic diagrams of a floor on which the mobile robot 20 moves, viewed from above. A room 901, a room 903, and a passage 902 are provided in the facility. The passage 902 connects the room 901 and the room 903. In FIG. 8, six environmental cameras 300 are identified as environmental cameras 300A to 300G. The environmental cameras 300A to 300G are installed at different positions and in different directions. The environmental cameras 300A to 300G capture images of different areas, and among them, the environmental camera 300G is placed at a position where it is possible to check the entry and exit of people at an entrance / exit 904 that functions as a security gate. The positions, image capture directions, image capture ranges, etc. of the environmental cameras 300A to 300F may be registered in advance in the floor map 121.

[0143] The areas assigned to the environmental cameras 300A to 300F are defined as monitoring areas 900A to 900F, respectively. For example, the environmental camera 300A captures the monitoring area 900A, and the environmental camera 300B captures the monitoring area 900B. Similarly, the environmental cameras 300C, 300D, 300E, and 300F capture the monitoring areas 900C, 900D, 900E, and 900F, respectively. The environmental camera 300G captures the range of the entrance / exit 904. In this manner, a plurality of environmental cameras 300A to 300G are installed in the target facility. The facility is divided into a plurality of monitoring areas. Information on the monitoring areas may be registered in advance in the floor map 121.

[0144] 8, when only staff U2A is present in the facility, the selection unit 1173 selects the low-load staff mode, and the switching unit 1174 performs control so as to use only the first camera in accordance with the selection result. At this time, the first camera may be, for example, only the surveillance camera 300G monitoring the entrance / exit 904.

[0145] On the other hand, as shown in FIG. 9, when a non-staff member U1B is present in the facility (whether or not there is a staff member), the selection unit 1173 selects the non-staff mode with a high load, and the switching unit 1174 switches all the environmental cameras according to the selection result. Ra3 00A~300G (or from staff mode Many The camera 300 controls the number of environmental cameras 300 to be used.

[0146] Here, for the sake of simplicity, it has been described that each of the environmental cameras 300A to 300G monitors one monitoring area, but one environmental camera 300 may monitor multiple monitoring areas. Alternatively, multiple environmental cameras 300 may monitor one monitoring area. In other words, the imaging ranges of two or more environmental cameras may overlap. In that case, it is also possible to control so that some of the overlapping environmental cameras are not used or are not used as information sources in staff mode, and all of the overlapping environmental cameras are used or are used as information sources in non-staff mode.

[0147] Furthermore, although the host management device 10 has been described as detecting the entry and exit of people based on captured images, other information may be used. For example, if an automatic door or security door is installed, entry and exit may be detected according to the operation of the door.

[0148] (Other examples of control methods) Fig. 10 is a flow chart showing another example of the control method according to this embodiment. Here, the same processes as those in Fig. 7 will be omitted and only an outline will be described. First, the image data acquisition unit 1170 acquires image data from the environment camera 300 (S101). Next, the feature extraction unit 1171 extracts features of people in the captured image (S102). Next, the classifier 1172 classifies people included in the captured image into a first group or a second group based on the features of the people (S103). The classifier 1172 refers to the staff information and determines whether or not each person belongs to the second group (staff) based on the features of each person.

[0149] Then, the classifier 1172 judges whether or not the first group of people is present in the monitoring area (S104). If the first group of people (non-staff) is present (YES in S104), the selection unit 1173 selects the non-staff mode, which is a high load, and the switching unit 1174 controls to use all the environmental cameras 300 (or a number of environmental cameras 300 greater than the number of environmental cameras 300 in the staff mode) according to the selection result (S105).

[0150] If there is no person in the first group (NO in S104), the selection unit 1173 selects the low-stress staff mode, and the switching unit 1174 acquires the traveling position of the mobile robot 20 according to the selection result, and determines the first camera according to the traveling position (S107). For example, the traveling position may be regarded as a position where light surveillance is not a problem, and the first camera may be the environmental camera 300 located outside the traveling position of the mobile robot 20, or the environmental camera 300 located outside the traveling position and traveling route. Then, the switching unit 1174 controls to use only the determined first camera (S106).

[0151] As shown in Fig. 8, when only staff member U2A is present in the facility, the selection unit 1173 selects the low-load staff mode, and the switching unit 1174 acquires the positions of the mobile robots 20A and 20B according to the selection result and determines the first camera. In this example, for example, according to the positions and moving directions of the mobile robots 20A and 20B, the first camera can be the environmental cameras 300C, 300D, and 300G so that the monitoring areas 900A and 900B and the monitoring areas 900E and 900F are not monitored, respectively. Then, the switching unit 1174 controls to use only these first cameras.

[0152] On the other hand, as shown in FIG. 9, when a non-staff member U1B is present in the facility (whether or not there is a staff member), the selection unit 1173 selects the non-staff mode with a high load, and the switching unit 1174 switches all the environmental cameras according to the selection result. Ra3 00A~300G (or from staff mode Many In the staff mode, the number of environmental cameras (300) can be controlled. Many Here is an example of the number of environmental cameras 300. For example, in the non-staff mode, the switching unit 1174 performs control according to the positions of the mobile robots 20A and 20B to exclude the monitoring areas 900A and 900E from monitoring, and to use the environmental cameras 300B, 300C, 300D, 300F, and 300G for the other monitoring areas.

[0153] Although the present embodiment has been described above, the control method according to the present embodiment may be performed by the host management device 10 or may be performed by an edge device. The edge device includes, for example, one or more of the environmental camera 300, the mobile robot 20, the communication unit 610, and the user terminal 400. The environmental camera 300, the mobile robot 20, and the host management device 10 may cooperate to execute the control method. That is, the control system according to the present embodiment may be mounted in the environmental camera 300 or the mobile robot 20. Alternatively, at least a part or the whole of the control system may be mounted in a device other than the mobile robot 20, for example, the host management device 10. The host management device 10 is not limited to being a single device physically, and may be distributed and arranged in multiple devices. That is, the host management device 10 may be equipped with multiple memories and multiple processors.

[0154] (Alternative example) The transport system 1, upper level management device 10, mobile robot 20, user terminal 400, environmental camera 300, and communication unit 610 according to the above-described embodiment are not limited to having the shapes or performing the controls illustrated, but rather need only be capable of fulfilling the functions of each of these devices.

[0155] In the above embodiment, the example was given of classification into a first group and a second group, and switching between the first operation mode and the second operation mode, but classification into three or more groups may be performed, and switching between three or more operation modes may be performed depending on whether one or more of the classified groups are included or not. In this case, each operation mode may be a mode in which the number of surveillance cameras to be used or the number of surveillance cameras to be used as information sources of image data is gradually reduced depending on the low necessity of surveillance.

[0156] In addition, in the above-described embodiment, an example was given in which the control system was incorporated into a transport system, but it does not have to be incorporated into a transport system, and the control system may be constructed as a control system for monitoring people even when a mobile robot is not used or when a mobile robot is used.

[0157] Each of the devices included in the transport system 1 according to the embodiment described above may have, for example, the following hardware configuration: Fig. 11 is a diagram showing an example of the hardware configuration of the device.

[0158] 11 may include a processor 101, a memory 102, and an interface 103. The interface 103 may include interfaces with, for example, a drive unit, a sensor, an input / output device, etc., as required depending on the device.

[0159] The processor 101 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU. The processor 101 may include multiple processors. The memory 102 is, for example, configured by a combination of a volatile memory and a non-volatile memory. The functions of each device are realized by the processor 101 reading a program stored in the memory 102 and executing the program while exchanging necessary information via the interface 103. In other words, a part or all of the processes in the above-mentioned upper management device 10, the environmental camera 300, the mobile robot 20, etc. can be realized as computer programs.

[0160] Such a program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable media can provide the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0161] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, the above-described embodiment describes a system in which a transport robot moves autonomously within a hospital, but the above-described transport system can transport specified items as luggage in a hotel, restaurant, office building, event venue, or complex. [Explanation of symbols]

[0162] 1. Transport system 10 Upper management device 11. Processing unit 12 Storage section 13 Buffer Memory 14 Communications Department 20 Mobile Robot 21 Processing unit 22 Memory section 23 Communications Department 24 Distance Sensor Group 25 Camera 26 Drive unit 27 Display section 28 Operation reception section 111 Robot control unit 115 Route Planning Department 116 Transport information acquisition unit 117 Mode control section 121 Floor Map 122 Robot Control Parameters 123 Robot Information 125 Route Planning Information 126 Delivery Information 128 Staff Information 129 Mode Information 211 Movement command extraction part 212 Drive control unit 221 Floor Map 222 Robot Control Parameters 226 Delivery Information 241 Front and rear distance sensor 242 Left and right distance sensor 260 Bogie section 261 Drive Wheel 262 Caster 281 Operation Interface 290 Main body 291 Storage 292 Door 400 User terminals 610 Communication Unit 1170 Image data acquisition unit 1171 Feature Extraction Unit 1172 classifier 1173 Selection Section 1174 Switching part

Claims

1. Executing system control to control a system including a plurality of cameras installed in the facility; execute a group classification process for recognizing features of a person captured by the camera and classifying the person into a first group or a second group that is set in advance based on the features; The system control includes: selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present; When the second operation mode is selected, compared to when the first operation mode is selected, one of the number of cameras to be operated among the plurality of cameras and the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process is reduced. Execute the group classification process; the system control controls the plurality of cameras so as to operate the plurality of cameras when the first operation mode is selected, and to stop operation of the cameras other than a first camera among the plurality of cameras when the second operation mode is selected; the system control includes control of a mobile robot that autonomously moves within a preset area within the facility; the plurality of cameras are provided at positions spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the mobile robot while the mobile robot is running; the system control controls the plurality of cameras so as to change the camera that functions as the first camera in accordance with a traveling position of the mobile robot; Control system.

2. Executing system control to control a system including a plurality of cameras installed in the facility; execute a group classification process for recognizing features of a person captured by the camera and classifying the person into a first group or a second group that is set in advance based on the features; The system control includes: selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present; When the second operation mode is selected, compared to when the first operation mode is selected, one of the number of cameras to be operated among the plurality of cameras and the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process is reduced. Execute the group classification process; the system control includes control of a mobile robot that autonomously moves within a preset area within the facility; the plurality of cameras are provided at positions spaced apart from a running surface of the mobile robot so as to capture images of the periphery of the mobile robot as it runs; Control system.

3. the system control controls the plurality of cameras so as to operate the plurality of cameras when the first operation mode is selected, and to stop operation of the cameras other than a first camera among the plurality of cameras when the second operation mode is selected. The control system of claim 2.

4. The first camera includes a camera provided at a position monitoring a security gate in the facility. The control system of claim 3.

5. The group classification process classifies the person according to the characteristics of the person's clothing or whether the person has a predetermined belonging. A control system according to any one of claims 1 to 4.

6. Executing system control to control a system including a plurality of cameras installed in the facility; execute a group classification process for recognizing features of a person captured by the camera and classifying the person into a first group or a second group that is set in advance based on the features; The system control includes: selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present; When the second operation mode is selected, compared to when the first operation mode is selected, one of the number of cameras to be operated among the plurality of cameras and the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process is reduced. Execute the group classification process; the system control controls the plurality of cameras so as to operate the plurality of cameras when the first operation mode is selected, and to stop operation of the cameras other than a first camera among the plurality of cameras when the second operation mode is selected; the system control includes control of a mobile robot that autonomously moves within a preset area within the facility; the plurality of cameras are provided at positions spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the mobile robot while the mobile robot is running; the system control controls the plurality of cameras so as to change the camera that functions as the first camera in accordance with a traveling position of the mobile robot; Control methods.

7. Executing system control to control a system including a plurality of cameras installed in the facility; execute a group classification process for recognizing features of a person captured by the camera and classifying the person into a first group or a second group that is set in advance based on the features; The system control includes: selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present; When the second operation mode is selected, compared to when the first operation mode is selected, one of the number of cameras to be operated among the plurality of cameras and the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process is reduced. Execute the group classification process; the system control includes control of a mobile robot that autonomously moves within a preset area within the facility; the plurality of cameras are provided at positions spaced apart from a running surface of the mobile robot so as to capture images of the periphery of the mobile robot as it runs; Control methods.

8. the system control controls the plurality of cameras so as to operate the plurality of cameras when the first operation mode is selected, and to stop operation of the cameras other than a first camera among the plurality of cameras when the second operation mode is selected. The control method according to claim 7.

9. The first camera includes a camera provided at a position monitoring a security gate in the facility. The control method according to claim 8.

10. The group classification process classifies the person according to the characteristics of the person's clothing or whether the person has a predetermined belonging. The control method according to any one of claims 6 to 9.

11. A program for causing a computer to execute a control method, The control method includes: Executing system control to control a system including a plurality of cameras installed in the facility; execute a group classification process for recognizing features of a person captured by the camera and classifying the person into a first group or a second group that is set in advance based on the features; The system control includes: selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present; When the second operation mode is selected, compared to when the first operation mode is selected, one of the number of cameras to be operated among the plurality of cameras and the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process is reduced. Execute the group classification process; the system control controls the plurality of cameras so as to operate the plurality of cameras when the first operation mode is selected, and to stop operation of the cameras other than a first camera among the plurality of cameras when the second operation mode is selected; the system control includes control of a mobile robot that autonomously moves within a preset area within the facility; the plurality of cameras are provided at positions spaced apart from a running surface of the mobile robot so as to capture an image of the periphery of the mobile robot while the mobile robot is running; the system control controls the plurality of cameras so as to change the camera that functions as the first camera in accordance with a traveling position of the mobile robot; program.

12. A program for causing a computer to execute a control method, The control method includes: Executing system control to control a system including a plurality of cameras installed in the facility; execute a group classification process for recognizing features of a person captured by the camera and classifying the person into a first group or a second group that is set in advance based on the features; The system control includes: selecting a first operation mode when the person belonging to the first group is present, and selecting a second operation mode different from the first operation mode when the person belonging to the first group is not present; When the second operation mode is selected, compared to when the first operation mode is selected, one of the number of cameras to be operated among the plurality of cameras and the number of cameras to be used as information sources among the plurality of cameras in the classification in the group classification process is reduced. Execute the group classification process; the system control includes control of a mobile robot that autonomously moves within a preset area within the facility; the plurality of cameras are provided at positions spaced apart from a running surface of the mobile robot so as to capture images of the periphery of the mobile robot as it runs; program.

13. the system control controls the plurality of cameras so as to operate the plurality of cameras when the first operation mode is selected, and to stop operation of the cameras other than a first camera among the plurality of cameras when the second operation mode is selected. The program according to claim 12.

14. The first camera includes a camera provided at a position monitoring a security gate in the facility. The program according to claim 13.

15. The group classification process classifies the person according to the characteristics of the person's clothing or whether the person has a predetermined belonging. The program according to any one of claims 11 to 14.

Images