Method for controlling a mobile device, control system for a mobile device, information processing method, information processing device, mobile device, method for manufacturing an article, program, and recording medium.

Abstract

To reduce the harm caused by the target individuals. [Solution] In a control method in which a mobile device (200) capable of performing a first mode of work and being movable is controlled by a processing unit (101), the mobile device (200) is capable of performing a second mode of operation that deals with the appearance of a target person (306) that meets predetermined conditions, and when the processing unit (101) detects the target person (306), it switches the operating mode of the mobile device (200) from the first mode to the second mode.

Description

Technical Field

[0001] The present invention relates to a method for controlling a mobile device, a control system for a mobile device, an information processing method, an information processing apparatus, a mobile device, a method for manufacturing an article, a program, and a recording medium.

Background Art

[0002] In recent years, there has been proposed a work robot that performs a predetermined operation and executes an abnormal operation when an abnormality occurs (see Patent Document 1). In this Patent Document 1, it is proposed that when an abnormality occurs, the robot retreats from the user's movement route, moves to the location where the abnormality occurred, or returns to the home position.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in recent years, automation has been progressing in stores, factories, etc. using robots, and it has become possible to perform operations with a small number of workers. However, for example, when a person who causes an incident such as robbery appears and attempts to deal with it with a small number of people, there is a problem that the damage may increase.

[0005] Therefore, an object of the present invention is to provide a method for controlling a mobile device, a control system for a mobile device, an information processing method, an information processing apparatus, a mobile device, a method for manufacturing an article, a program, and a recording medium that can reduce damage caused by a target person.

Means for Solving the Problems

[0006] One aspect of the present invention is a control method for controlling a mobile device that is capable of performing a first mode of work and is movable, wherein the mobile device is capable of performing a second mode of operation that deals with the appearance of a person who meets predetermined conditions, and the control unit switches the operating mode of the mobile device from the first mode to the second mode when it detects the person.

[0007] One aspect of the present invention is a control system for a mobile device, comprising a mobile device that is capable of performing a first mode of work and is movable, and an information processing device having a processing unit for controlling the mobile device, wherein the mobile device is capable of performing a second mode of action in response to the appearance of a person who meets predetermined conditions, and the processing unit switches the operating mode of the mobile device from the first mode to the second mode when it detects the person.

[0008] One aspect of the present invention is an information processing method for processing information using an information processing apparatus having a processing unit that controls a mobile device capable of performing a first mode of work and being movable, wherein the mobile device is capable of performing a second mode of operation that deals with the appearance of a subject that meets predetermined conditions, and the processing unit switches the operating mode of the mobile device from the first mode to the second mode when it detects the subject.

[0009] One aspect of the present invention is an information processing device having a processing unit that controls a mobile device capable of performing a first mode of work and being movable, the mobile device being capable of performing a second mode of action in response to the appearance of a subject that meets predetermined conditions, and the processing unit switching the operating mode of the mobile device from the first mode to the second mode when it detects the subject.

[0010] One aspect of the present invention is a mobile device that is capable of performing a first mode of work and is mobile, and has a control unit, wherein the control unit is capable of performing a second mode of action to deal with the appearance of a person who meets predetermined conditions, and when the person is detected, the control unit switches the operation mode from the first mode to the second mode. [Effects of the Invention]

[0011] According to the present invention, it is possible to reduce the damage caused by the target person. [Brief explanation of the drawing]

[0012] [Figure 1] This is a diagram showing the monitoring system according to the first embodiment. [Figure 2] (a) is a block diagram showing the configuration of a monitoring device according to the first embodiment. (b) is a block diagram showing the configuration of a robot according to the first embodiment. [Figure 3] This is a flowchart showing the process of detecting thieves in the monitoring device according to the first embodiment. [Figure 4] This is a flowchart showing the processing of the first mode in the robot according to the first embodiment. [Figure 5] This flowchart shows the processing of the monitoring system when a thief is detected according to the first embodiment. [Figure 6] This is a flowchart showing the processing of the second mode in the robot according to the first embodiment. [Figure 7] (a) is a diagram showing the layout of the store. (b) is a diagram showing the seating arrangement inside the store. [Figure 8] (a) is a diagram showing the surveillance cameras inside the store and the imaging range of each camera. (b) is a diagram showing the paths that the robot can move along. [Figure 9] (a) is a diagram showing the robot's position at the time the thief was detected. (b) is a diagram showing the robot's movement path when moving towards the thief. [Figure 10](a) shows the case where the robot is operated to move into the imaging range of the surveillance camera. (b) shows the case where the robot is operated to approach the thief. [Figure 11] (a) shows the case where the robot is operated to interrupt between the thief and the customer. (b) shows the case where one robot is operated to prevent the thief from escaping and the other robot is operated to prompt the customer to evacuate. [Figure 12] The figure shows the case where two robots are operated to prevent the thief from escaping and to prompt the customer to evacuate. [Figure 13] (a) is a block diagram showing the configuration of the surveillance device according to the second embodiment. (b) is a block diagram showing the configuration of the robot according to the second embodiment. [Figure 14] It is a flowchart showing the processing of the surveillance system when a thief according to the second embodiment is detected. [Figure 15] It is a flowchart showing the processing of the second mode in the robot according to the second embodiment.

Embodiments for Carrying out the Invention

[0013] <First Embodiment> Hereinafter, the first embodiment for carrying out the present invention will be described with reference to the drawings.

[0014] [Schematic Configuration of Surveillance System] First, the schematic configuration of the surveillance system 1 according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing the surveillance system according to the first embodiment.

[0015] As shown in Figure 1, the monitoring system 1, which acts as a control system for mobile devices, is a system that monitors a store 400, such as a restaurant. The monitoring system 1 includes a monitoring device 100, which acts as an information processing device for monitoring the store 400, and a plurality of robots 200-1, 200-2, which act as mobile devices that perform serving work (serving services) as a service in the store 400. The robots 200-1, 200-2 shown in Figure 1 are what are commonly called service robots or serving robots. Figure 1 shows a case where two robots are deployed in the store 400, but the number may be one or three or more. In the following description, if it is not necessary to distinguish between individual robots, they will simply be referred to as robot 200, and if it is necessary to distinguish them, they will be referred to as robot 200-1, 200-2, etc., using a hyphen.

[0016] Furthermore, in the store 400 shown in Figure 1, we will explain assuming the presence of an employee 301, customers 302, 303, 304, and 305, and a thief 306. The thief 306 is, for example, a person who has committed a crime such as robbery in the past, or a person who is likely to commit a crime such as robbery in the future, and in this embodiment, is a target person who meets predetermined conditions. These predetermined conditions refer to matching the information in the database, which will be described in more detail later. On the other hand, employee 301 and customers 302, 303, 304, and 305 are persons other than the target persons. In particular, customers 302, 303, 304, and 305 are persons who should be prioritized for evacuation.

[0017] [Configuration of monitoring device and robot control system] Next, the configuration of the control systems for the monitoring device 100 and the robot 200 will be explained using Figures 1 and 2. Figure 2(a) is a block diagram showing the configuration of the monitoring device according to the first embodiment. Figure 2(b) is a block diagram showing the configuration of the robot according to the first embodiment.

[0018] (Control system for monitoring devices) As shown in Figures 1 and 2(a), the monitoring device 100 is broadly composed of a processing unit 101, a storage unit 102, a detection unit 103, a communication unit 104, and a network unit 105. The processing unit 101 has a CPU 101a, and is configured to perform its function as a processing unit by executing a program stored in the storage unit 102, and performs various processes in the monitoring device 100 (see Figures 3 and 5), which will be described in detail later.

[0019] The memory unit 102 is configured to achieve its function as a memory unit by having recording media such as ROM 102a, RAM 102b, and HDD (hard disk drive) 102c. The memory unit 102 records data (information) used in various processes, which will be described in detail later, and in other words functions as a database.

[0020] The detection unit 103 is configured to function as a detection unit that detects people, including the thief 306, by having multiple surveillance cameras 103A fixed above the ceiling of the store 400 as imaging devices that acquire images (still images, moving images), as described later. In the following description, if it is not necessary to distinguish between individual surveillance cameras, they will simply be referred to as surveillance camera 103A, and if it is necessary to distinguish between them, they will be referred to as surveillance camera 103A-1 to 103A-5 using a hyphen.

[0021] The communication unit 104 is configured to function as a communication unit that communicates wirelessly with the robot 200, by having an antenna and communication equipment (not shown in the figure). The communication method of the communication unit 104 can be any wireless communication method, such as wireless LAN, infrared communication, Bluetooth (registered trademark), etc.

[0022] The network unit 105 is configured to function as a network unit that transmits and receives data to and from an external network by having an interface (I / F) 105a. The network unit 105 refers to a WAN, etc., and is connected to the so-called internet. As will be described in more detail later, it can receive input such as a learning model that has learned the operation of the continuation 06 by AI (artificial intelligence), and publicly available information from the police.

[0023] (Robot control system) As shown in Figures 1 and 2(b), the robot 200 is broadly composed of a control unit 201, an input unit 203, a communication unit 204, a drive unit 206, and an output unit 207. The control unit 201 has a CPU 201a, a ROM 202b, a RAM 202c, etc., and is configured to perform its function as a control unit by having the CPU 201a execute a program stored in the ROM 202b, etc. That is, the control unit 201 performs various processes in the robot 200 (see Figures 4, 5, and 6), which will be described in more detail later. The control unit 201 is configured to be able to execute two operation modes, which will be described in more detail later: a first mode that performs operations related to work (tasks), and a second mode that performs operations to deal with the appearance of a thief 306 as a target person that meets predetermined conditions.

[0024] The input unit 203 includes a camera 203A, a microphone 203B as an audio input unit, a distance sensor 203C, an operation unit 203D, etc., and outputs the information acquired by these as input data to the control unit 201. Specifically, the camera 203A is mounted on the robot 200 and, for example, takes images of the area in front. This captured image information is output to the control unit 201 and used for autonomous driving (collision avoidance) of the robot 200, and is also output to the monitoring device 100 via the communication unit 204 and can be used for detecting the thief 306, which will be described in more detail later. The microphone 203B is mounted on the robot 200 and, for example, acquires (records) surrounding sounds. This acquired sound information is output to the monitoring device 100 via the communication unit 204 and can be used for detecting the thief 306, which will be described in more detail later. The distance sensor 203C measures the distance to surrounding obstacles. This measured distance information is output to the control unit 201 and is used for autonomous driving (collision avoidance) of the robot 200. The control unit 203D consists of, for example, an operation panel and buttons mounted on the robot 200, and accepts various operations from the store clerk 301 and customers 302, 303, 304, and 305 (see S201, S205, S226, etc.).

[0025] The communication unit 204 is configured to function as a communication unit that communicates wirelessly with the monitoring device 100, by having an antenna and communication equipment (not shown in the figure). In other words, the communication unit 204 is configured to receive commands from the processing unit 101 of the monitoring device 100 and input them to the control unit 201. The communication method of the communication unit 204 can be any method that is compatible with the communication method of the monitoring device 100.

[0026] The drive unit 206 is configured to function as a drive unit that enables the robot 200 to move within the store 400, by having an electric motor, battery, wheels, etc. (not shown in the figure). The drive state of the drive unit 206 is controlled by the control unit 201 based on the selected path and target coordinates, and the control unit 201 also controls the robot to avoid collisions based on images captured by the camera 203A.

[0027] The output unit 207 is configured to have a display unit 207A, which is composed of a display or the like, in order to achieve the function of displaying various information to the store clerk 301, customers 302, 303, 304, 305, and thief 306. In this embodiment, the display unit 207A and the operation unit 203D are described as separate units, but the system is not limited to this, and they may be integrated using a touch panel or the like.

[0028] [Processing by monitoring system 1] Next, the various processes in the above-mentioned monitoring system 1 will be explained using diagrams. In recent years, for example, in restaurants 400, there has been a trend towards replacing the food service work from employees 301 to so-called service robots (serving robots) in order to reduce manpower. As a result, the number of restaurant employees 301 is decreasing, but if a store is attacked by thieves 306, for example, the number of employees 301 available to respond will be reduced, which could lead to greater damage. Therefore, in this embodiment, the monitoring system 1 uses a robot 200 that performs normal service work (tasks) to prevent crimes committed by thieves 306 or to reduce damage when the store is attacked. The following will explain the processes of the monitoring system 1 (control method of the mobile device, information processing method for processing information by the monitoring system 1), dividing them into the processes of the monitoring device 100 and the processes of the robot 200.

[0029] (Store details) Here, we will first explain the details of the store layout 400, the surveillance camera 103A and surveillance area Arn, the robot 200's movement path RT, etc., using Figures 7 and 8. Figure 7(a) is a diagram showing the store layout. Figure 7(b) is a diagram showing the seating arrangement inside the store. Figure 8(a) is a diagram showing the surveillance cameras inside the store and the range of each camera capable of capturing images of intent to kill. Figure 8(b) is a diagram showing the paths the robot can move.

[0030] As shown in Figure 7(a), the store 400 located in the security area ArX, which is a security zone, has a layout in which the indoor area ArIn is enclosed as a room by a structural wall 400W. The store 400 is also equipped with an automatic door 403 as a device. Opening the automatic door 403 connects the indoor area ArIN to the outdoor area ArOUT, which is an area within the store 400's premises or a public area facing the premises and is part of the security area ArX. In other words, customers 302-305 and thieves 306 enter the indoor area ArIn from the outdoor area ArOUT by passing through the automatic door 403.

[0031] As shown in Figure 7(b), the indoor area ArIN of the store 400 is equipped with multiple seats 401a to 401o, which are structural components, in each section separated by the wall 400W. The indoor area ArIN of the store 400 is also equipped with a waiting position 402 where the robot 200 waits. The robot 200 waits at the waiting position 402 until, for example, food is placed on it. Once the food is placed on it by the employee 301 and serving is instructed via the control unit 203D, the robot 200 begins to move. When the robot 200 moves, it follows the movement path RT in the movable area ArM shown in Figure 8(b) and heads towards the designated seat among the multiple seats 401a to 401o. After serving is complete, it moves back to the waiting position 402 by following the movement path RT. The robot 200 is capable of autonomous movement within the movable area ArM and is not necessarily restricted to the movement path RT, but can move while avoiding obstacles and people.

[0032] Furthermore, as shown in Figure 8(a), four surveillance cameras 103A-1 to 103A-4 are installed on the ceiling of the indoor area ArIN of store 400, and one surveillance camera 103A-5 is also installed in the outdoor area ArOUT. Of these, surveillance camera 103A-1 is capable of capturing images of surveillance area Ar1, in other words, surveillance area Ar1 is within the field of view of surveillance camera 103A-1. Similarly, surveillance camera 103A-2 is capable of capturing images of surveillance area Ar2, surveillance camera 103A-3 is capable of capturing images of surveillance area Ar3, surveillance camera 103A-4 is capable of capturing images of surveillance area Ar4, and surveillance camera 103A-5 is capable of capturing images of surveillance area Ar5.

[0033] In the indoor area ArIN, the monitoring areas Ar1 to Ar4 of surveillance cameras 103A-1 to 103A-4 are positioned to slightly overlap with adjacent monitoring areas. In other words, the indoor area ArIN is within the imaging range of one of the surveillance cameras 103A. Furthermore, the monitoring area Ar5 of surveillance camera 103A-5 covers the passageway leading to the indoor area ArIN (automatic door 403) in the outdoor area ArOUT. That is, for example, customers 302 to 305 and thieves 306 would reach the indoor area ArIN by passing through monitoring area Ar5. If the indoor area ArIN is directly connected to electrically powered moving equipment such as elevators or escalators, the monitoring area Ar5 of surveillance camera 103A-5 may cover the inside of the elevator or the top of the escalator.

[0034] (Detection of thieves in surveillance devices) First, the detection process of the thief 306 processed by the monitoring device 100 will be explained using Figure 3. Figure 3 is a flowchart showing the detection process of the thief in the monitoring device according to the first embodiment.

[0035] For example, when the power of the monitoring device 100 is turned ON, the detection process for the thief 306 is executed as shown in Figure 3. First, in step S101, the processing unit 101 of the monitoring device 100 acquires video (which may also be a still image) captured by each of the multiple monitoring cameras 103A located in the security area ArX. Similarly, in step S101, the processing unit 101 acquires video (which may also be a still image) captured by the camera 203A of the robot 200 located in the security area ArX. Furthermore, in step S101, the processing unit 101 acquires audio picked up by the microphone 203B of the robot 200 located in the security area ArX.

[0036] Next, the processing unit 101 performs image analysis on the captured video footage to detect people from the video footage using image recognition, or performs sound analysis on the recorded audio to detect people from the audio using voice recognition (S102). This image recognition or voice recognition of people can be performed using commonly used image recognition technologies such as face recognition or recognition of full-body features, or commonly used voice recognition technologies. Then, the processing unit 101 compares (verifies) the video and audio information of the detected people with the information of the person 306 in a database stored, for example, in the HDD 102c of the storage unit 102 (S103).

[0037] Here, we will explain the information about thief 306 stored in the database (i.e., HDD102c). The database stores information related to thief 306's image, such as information on the perpetrator's appearance released by the police, a learning model for image recognition of the criminal's actions, and a learning model for image recognition of the actions of people around the criminal. The database also stores information related to thief 306's image, such as information on the appearance of a person banned from entering store 400. Furthermore, the database stores information related to thief 306's voice, such as a learning model for speech recognition of the criminal's statements and a learning model for speech recognition of the statements of people around the criminal. This information is acquired via the network unit 105 and stored, for example, on HDD102c, to form a database.

[0038] The information about the suspect's appearance released by the police specifically refers to image information such as facial photographs and sketches of wanted criminals (i.e., the perpetrators) released on the internet by organizations such as the National Police Agency or International Criminal Police Organization. This information released by the police may also be a machine learning model created using AI. Furthermore, this machine learning model obtained from the police may include not only the perpetrator's physical characteristics but also behavioral characteristics such as distinctive movements and habits.

[0039] Furthermore, the learning models used for image recognition of criminal behavior are, for example, learning models of human actions that represent the actions of criminals, created through machine learning using AI at research institutions, security companies, and surveillance system development companies. Criminal behavior here refers to actions such as intentional acts of vandalism, destruction, and violence. Specifically, for example, a customer or store clerk might drop a plate by accident, but a criminal might intentionally throw a plate and drop it, and even though the result is that the plate breaks, there are differences in the actions leading up to that point. Learning models for image recognition of criminal behavior are created by learning and modeling such criminal actions.

[0040] Furthermore, the learning models used for image recognition of the behavior of those surrounding criminals are also learning models of human actions that show the behavior of those around criminals, created through machine learning using AI, etc., at research institutions, security companies, surveillance system development companies, etc. Specifically, for example, customers and store clerks may behave in fear or flee when faced with a criminal. Learning models that model the behavior of such people around criminals are used for image recognition of the behavior of those surrounding criminals. In short, if the people around a criminal can be identified, then anyone behaving differently can be detected as a criminal.

[0041] Furthermore, the learning models used for speech recognition of criminals' statements are human speech models that represent the statements of criminals, created through machine learning using AI, etc., at research institutions, security companies, surveillance system development companies, etc. Specifically, criminals may make statements that involve shouting, making strange noises, or making threats, and learning models for speech recognition of criminals are created by learning and modeling such statements made by criminals.

[0042] Furthermore, the learning models used for speech recognition of speech from people surrounding criminals are also human speech learning models that represent speech from people surrounding criminals, created through machine learning using AI, etc., at research institutions, security companies, surveillance system development companies, etc. Specifically, for example, people surrounding criminals may scream or make statements to stop the criminal, and learning models for speech recognition of speech from people surrounding criminals are created by learning and modeling such speech from people surrounding criminals.

[0043] As described above, the processing unit 101 compares the video of a person detected by multiple surveillance cameras 103A or the camera 203A of the robot 200 with the information of the thief 306 in the database to determine whether the detected person (target person) is the thief 306 (S104). Alternatively, the processing unit 101 compares the voice of a person detected by the microphone 203B of the robot 200 with the information of the thief 306 in the database to determine whether the detected person (target person) is the thief 306 (S104). In other words, it determines whether or not there is a match with the information in the database, and if there is a match with the information in the database, it is determined that the person satisfies predetermined conditions and the appearance of the thief 306 is detected. If it is determined that the detected person is not the thief 306 (normally determined as an employee 301 or customer 302-305) (No. in S104), the process returns to step S101 and the process described above is repeated.

[0044] On the other hand, if the detected person is determined to be thief 306, that is, if thief 306 is detected from the acquired video and audio information (Yes in S104), the detected person (the target person) is first designated as thief 306 (S105). In other words, the processing unit 101 turns on various flags and proceeds to execute the thief handling process (S106). Here, for example, if customers 302 to 305 behave similarly to thief 306, that is, if they behave suspiciously, they may also be designated as thieves. In that case, the store clerk 301 should re-evaluate them as customers and revoke this designation. Details of the thief handling process will be described later as the processing when a thief is detected in the surveillance system.

[0045] In the above description, we have explained a system in which the detected person is determined to be thief 306 if they match the information in the database, but this is not the only way. For example, the system may determine the detected person as a candidate for thief 306 if they match only a part of the information in the database. In this embodiment, in this case as well, the operating mode of the robot 200 is switched from the first mode to the second mode. Then, the robot 200 may be moved to deal with the candidate for thief 306, for example, by having the robot 200 wait near the candidate.

[0046] (Processing of the first mode in robots) Next, the processing in the first mode of the robot 200, that is, the processing in the mode of performing normal work (service), will be explained using Figure 4. When the control unit 201 of the robot 200 is controlled in the first mode, which is the mode of performing normal work, it executes the processing shown in Figure 4.

[0047] In the first mode, the control unit 201 of the robot 200 first waits for instructions from the employee 301. When the employee 301 gives an instruction, for example, to place food or other items on a stand (not shown) of the robot 200 and serve them via the operation unit 203D, the control unit 201 accepts the input (S201). The control unit 201 then moves towards the location instructed by the employee 301 from the store map (see Figures 7(a) and 7(b)) stored in the ROM 201b, etc. (S202). In other words, it sets a route RT (see Figure 8(b)) from its current position to the place where the food will be served (the destination seat), and moves along that route RT. During this movement, the control unit 201 drives the drive unit 206 to autonomously navigate, avoiding collisions with customers 302-305 and obstacles, based on the image from the camera 203A and the distance to nearby objects measured by the distance sensor 203C. Then, the movement process is repeated until the designated location is reached (No in S203), and once the designated location is reached (Yes in S203), the drive unit 206 is stopped and movement stops (S204).

[0048] Next, the control unit 201 waits until, for example, a customer at the seat where the food was served takes the food and the customer inputs the end of serving via the operation unit 203D, or until it detects that the food or other items on the vehicle have been removed from the camera 203A image, etc. (No in S205). When the control unit 201 detects that the customer has input the end of serving or that the items on the vehicle have been removed (Yes in S205), it moves toward the waiting position 402 (see Figure 8(b)) (S206). During this time, the control unit 201 also drives the drive unit 206 to autonomously navigate, avoiding collisions with customers 302-305 or obstacles, based on the image from the camera 203A and the distance to nearby objects detected by the distance sensor 203C. In addition, although it has been explained that the image from the camera 203A is used to detect that the items on the vehicle have been removed, it may also be possible to simply provide a weight sensor or pressure sensor and detect it using a sensor.

[0049] (Processing flow of the surveillance system when a thief is detected) Next, the processing flow of the monitoring system 1 when the monitoring device 100 detects the thief 306 as described above (see S106) will be explained using Figure 5. Figure 5 is a flowchart showing the processing of the monitoring system when a thief is detected according to the first embodiment.

[0050] As described above, when the monitoring device 100 detects the thief 306 (Yes in S104), it proceeds to execute the countermeasures against the thief 306 (S106). Then, as shown in Figure 5, the overall processing of the monitoring system 1 is as follows: First, the processing unit 101 of the monitoring device 100 commands the control unit 201 of the robot 200, and the robot 200 transitions from the first mode to the second mode. As a result, as will be described in more detail later, the processing in the second mode causes the robot 200 to move into the field of view of the monitoring camera 103A that shows the thief 306.

[0051] Next, the processing unit 101 of the monitoring device 100 acquires the layout of the store 400 (i.e., the location information of the building's structure) (see Figures 7(a) and 7(b)) which is pre-recorded in the HDD 102c (S111). Then, the processing unit 101 detects and acquires the locations of the thief 306, the store clerk 301, and the customers 302-305 by performing image recognition on the video footage captured by the multiple monitoring cameras 103A (S112). Then, as will be described in more detail later, it determines the position for the robot 200 to deal with the thief 306 and instructs the robot 200 to move to that position (S113). Upon receiving this command, the robot 200 moves to the position calculated by the monitoring device 100 according to the command (S212).

[0052] (Processing of the second mode in robots) Next, the processing when the robot 200 switches from the first mode to the second mode, that is, the processing in the second mode, will be explained using Figures 6 and 9. Figure 6 is a flowchart showing the processing in the second mode in the robot according to the first embodiment. Figure 9(a) shows the position of the robot at the time the thief is detected. Figure 9(b) shows the movement path when the robot moves toward the thief.

[0053] As shown in Figure 6, first, the control unit 201 of the robot 200 receives a command from the monitoring device 100 to transition the robot 200 to the second mode and to move to the field of view of the monitoring camera 103A that shows the thief 306 (see S113) (S221). Then, if the control unit 201 is working in the first mode, it interrupts that work (serving) (S222). The control unit 201 then transitions the robot 200's operating mode from the first mode to the second mode and performs the process of moving to the field of view of the monitoring camera 103A instructed by the processing unit 101 of the monitoring device 100 until it arrives at that location (No in S223 and S224). After that, once the robot 200 arrives at that location, it proceeds to the next process (Yes in S224). Up to this point, this corresponds to the process of step S211 explained in Figure 5.

[0054] A specific example of the process from step S221 to step S224 will be explained. For example, the robot 200 that is serving food is located outside the monitoring area Ar4, which is the field of view of the surveillance camera 103A-4 that captures the thief 306, in the movable area ArM of the store 400, as shown in Figure 9(a). When the monitoring device 100 detects the thief 306 while the robot 200 is in this position, the processing unit 101 of the monitoring device 100 commands the control unit 201 of the robot 200 to transition from the first mode to the second mode and to move into the monitoring area Ar4 (i.e., within the field of view). Upon receiving this command, the robot 200 sets a movement path RT4, as shown in Figure 9(b), and moves to the monitoring area Ar4, that is, moves so that the robot 200 is visible to the surveillance camera 103A-4. In short, the processing unit 101 calculates the position of the monitoring area Arn to which the robot 200 should move based on the surveillance camera 103A-n showing the thief 306, commands the robot 200, and the robot 200 moves to the commanded monitoring area Arn (see Figure 10(a)).

[0055] Next, as described above, when the robot 200 arrives at the designated field of view of the surveillance camera 103A, the control unit 201 moves to the position designated by the processing unit 101 of the monitoring device 100 (S225), that is, it actually performs the process of dealing with the thief 306. A specific example of how to deal with the thief 306 will be described later. This process is repeated until the processing unit 101 of the monitoring device 100 gives an instruction to deactivate the second mode (No in S228). That is, when the processing unit 101 of the monitoring device 100 determines that it has finished dealing with the thief 306, the processing unit 101 instructs the control unit 201 of the robot 200 to deactivate the second mode, that is, to switch from the second mode to the first mode (Yes in S228). Also, when the store clerk 301 determines that it has finished dealing with the thief 306, it inputs the deactivation of the second mode using the operation unit 203D of the robot 200 (Yes in S226). Then, the control unit 201 of the robot 200 sends a request to the processing unit 101 of the monitoring device 100 to deactivate the second mode (S227). Upon receiving this, the processing unit 101 of the monitoring device 100 determines that the second mode should be deactivated and instructs the control unit 201 of the robot 200 to deactivate the second mode, that is, to switch from the second mode to the first mode (Yes in S228). As a result, the control unit 201 of the robot 200 switches the operating mode to the first mode (S229), and thus the processing of the second mode is completed. During this time, the processing unit 101 of the monitoring device 100 repeatedly moves the robot 200 relative to the position of the thief 306, that is, controls it using visual feedback, so the robot 200 needs to be within the field of view of the monitoring camera 103A. Also, the processing in step S225 corresponds to the processing in step S212 explained in Figure 5.

[0056] Furthermore, in the monitoring system 1, if there are multiple robots 200, the processing unit 101 of the monitoring device 100 commands each control unit 201 of the multiple robots 200 to switch to the first mode all at once. Also, for example, if a store employee 301 makes an input to deactivate the second mode, the operation is performed on one of the multiple robots 200. In response, the processing unit 101 of the monitoring device 100 commands each control unit 201 of the multiple robots 200 to switch to the first mode all at once.

[0057] [Specific examples of how to deal with thieves] Next, specific examples of how robot 200 deals with the thief 306 will be explained using Figures 10 to 12. Figure 10(a) shows the case where the robot is operated to move into the imaging range of the surveillance camera. Figure 10(b) shows the case where the robot is operated to move closer to the thief. Figure 11(a) shows the case where the robot is operated to intervene between the thief and the customer. Figure 11(b) shows the case where one robot is operated to prevent the thief from escaping, and another robot is operated to encourage the customer to evacuate. Figure 12 shows the case where both robots are operated to prevent the thief from escaping and to encourage the customer to evacuate.

[0058] As shown in Figure 10(a), the robot 200 is moved to the monitoring area Arn, which is within the field of view of the monitoring camera 103A-n that shows the thief 306, by instruction from the processing unit 101 of the monitoring device 100 (see S211). Then, in this first embodiment, the process of dealing with the thief 306 by moving the robot 200 is initiated (see S212). The process of dealing with the thief 306 can be broadly considered as follows: the process of securing the thief 306, the process of prioritizing the safety of customers 302-305, the process of prioritizing the evacuation of customers 302-305, and the process of preventing the thief 306 from entering the room. Note that the process of dealing with the thief 306 is not necessarily limited to the robot 200 directly dealing with the thief 306, but refers to the process in which the robot 200 deals with the thief 306 as a result of its appearance, that is, the process of dealing with customers 302-305 is also included.

[0059] (Handling the apprehension of the bandits) First, let's explain the process of securing the thief 306 by moving the robot 200. Suppose the processing unit 101 of the monitoring device 100 has determined that it will perform the process of securing the thief 306 when it switches the robot 200 to the second mode (i.e., when it has determined that an emergency has occurred). Then, the processing unit 101 controls the movement of the robot 200 to perform actions that prevent the thief 306 from escaping from the robot 200's movable area, ArmM. That is, for example, as shown in Figure 10(b), the robot 200 moves close to the thief 306, and in particular moves to position itself in the thief 306's escape path, thereby preventing the thief 306 from escaping.

[0060] In this case, the processing unit 101 can control the movement of the robot 200 according to the layout of the store 400 (i.e., the position information of the structures) in the robot 200's movable area Arm. Specifically, the processing unit 101 can control the movement of the robot 200 in such a way that it corners the thief 306 between the robot 200 and structures such as the walls 400W and seats 401a to 401o of the store 400, thereby preventing his escape and capturing him.

[0061] Furthermore, in the monitoring system 1, if there are multiple robots 200, the processing unit 101 switches the operating mode of two or more of the multiple robots 200, or all of them, to the second mode. Then, the processing unit 101 can control a selected robot 200 from among the multiple (two or more) robots 200 that have been switched to the second mode to perform an action to prevent the escape of the thief 306. That is, for example, as shown in Figure 11(b), the selected robot 200-1 is controlled to move in a way that prevents the escape of the thief 306 and secures him, while the other robots 200-2 are controlled to move in a way that prioritizes the evacuation of the customer 302, as will be described later. It is conceivable that robot 200-1 would be the robot that is closest to the thief 306 at the time of detection, or a robot that was not serving food.

[0062] Furthermore, the processing unit 101 can control two or more robots 200 selected from the multiple robots 200 switched to the second mode to work in coordination without interfering with each other, thereby hindering the escape of the thief 306. That is, for example, as shown in Figure 12, it is conceivable to control the movement of two selected robots 200-1 and 200-2 to hinder the escape of the thief 306 and secure him, in other words, to secure the thief 306 by narrowing his escape route.

[0063] Furthermore, it is conceivable that the processing unit 101 of the monitoring device 100 may detect multiple thieves 306. In this case, the processing unit 101 may coordinate multiple robots 200 to prevent the escape of multiple thieves 306 and capture them. For example, one robot 200 may be moved to capture the first thief 306, and another robot 200 may be moved to capture the second thief 306. Of course, if there are three or more thieves 306, three or more robots 200 may be moved and assigned to each thief 306 to prevent their escape.

[0064] Furthermore, the processing unit 101 may control devices such as automatic doors 403 that are on the escape route of the thief 306 in the security area ArX, and perform actions to prevent the thief 306 from escaping. That is, for example, as shown in Figure 11(b), as will be described in more detail later, it is conceivable that the automatic doors 403 could be opened when the customer 302 evacuates, and that the automatic doors 403 could be closed when the thief 306 attempts to escape, thereby securing the thief 306. Note that while automatic doors 403 are used as an example of devices on the escape route of the thief 306, the system is not limited to this, and could also include electric mobility devices such as elevators and escalators, and the thief 306 could be secured by controlling these devices to stop.

[0065] (Processing that prioritizes customer safety) Next, we will explain the process of prioritizing the safety of customers 302-305 through the movement of the robot 200. Suppose the processing unit 101 of the monitoring device 100 determines that when it switches the robot 200 to the second mode (i.e., when it determines that an emergency has occurred), it should prioritize the safety of customers 302-305. Then, the processing unit 101 can control the movement of the robot 200 to perform an action that prevents the approach of the thief 306 and customers 302-305, who are not the thief 306. That is, for example, as shown in Figure 11(a), the robot 200 moves to intercept between the thief 306 and customer 302, preventing them from approaching each other. This can be considered as prioritizing the safety of customer 302. Note that the robot 200 may also move to intercept between the thief 306 and the store clerk 301, who is not the thief 306, not just customers 302-305.

[0066] Furthermore, in the monitoring system 1, if there are multiple robots 200, the processing unit 101 switches the operating mode of two or more of the multiple robots 200, or all of them, to the second mode. Then, the processing unit 101 can control a selected robot 200 from among the multiple (two or more) robots 200 that have been switched to the second mode to perform an action that prevents the thief 306 and the customers 302-305 from approaching. That is, for example, as shown in Figure 11(b), the selected robot 200-1 is controlled to move in a way that prevents the thief 306 and the customer 302 from approaching. Then, the other robots 200-2 can be controlled to move in a way that prioritizes the evacuation of the customer 302, as will be described later.

[0067] Furthermore, the processing unit 101 may control the movement of two or more robots 200 selected from among the multiple robots 200 switched to the second mode, so as to coordinate actions to prevent the thief 306 from approaching the customer 302. That is, for example, as shown in Figure 12, it is conceivable to control the movement of two selected robots 200-1 and 200-2 so as to perform actions to prevent the thief 306 from approaching the customer 302.

[0068] (Prioritizing the evacuation of customers) Next, we will explain the process of prioritizing the evacuation of customers 302-305 by moving the robot 200. Suppose the processing unit 101 of the monitoring device 100 determines that when the robot 200 is switched to the second mode (i.e., when it determines that an emergency has occurred), it will prioritize the evacuation of customers 302-305, who are not the thief 306. Then, the processing unit 101 controls the movement of the robot 200 to perform an action that encourages the evacuation of customers 302-305 from the robot 200's movable area, ArmM. That is, for example, as shown in Figure 11(b), the robot 200-2 moves away from the evacuation route of customer 302 and moves the robot 200 to encourage the evacuation of customers 302-305. This can be used to prioritize the evacuation of customer 302. Alternatively, the robot 200 may be moved to encourage the evacuation of store employee 301, who is not the thief 306.

[0069] Furthermore, in the monitoring system 1, if there are multiple robots 200, the processing unit 101 switches the operating mode of two or more of the multiple robots 200, or all of them, to the second mode. Then, the processing unit 101 can control the movement of a selected robot 200 from among the multiple (two or more) robots 200 that have been switched to the second mode, so that it performs an action to encourage the evacuation of customers 302 to 305. That is, for example, as shown in Figure 11(b), the selected robot 200-2 is controlled to move so that it performs an action to encourage customer 302 to evacuate. The other robot 200-1 is then controlled to move so that it prevents the escape of the thief 306 and secures him.

[0070] Furthermore, the processing unit 101 may control devices such as automatic doors 403 located along the evacuation route for customers 302-305 in the security area ArX, and perform actions to assist in the evacuation of customers 302-305. That is, for example, as shown in Figure 11(b), the automatic door 403 may be opened when customer 302 evacuates. Note that the automatic door 403 is used as an example of a device located along the evacuation route for customers 302-305. However, it is not limited to this, and other electrically powered moving devices such as elevators and escalators may also be used, and these may be driven and controlled in the direction in which customers 302-305 evacuate to assist in their evacuation.

[0071] (A procedure to prevent intruders from entering the room) Finally, we will explain the process of preventing the intruder 306 from entering the store 400 by moving the robot 200, that is, the process of preventing entry. Suppose the processing unit 101 of the monitoring device 100 has determined that it will perform the process of preventing the intruder 306 from entering when it switches the robot 200 to the second mode (i.e., when it determines that an emergency has occurred). That is, the processing unit 101 may detect the intruder 306 based on the image captured by the monitoring camera 103A-5 (see Figure 8(a)) which is located outside the robot 200's movable area ArM in the store 400, i.e., in the outdoor area ArOUT. In this case, the processing unit 101 may command the control unit 201 of the robot 200 to move in a way that prevents the intruder 306 from entering the store 400 (indoor area ArIN), such as blocking the automatic door 403.

[0072] Furthermore, it is conceivable that the processing unit 101 controls devices such as automatic doors 403 in the path that the intruder 306 may take to enter the room, thereby preventing the intruder 306 from entering. While automatic doors 403 are used as an example of a device in the path that the intruder 306 may take to enter, the device is not limited to this; for example, it could be an elevator, escalator, etc., and it is conceivable that the intruder 306 could be prevented from entering by controlling these devices to stop.

[0073] [Summary of the First Embodiment] As described above, in the monitoring system 1 according to this first embodiment, when the processing unit 101 of the monitoring device 100 detects a thief 306 in the security area ArX, it switches the operating mode of the robot 200 from the first mode to the second mode. As a result, even if there are only a few employees in the store 400, for example, the robot 200 will deal with the thief 306, thereby reducing the damage caused by the thief 306.

[0074] In particular, when the robot 200 is switched to the second mode, it performs actions to prevent the thief 306 from approaching the passengers 302-305, that is, it moves to intercept the thief 306 and the passengers 302-305. This allows the safety of the passengers 302-305 to be prioritized.

[0075] Furthermore, when the robot 200 switches to the second mode, it performs an action to prompt customers 302-305 to evacuate from the robot 200's movable area, ArmM, meaning the robot 200 moves to evacuate customers 302-305 from their evacuation route in the store 400. This allows for prioritizing the evacuation of customers 302-305.

[0076] Furthermore, when the robot 200 switches to the second mode, it performs actions to prevent the thief 306 from escaping from the robot 200's movable area, ArmM, that is, the robot 200 moves to position itself in the thief 306's escape route within the store 400. This makes it possible to apprehend the thief 306.

[0077] <Second Embodiment> Next, a second embodiment, which is a modified version of the first embodiment described above, will be explained using Figures 13 to 15. Figure 13(a) is a block diagram showing the configuration of the monitoring device according to the second embodiment. Figure 13(b) is a block diagram showing the configuration of the robot according to the second embodiment. Figure 14 is a flowchart showing the processing of the monitoring system when a thief is detected according to the second embodiment. Figure 15 is a flowchart showing the processing of the second mode in the robot according to the second embodiment. In this explanation of the second embodiment, the same reference numerals are used for parts similar to those in the first embodiment, and their explanations are omitted.

[0078] In the monitoring system 1 according to this second embodiment, as shown in Figure 13(a), the monitoring device 100 is equipped with an output unit 107, and this output unit 107 has a speaker 107B as an audio output unit. Also, in this monitoring system 1, as shown in Figure 13(b), the robot 200 has a speaker 207B as an audio output unit in its output unit 207. These speakers 107B and 207B are used to output sound to the thief 306, thereby persuading or warning the thief 306 by voice. Note that speaker 107B may be, for example, a speaker installed on the ceiling of the store 400, and may be a speaker that plays music or announcements during normal business hours.

[0079] (Processing flow of the monitoring system when a thief is detected according to the second embodiment) First, in the second embodiment, the processing flow of the monitoring system 1 when the monitoring device 100 detects the thief 306 (see S106) will be explained using Figure 14.

[0080] Similar to the first embodiment, when the monitoring device 100 detects the thief 306 (Yes in S104), the system proceeds to execute the countermeasures against the thief 306 (S106). Then, as shown in Figure 14, the overall processing of the monitoring system 1 is as follows: First, the processing unit 101 of the monitoring device 100 commands the control unit 201 of the robot 200, and the robot 200 transitions from the first mode to the second mode. This causes the robot 200 to move into the field of view of the monitoring camera 103A that shows the thief 306.

[0081] Next, the processing unit 101 of the monitoring device 100 acquires the layout of the store 400 (S111). Then, the processing unit 101 detects and acquires the positions of the thief 306, the store clerk 301, and the customers 302-305 by performing image recognition on the video captured by the multiple monitoring cameras 103A (S112). Then, it determines the position where the robot 200 will deal with the thief 306 and instructs the robot 200 to move to that position (S123). In addition, in this second embodiment, it also calculates and determines whether or not to speak to the thief 306 from the speaker 107B and / or the speaker 207B of the robot 200, and outputs an instruction to the robot 200 accordingly.

[0082] Then, in the monitoring device 100, the processing unit 101 determines whether or not to speak to the thief 306 using the speaker 107B (S124). If it does not speak (No in S124), it returns to step S112. If it does speak to the thief 306 using the speaker 107B (Yes in S124), it speaks into the store 400 with the content determined by the processing unit 101 (S125), and returns to step S112.

[0083] Meanwhile, in step S123, the robot 200, having received a command from the processing unit 101 of the monitoring device 100, moves to the position calculated by the monitoring device 100 according to that command (S212). Then, according to the content of the command from the processing unit 101, the robot 200 determines whether or not to speak to the thief 306 using its speaker 207B (S213). If it does not speak (No in S213), it returns to step S112. If it does speak to the thief 306 using its speaker 207B (Yes in S213), it speaks to the thief 306 according to the content determined by the processing unit 101 (S214), and returns to step S112.

[0084] (Processing of the second mode in the robot in the second embodiment) Next, the processing of the robot 200 in the second mode in the second embodiment will be explained using Figure 15. As shown in Figure 15, first, the control unit 201 of the robot 200 receives a command from the monitoring device 100 to transition the robot 200 to the second mode and to move to the field of view of the monitoring camera 103A that shows the thief 306 (see S113) (S221). Then, if the control unit 201 is working in the first mode, it interrupts that work (serving) (S222). The control unit 201 then transitions the operating mode of the robot 200 from the first mode to the second mode and performs the process of moving to the field of view of the monitoring camera 103A instructed by the processing unit 101 of the monitoring device 100 until it arrives at that location (No in S223 and S224). After that, once the robot 200 arrives at that location, it proceeds to the next process (Yes in S224). Up to this point, this corresponds to the processing of step S211 explained in Figure 14.

[0085] Next, as described above, when the robot 200 arrives at the designated field of view of the surveillance camera 103A, the control unit 201 moves to the position indicated by the processing unit 101 of the surveillance device 100 (S225), that is, it actually performs the process of dealing with the thief 306. This movement process corresponds to the process of step S212 explained in Figure 14.

[0086] In this second embodiment, the control unit 201 of the robot 200 determines whether or not there is a voice command from the processing unit 101 of the monitoring device 100 (S230). This determination corresponds to the process in step S213 as described in Figure 14. If there is a voice command from the processing unit 101 of the monitoring device 100 (Yes in S230), the robot plays the voice data received from the processing unit 101 of the monitoring device 100, that is, it speaks to the thief 306 (S231), and proceeds to step S226. This determination corresponds to the process in step S214 as described in Figure 14. If there is no voice command from the processing unit 101 of the monitoring device 100 (No in S230), the robot proceeds directly to step S226. Specific details regarding the voice command to the thief 306 will be described later.

[0087] The process is then repeated until the processing unit 101 of the monitoring device 100 issues an instruction to deactivate the second mode (No in S228). That is, when the processing unit 101 of the monitoring device 100 determines that it has finished dealing with the thief 306, it instructs the control unit 201 of the robot 200 to deactivate the second mode, i.e., to switch from the second mode to the first mode (Yes in S228). Also, when the store clerk 301 determines that it has finished dealing with the thief 306, it inputs to deactivate the second mode using the operation unit 203D of the robot 200 (Yes in S226). Then, the control unit 201 of the robot 200 sends a request to the processing unit 101 of the monitoring device 100 to deactivate the second mode (S227). In response, the processing unit 101 of the monitoring device 100 determines that the second mode has been deactivated, and instructs the control unit 201 of the robot 200 to deactivate the second mode, that is, to switch from the second mode to the first mode (Yes in S228). As a result, the control unit 201 of the robot 200 switches the operating mode to the first mode (S229), and thus the processing of the second mode is completed.

[0088] Also, similar to the first embodiment, if there are multiple robots 200 in the monitoring system 1, the processing unit 101 of the monitoring device 100 commands each control unit 201 of the multiple robots 200 to switch to the first mode all at once. Furthermore, for example, if a store employee 301 makes an input to deactivate the second mode, the operation is performed on one of the multiple robots 200. In response, the processing unit 101 of the monitoring device 100 commands each control unit 201 of the multiple robots 200 to switch to the first mode all at once.

[0089] [Specific examples of how to deal with thieves] Next, we will explain a specific example of how sound can be output to the thief 306 using the speaker 107B of the monitoring device 100 (the speaker of the store 400) or the speaker 207B of the robot 200.

[0090] (A process to persuade or warn the thief) Suppose the processing unit 101 of the monitoring device 100 determines that when the robot 200 is switched to the second mode (i.e., when it determines that an emergency has occurred), it will output sound to the thief 306 using speaker 107B or speaker 207B. Then, the processing unit 101 outputs sound to the thief 306 using speaker 107B or speaker 207B. In this case, when outputting sound from speaker 107B, speaker 107B will be directed towards the thief 306 in a manner similar to an in-store announcement. Also, when outputting sound from speaker 207B of the robot 200, as described above, the robot 200 will be moved so that it is within the field of view of the monitoring camera 103A that shows the thief 306, and then speaker 207B will be directed towards the thief 306.

[0091] The content that these speakers 107B or 207B emit towards the thief 306 is likely to first output a message persuading the thief 306 not to commit the crime. The processing unit 101 obtains information about the thief 306 from the database information recorded on the HDD 102c or via the network unit 105 (i.e., the external network), and generates persuasive content as audio data based on the obtained information. By emitting this message, the thief 306 can be persuaded not to commit the crime, thereby reducing the damage caused by the thief 306.

[0092] Furthermore, the processing unit 101 retrieves information about the sentence the thief 306 would receive if he committed the crime, either from the database information recorded on the HDD 102c or via the network unit 105 (i.e., the external network), and generates voice data conveying the retrieved sentence. This voice notification warns the thief 306 of the risks of committing the crime and, in a broader sense, persuades the thief 306 not to commit the crime, thereby reducing the damage caused by the thief 306.

[0093] Furthermore, in the monitoring system 1, if there are multiple robots 200, the processing unit 101 switches the operating mode of two or more of the multiple robots 200, or all of them, to the second mode. Then, the processing unit 101 can output sound to the thief 306 through the speaker 207B installed in a robot 200 selected from among the multiple robots 200 that have been switched to the second mode. In other words, the processing unit 101 can use the robot 200 that has moved toward the thief 306, or in other words, the robot 200 closest to the thief 306, to deliver persuasion or warning.

[0094] Furthermore, the processing unit 101 may determine the language used by the thief 306 from, for example, the video footage of the thief 306 captured by the camera 203A of the robot 200, or from the voice of the thief 306 obtained from the microphone 203B. In other words, the language used by the thief 306 may be determined from the database information recorded on the HDD 102c, or from the information obtained via the network unit 105 (i.e., the external network). Then, when the processing unit 101 generates audio data containing persuasive or warning messages, it may translate the audio data into that language so that the thief 306 can understand it. This makes it possible to persuade the thief 306 not to commit the crime and reduce the damage caused by the thief 306.

[0095] [Summary of the second embodiment] As described above, in the monitoring system 1 according to this second embodiment, when the processing unit 101 of the monitoring device 100 detects the thief 306 in the security area ArX, it outputs a sound to the thief 306 via speaker 107B or speaker 207B. This makes it possible to persuade the thief 306 not to commit the crime and reduce the damage caused by the thief 306.

[0096] Furthermore, the configuration, operation, and effects of the second embodiment described above are the same as those of the first embodiment, so their description will be omitted.

[0097] <Possibility of other embodiments> In the first and second embodiments described above, a robot 200 that performs services such as serving food was described as an example of a mobile device, but it is not limited to this. For example, any mobile device that performs the first mode of work, such as a mobile industrial robot, cleaning robot, transport robot, or drone, may be used.

[0098] Furthermore, the moving device may be a robot used to manufacture goods (industrial robot). In this case, for example, a robot that transports parts necessary for manufacturing goods, or a robot that transports finished goods, are possible. In other words, a manufacturing method can be carried out using a robot in the first mode to manufacture goods. Industrial robots can also come in various forms, such as vertical articulated robots, parallel link robots, and linear articulated robots.

[0099] Furthermore, although the monitoring device 100 was described as being installed in the store 400 in the first and second embodiments, it is not limited to this. For example, the monitoring device 100 may be a server device located in a different location from the store 400 via the internet or the like. Also, the configuration of the monitoring device 100 does not need to consist of all the components shown in Figure 2 as a single unit; each component may be located in multiple different locations. For example, the processing unit 101 may be located in the store 400, while the storage unit 102 may be located in a different location from the store 400 as an external server or cloud server, etc.

[0100] Furthermore, although the second embodiment described a voice directed at the thief 306, it is not limited to this, and could also be a voice directed at the store clerk 301 or customers 302-305 to evacuate.

[0101] This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0102] <Disclosure of this embodiment> This disclosure includes at least the following: (Method 1) In a control method for controlling a mobile device that is capable of performing a first mode of work and is also mobile, a processing unit controls the device, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions, When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. A method for controlling a mobile device, characterized by the following: (Method 2) When the moving device is switched to the second mode, the processing unit controls the moving device to perform an action that prevents the subject from approaching other persons. A method for controlling a mobile device according to Method 1, characterized in that it is a method for controlling a mobile device. (Method 3) The aforementioned processing unit, When the aforementioned subject is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The mobile device selected from the plurality of mobile devices switched to the second mode is controlled to perform an action that prevents the subject from approaching persons other than the subject. A method for controlling a mobile device according to method 2, characterized in that it is a method for controlling a mobile device. (Method 4) When the mobile device is switched to the second mode, the processing unit controls the mobile device to perform an action to encourage the evacuation of persons other than the target person from the mobile device's movable area. A method for controlling a mobile device according to any one of methods 1 to 3, characterized in that (Method 5) The aforementioned processing unit, When the aforementioned subject is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The mobile device selected from the plurality of mobile devices that have been switched to the second mode is controlled to perform an action to encourage the evacuation of persons other than the target person. A method for controlling a mobile device according to method 4, characterized in that it is a method for controlling a mobile device. (Method 6) When the mobile device is switched to the second mode, the processing unit controls devices located along evacuation routes accessible to persons other than the target person, and assists in the evacuation of persons other than the target person. A method for controlling a mobile device according to any one of methods 1 to 5, characterized in that (Method 7) When the mobile device is switched to the second mode, the processing unit controls the mobile device to perform an action that prevents the subject from escaping from the mobile device's movable area. A method for controlling a mobile device according to any one of methods 1 to 6, characterized in that (Method 8) The processing unit controls the moving device to move the subject between the structure and the object, according to the positional information of the structure in the movable area. A method for controlling a mobile device according to method 7, characterized in that it is a method for controlling a mobile device. (Method 9) The aforementioned processing unit, When the aforementioned subject is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The mobile device selected from the plurality of mobile devices that have been switched to the second mode is controlled to perform an action that hinders the escape of the subject. A method for controlling a mobile device according to method 7 or 8, characterized in that it is a method for controlling a mobile device. (Method 10) The processing unit controls two or more of the multiple mobile devices that have been switched to the second mode to coordinate actions to prevent the target from escaping. A method for controlling a mobile device according to method 9, characterized in that it is a method for controlling a mobile device. (Method 11) When the processing unit detects multiple subjects who meet predetermined conditions, it controls two or more selected mobile devices to coordinate and perform actions to prevent the escape of the subjects. A method for controlling a mobile device according to method 10, characterized in that it is a method for controlling a mobile device. (Method 12) When the mobile device is switched to the second mode, the processing unit controls devices in the path from which the subject can escape, and prevents the subject from escaping. A method for controlling a mobile device according to any one of methods 1 to 11, characterized in that (Method 13) When the mobile device is switched to the second mode, the processing unit outputs sound to the target person using the sound output unit. A method for controlling a mobile device according to any one of methods 1 to 12, characterized in that (Method 14) The processing unit outputs a voice message to the target person using the voice output unit to persuade them not to commit the crime. A method for controlling a mobile device according to method 13, characterized in that it is a method for controlling a mobile device. (Method 15) The processing unit acquires information about the subject via an external network and generates persuasive content based on the acquired information. A method for controlling a mobile device according to method 14, characterized in that it is a method for controlling a mobile device. (Method 16) The processing unit determines the sentence to be imposed if the subject commits the crime, and generates content that conveys the determined sentence. A method for controlling a mobile device according to method 14 or 15, characterized in that it is a method for controlling a mobile device. (Method 17) The aforementioned processing unit, When the aforementioned subject is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The audio output unit provided in the mobile device selected from among the multiple mobile devices that have been switched to the second mode outputs audio toward the target person. A method for controlling a mobile device according to any one of methods 13 to 16, characterized in that (Method 18) The processing unit detects the target person by comparing the image captured by the imaging device located in the security area with information in the database. A method for controlling a mobile device according to any one of methods 1 to 17, characterized in that (Method 19) The processing unit performs image recognition on the video to determine the appearance of the subject, and then compares this with information in a database to detect the subject. A method for controlling a mobile device according to method 18, characterized in that it is a method for controlling a mobile device. (Method 20) The information in the aforementioned database includes information on the appearance of the suspect released by the police. A method for controlling a mobile device according to method 19, characterized in that it is a method for controlling a mobile device. (Method 21) The processing unit performs image recognition on the video to determine the subject's movements, compares them with information in a database, and detects the subject. A method for controlling a mobile device according to any one of methods 18 to 20, characterized in that... (Method 22) The information in the aforementioned database includes a learning model for image recognition of criminal behavior. A method for controlling a mobile device according to method 21, characterized in that it is a method for controlling a mobile device. (Method 23) The processing unit performs image recognition on the video to detect the actions of people around the subject, and compares them with information in a database to detect the subject. A method for controlling a mobile device according to any one of methods 18 to 22, characterized in that (Method 24) The information in the aforementioned database includes a learning model for image recognition of the behavior of people around criminals. A method for controlling a mobile device according to method 23, characterized in that it is a method for controlling a mobile device. (Method 25) The information in the aforementioned database is obtained via an external network. A method for controlling a mobile device according to any one of methods 18 to 24, characterized in that... (Method 26) Multiple imaging devices are installed in the security area. The processing unit controls the moving device, which has been switched to the second mode, to move into the field of view of one of the multiple imaging devices in which the subject is within the field of view. A method for controlling a mobile device according to any one of methods 18 to 25, characterized in that... (Method 27) Multiple imaging devices are installed in the security area. At least one of the imaging devices is also installed outside the room where the mobile device is located within the security area. A method for controlling a mobile device according to any one of methods 18 to 26, characterized in that... (Method 28) When the processing unit detects the subject based on the image captured by the imaging device located outside the room, it switches the mobile device to the second mode and controls the mobile device to perform an action to prevent the subject from entering the room. A method for controlling a mobile device according to method 27, characterized in that it is a method for controlling a mobile device. (Method 29) The imaging device is a surveillance camera installed above the security area. A method for controlling a mobile device according to any one of methods 18 to 28, characterized in that... (Method 30) The imaging device is a camera installed on the mobile device. A method for controlling a mobile device according to any one of methods 18 to 28, characterized in that... (Method 31) The processing unit detects the target person by comparing the audio acquired by the voice input unit located in the security area with the information in the database. A method for controlling a mobile device according to any one of methods 1 to 30, characterized in that (Method 32) The processing unit performs speech recognition on the audio to identify the subject's statements, compares them with the information in the database, and detects the subject. A method for controlling a mobile device according to method 31, characterized in that it is a method for controlling a mobile device. (Method 33) The information in the aforementioned database includes a learning model for speech recognition of criminals' statements. A method for controlling a mobile device according to method 32, characterized in that it is a method for controlling a mobile device. (Method 34) The processing unit performs speech recognition on the audio to identify the statements of people around the subject, and compares them with information in the database to detect the subject. A method for controlling a mobile device according to any one of methods 31 to 33, characterized in that... (Method 35) The information in the aforementioned database includes a learning model for speech recognition of speeches made by people around the criminal. A method for controlling a mobile device according to method 34, characterized in that it is a method for controlling a mobile device. (Method 36) The information in the aforementioned database is obtained via an external network. A method for controlling a mobile device according to any one of methods 31 to 35, characterized in that... (Method 37) The aforementioned audio input unit is a microphone installed on the mobile device. A method for controlling a mobile device according to any one of methods 31 to 36, characterized in that... (Method 38) When the processing unit detects a candidate who may be the target person in the security area, it switches the operating mode of the mobile device from the first mode to the second mode. A method for controlling a mobile device according to any one of methods 1 to 37, characterized in that (Method 39) When the moving device is switched to the second mode, the processing unit controls the moving device to move closer to the candidate. A method for controlling a moving device according to method 38, characterized in that it is a method for controlling a moving device. (Method 40) The mobile device is a robot that performs work on persons other than the target person in the first mode. A method for controlling a mobile device according to any one of methods 1 to 39, characterized in that (Composition 41) A mobile device capable of performing a first mode of work and being movable, The system comprises an information processing device having a processing unit for controlling the aforementioned mobile device, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions, When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. A control system for a mobile device characterized by the following features. (Composition 42) The mobile device is a robot that manufactures articles as an operation in the first mode. A control system for a mobile device according to configuration 41, characterized by the features described above. (Method 43) An information processing method for processing information using an information processing device having a processing unit that controls a mobile device capable of performing a first mode of work and being movable, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions, When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. An information processing method characterized by the following: (Composition 44) It has a processing unit that controls a mobile device capable of performing a first mode of work and is also mobile, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions, When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. An information processing device characterized by the following: (Composition 45) A mobile device that is capable of performing a first mode of work, is mobile, and has a control unit, The control unit, A second mode is available that performs actions to deal with the appearance of a target person who meets predetermined conditions. When the aforementioned target person is detected, the operating mode is switched from the first mode to the second mode. A mobile device characterized by the following features. (Method 46) A control system for the mobile device described in configuration 42 is used to manufacture an article. A method for manufacturing an article characterized by the following: (Composition 47) A program for causing a computer to execute the control method of a mobile device described in any one of methods 1 to 40. (Composition 48) A computer-readable recording medium on which the program described in Configuration 47 is recorded. [Explanation of symbols]

[0103] 1…Surveillance system (control system for mobile devices) / 100…Surveillance device (information processing device) / 101…Processing unit / 103A…Surveillance camera (imaging device) / 107B…Speaker (audio output unit) / 200…Robot (mobile device) / 201…Control unit / 203A…Camera (imaging device) / 203B…Microphone (audio input unit) / 207B…Speaker (audio output unit) / 301…Employee (person other than the target) / 302~305…Customer (person other than the target) / 306…Thief (target) / 400W…Wall (structure) / 401a~401o…Seat (structure) / 403…Automatic door (device) / ArIN…Indoor area (room) / ArM…Movable area / ArX…Security area (security area)

Claims

1. In a control method for controlling a mobile device that is capable of performing a first mode of work and is also mobile, a processing unit controls the device, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions. When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. A method for controlling a mobile device, characterized by the following:

2. When the moving device is switched to the second mode, the processing unit controls the moving device to perform an action that prevents the subject from approaching other persons. A method for controlling a mobile device according to feature 1.

3. The aforementioned processing unit, When the aforementioned target person is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The mobile device selected from the plurality of mobile devices switched to the second mode is controlled to perform an action that prevents the subject from approaching persons other than the subject. The control method for the mobile device according to feature 2.

4. When the mobile device is switched to the second mode, the processing unit controls the mobile device to perform an action to encourage the evacuation of persons other than the target person from the mobile device's movable area. A method for controlling a mobile device according to feature 1.

5. The aforementioned processing unit, When the aforementioned target person is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The mobile device selected from the plurality of mobile devices that have been switched to the second mode is controlled to perform an action to encourage the evacuation of persons other than the target person. The control method for the mobile device according to feature 4.

6. When the mobile device is switched to the second mode, the processing unit controls devices located along evacuation routes accessible to persons other than the target person, and assists in the evacuation of persons other than the target person. A method for controlling a mobile device according to feature 1.

7. When the mobile device is switched to the second mode, the processing unit controls the mobile device to perform an action that prevents the subject from escaping from the mobile device's movable area. A method for controlling a mobile device according to feature 1.

8. The processing unit controls the moving device to move the subject between the structure and the object, according to the positional information of the structure in the movable area. A method for controlling a mobile device according to feature 7.

9. The aforementioned processing unit, When the aforementioned target person is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The mobile device selected from the plurality of mobile devices that have been switched to the second mode is controlled to perform an action that hinders the escape of the subject. A method for controlling a mobile device according to feature 7.

10. The processing unit controls two or more of the multiple mobile devices selected from the multiple mobile devices switched to the second mode to coordinate actions that prevent the target from escaping. A method for controlling a mobile device according to feature 9.

11. When the processing unit detects multiple subjects who meet predetermined conditions, it controls two or more selected mobile devices to coordinate and perform actions to prevent the escape of the subjects. A method for controlling a mobile device according to the 10th feature.

12. When the mobile device is switched to the second mode, the processing unit controls devices located along the escape route of the subject and prevents the subject from escaping. A method for controlling a mobile device according to feature 1.

13. When the mobile device is switched to the second mode, the processing unit outputs sound to the target person using the sound output unit. A method for controlling a mobile device according to feature 1.

14. The processing unit outputs a voice message to the target person using the voice output unit to persuade them not to commit the crime. A control method for a mobile device according to feature 13.

15. The processing unit acquires information about the subject via an external network and generates persuasive content based on the acquired information. The control method for a mobile device according to feature 14.

16. The processing unit determines the sentence to be imposed if the subject commits the crime, and generates content that conveys the determined sentence. The control method for a mobile device according to feature 14.

17. The aforementioned processing unit, When the aforementioned target person is detected, the operating mode of the multiple mobile devices is switched from the first mode to the second mode. The audio output unit provided in the mobile device selected from the plurality of mobile devices that have been switched to the second mode outputs audio toward the target person. A control method for a mobile device according to feature 13.

18. The processing unit detects the target person by comparing the image captured by the imaging device located in the security area with information in the database. A method for controlling a mobile device according to feature 1.

19. The processing unit performs image recognition on the video to determine the appearance of the subject, and then compares this with information in a database to detect the subject. A method for controlling a mobile device according to claim 18, characterized by its features.

20. The information in the aforementioned database includes information on the appearance of the suspect released by the police. A method for controlling a mobile device according to feature 19.

21. The processing unit performs image recognition on the video to determine the subject's movements, compares them with information in a database, and detects the subject. A method for controlling a mobile device according to claim 18, characterized by its features.

22. The information in the aforementioned database includes a learning model for image recognition of criminal behavior. The control method for a mobile device according to claim 21.

23. The processing unit performs image recognition on the video to detect the actions of people around the subject, and compares them with information in a database to detect the subject. A method for controlling a mobile device according to claim 18, characterized by its features.

24. The information in the aforementioned database includes a learning model for image recognition of the behavior of people around criminals. The control method for the mobile device according to feature 23.

25. The information in the aforementioned database is obtained via an external network. A method for controlling a mobile device according to claim 18, characterized by its features.

26. Multiple imaging devices are installed in the security area. The processing unit controls the moving device, which has been switched to the second mode, to move into the field of view of one of the plurality of imaging devices in which the subject is within the field of view. A method for controlling a mobile device according to claim 18, characterized by its features.

27. Multiple imaging devices are installed in the security area. At least one of the imaging devices is also installed outside the room where the mobile device is located within the security area. A method for controlling a mobile device according to claim 18, characterized by its features.

28. When the processing unit detects the subject based on the image captured by the imaging device located outside the room, it switches the mobile device to the second mode and controls the mobile device to perform an action to prevent the subject from entering the room. The control method for a mobile device according to the feature described in 27.

29. The imaging device is a surveillance camera installed above the security area. A method for controlling a mobile device according to claim 18, characterized by its features.

30. The imaging device is a camera installed on the mobile device. A method for controlling a mobile device according to claim 18, characterized by its features.

31. The processing unit detects the target person by comparing the audio acquired by the voice input unit located in the security area with the information in the database. A method for controlling a mobile device according to feature 1.

32. The processing unit performs speech recognition on the audio to identify the subject's statements, compares them with the information in the database, and detects the subject. The control method for a mobile device according to feature 31.

33. The information in the aforementioned database includes a learning model for speech recognition of criminals' statements. The control method for a mobile device according to feature 32.

34. The processing unit performs speech recognition on the audio to identify the statements of people around the subject, and compares them with information in the database to detect the subject. The control method for a mobile device according to feature 31.

35. The information in the aforementioned database includes a learning model for speech recognition of speeches made by people around the criminal. The control method for a mobile device according to feature 34.

36. The information in the aforementioned database is obtained via an external network. The control method for a mobile device according to feature 31.

37. The aforementioned audio input unit is a microphone installed on the mobile device. The control method for a mobile device according to feature 31.

38. When the processing unit detects a potential target in the security area, it switches the operating mode of the mobile device from the first mode to the second mode. A method for controlling a mobile device according to feature 1.

39. When the moving device is switched to the second mode, the processing unit controls the moving device to move closer to the candidate. The control method for a mobile device according to the feature described in 38.

40. The mobile device is a robot that performs work on persons other than the target person in the first mode. A method for controlling a mobile device according to feature 1.

41. A mobile device capable of performing a first mode of work and being movable, The system comprises an information processing device having a processing unit for controlling the aforementioned mobile device, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions. When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. A control system for a mobile device characterized by the following features.

42. The mobile device is a robot that manufactures articles as an operation in the first mode. A control system for a mobile device according to feature 41.

43. An information processing method for processing information using an information processing device having a processing unit that controls a mobile device capable of performing a first mode of work and being movable, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions. When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. An information processing method characterized by the following:

44. It has a processing unit that controls a mobile device capable of performing a first mode of work and is also mobile, The aforementioned mobile device is capable of executing a second mode that performs an action to deal with the appearance of a person who meets predetermined conditions. When the processing unit detects the target person, it switches the operating mode of the mobile device from the first mode to the second mode. An information processing device characterized by the following:

45. A mobile device that is capable of performing a first mode of work, is mobile, and has a control unit, The control unit, A second mode is available that performs actions to deal with the appearance of a target person who meets predetermined conditions. When the aforementioned target person is detected, the operating mode is switched from the first mode to the second mode. A mobile device characterized by the following features.

46. A control system for a mobile device described in claim 42 is used to manufacture an article. A method for manufacturing an article characterized by the following:

47. A program for causing a computer to execute the control method of the mobile device described in claim 1.

48. A computer-readable recording medium on which the program described in claim 47 is recorded.

Images