Person monitoring device, person monitoring system, person monitoring method, and recording medium

Abstract

This person monitoring device comprises: a detection means that detects, using an image captured by a camera installed in a store, a monitoring subject, who is likely to perform a fraudulent act; a first determination means that determines, on the basis of the behavior and / or belongings of the monitoring subject, whether or not a mobile robot needs to be deployed for the monitoring subject; and a control means that, when it is determined that a mobile robot needs to be deployed, deploys a mobile robot for the monitoring subject.

Description

Person monitoring device, person monitoring system, person monitoring method, and recording medium

[0001] The present disclosure relates to a person monitoring device, a person monitoring system, a person monitoring method, and a recording medium.

[0002] There are mobile robots that provide assistance to customers in public places. For example, Patent Literature 1 discloses a mobile robot that assists customers by moving around while carrying their luggage.

[0003] Japanese Patent Application Laid-Open No. 2021-174460

[0004] In public places, there are mobile robots that monitor specific individuals to prevent shoplifting and other illegal activities. However, since the number of mobile robots is limited, it is necessary to deploy mobile robots around the individuals who need to be tracked.

[0005] An example of an objective of the present disclosure is to provide a person monitoring device that effectively deters fraudulent activities using a mobile robot.

[0006] A person monitoring device in one aspect of the present disclosure includes a detection means that detects a monitored person who may be committing fraud using images captured by a camera installed in a store, a first determination means that determines whether or not it is necessary to deploy a mobile robot around the monitored person based on at least one of the monitored person's behavior or belongings, and a control means that deploys a mobile robot around the monitored person when it is determined that it is necessary to deploy a mobile robot.

[0007] In one aspect of the present disclosure, a person monitoring method involves a computer using images captured by a camera installed in a store to detect a monitored person who may be committing fraud, determining whether or not a mobile robot needs to be deployed around the monitored person based on at least one of the monitored person's behavior or belongings, and if it is determined that a mobile robot needs to be deployed, deploying the mobile robot around the monitored person.

[0008] In one aspect of the present disclosure, a recording medium stores a program that causes a computer to execute a process of using images captured by a camera installed in a store to detect a monitored individual who may be committing fraud, determining whether or not a mobile robot needs to be deployed on the monitored individual based on at least one of the monitored individual's behavior or belongings, and deploying a mobile robot on the monitored individual if it is determined that a mobile robot needs to be deployed.

[0009] According to the present disclosure, it is possible to provide a person monitoring device that uses a mobile robot to effectively deter fraudulent activities.

[0010] FIG. 1 is a block diagram of a person monitoring system according to the present disclosure. FIG. 2 is an external view showing an example of a mobile robot according to the present disclosure. FIG. 3 is a diagram showing a hardware configuration in which a person monitoring device and a mobile robot according to the present disclosure are realized by a computer device and its peripheral devices. FIG. 4 is a flowchart showing the operation of person monitoring according to the present disclosure. FIG. 5 is a block diagram of a person monitoring system according to the present disclosure. FIG. 6A is a flowchart showing the operation of person monitoring according to the present disclosure. FIG. 6B is a flowchart showing the operation of person monitoring according to the present disclosure.

[0011] Hereinafter, with reference to the drawings, embodiments of a person monitoring device, a mobile robot, a person monitoring system, a person monitoring method, and a non-transitory recording medium for recording a program according to the present disclosure will be described in detail. The disclosed technology is not limited to these embodiments.

[0012] 1 is a block diagram of a person monitoring system 10 according to the present disclosure. The person monitoring system 10 is a system for monitoring people who are behaving suspiciously in large buildings such as airports and shopping malls, which may be a precursor to shoplifting or other fraudulent activities. In addition to shoplifting, fraudulent activities also include voyeurism, molestation, vandalism, and the like.

[0013] The person monitoring system 10 includes a person monitoring device 100 that transmits monitoring commands related to operations to mobile robots under its control, and a mobile robot 200 that is placed near a person to be monitored. As shown in FIG. 1 , the person monitoring device 100 includes a detection unit 101, a first determination unit 102, and a control unit 103.

[0014] The mobile robot 200 is connected to the person monitoring device 100 via a network. For example, when the mobile robot 200 receives a control signal related to a monitoring command from the person monitoring device 100, it starts monitoring the person to be monitored based on the command. If there are multiple mobile robots 200, the person monitoring device 100 may transmit a control signal related to the monitoring command to a mobile robot 200 that is within a predetermined distance from the person to be monitored, and have the mobile robot 200 monitor the person to be monitored. The predetermined distance is, for example, a distance at which the behavior or belongings of the person to be monitored can be confirmed.

[0015] The mobile robot 200 has a plurality of operation modes, and the operation mode can be switched by receiving a control signal from the person monitoring device 100 or by changing the settings of the mobile robot 200. Examples of the operation modes include cleaning, guidance, tracking, charging, and inventory replenishment. The mobile robot 200 may have operation modes other than those described above. The mobile robot 200 can monitor a monitoring target person while operating in the current operation mode.

[0016] Here, each operation mode will be explained. In the cleaning mode, the mobile robot 200 cleans a predetermined location in the store. In the guidance mode, the mobile robot 200 leads a specific customer to a destination. In the tracking mode, the mobile robot 200 tracks the monitored person while maintaining a certain distance. In the charging mode, the mobile robot 200 charges the battery at a charging station. In the inventory replenishment mode, the mobile robot 200 replenishes inventory on the shelves. The mobile robot 200 is equipped with an arm capable of grasping products in order to replenish inventory.

[0017] 2 is an external view showing an example of a mobile robot 200 according to this embodiment. The shape of the mobile robot 200 is not limited to that shown in FIG. 2 and may be a bipedal robot. The mobile robot 200 includes a moving unit 201, an image capturing unit 202, a display unit 203, and a distance measuring unit 204.

[0018] The moving unit 201 includes tires and a motor for driving the tires. The photographing unit 202 includes a camera, a microphone, a speaker, and the like. The camera is provided in a position where it can photograph the behavior or belongings of the person being monitored, and photographs the person being monitored after being placed near the person being monitored. The speaker may output background music in the store or announcements to nearby customers.

[0019] The display unit 203 is provided on the top of the mobile robot 200 and is used to display information to deter the monitored person from committing fraudulent acts. The display unit 203 displays information based on a command from the person monitoring device 100. For example, the mobile robot 200 may display on the display unit 203 a message that shoplifting is a crime, thereby deterring the monitored person from committing fraudulent acts.

[0020] The distance measuring unit 204 can detect the distance to the monitored person. The distance measuring unit 204 allows the mobile robot 200 to move while maintaining a constant distance from the monitored person, for example, and therefore can track the monitored person without being noticed by the monitored person. The distance measuring unit 204 also allows the mobile robot 200 to prevent collisions with the surroundings.

[0021] 3 is a diagram showing an example of a hardware configuration in which each of the person monitoring device 100 and the mobile robot 200 according to the present disclosure is realized by a computer device 500 including a processor. As shown in FIG. 3, the person monitoring device 100 includes a CPU (Central Processing Unit) 501, memories such as a ROM (Read Only Memory) 502 and a RAM (Random Access Memory) 503, a storage device 505 such as a hard disk for storing a program 504, a communication interface 508 for network connection, and an input / output interface 509 for inputting and outputting data. In the present disclosure, each of the person monitoring device 100 and the mobile robot 200 is connected to each component via a bus 510.

[0022] The CPU 501 runs an operating system to control the entire person monitoring device 100 and mobile robot 200 according to the present disclosure. The CPU 501 also reads programs and data into memory from a recording medium 506 attached to, for example, a drive device 507. The CPU 501 also functions as each component of the person monitoring device 100 and the mobile robot 200 or as a part thereof, and executes processing or commands in the flowchart shown in FIG. 4 (described later) based on the program.

[0023] The recording medium 506 is, for example, an optical disk, a flexible disk, a magneto-optical disk, an external hard disk, or a semiconductor memory. The semiconductor memory or the like that is part of the recording medium is a non-volatile storage device that stores the program. The program may also be downloaded from an external computer (not shown) that is connected to a communication network.

[0024] As described above, the first embodiment shown in Fig. 1 is realized by the computer hardware shown in Fig. 3. However, the means for realizing each part of the person monitoring device 100 and the mobile robot 200 in Fig. 1 are not limited to the configurations described above. Furthermore, the person monitoring device 100 and the mobile robot 200 may each be realized by a single physically coupled device, or may be realized by a system consisting of two or more physically separate devices connected by wire or wirelessly.

[0025] Next, the person monitoring device 100 of the present disclosure will be described. The detection unit 101 is a means for detecting a person under surveillance who may be committing fraudulent acts using images captured by a camera installed in a store. The detection unit 101 detects, for example, a person who is previously included in the store's blacklist. More specifically, the detection unit 101 acquires image data showing the face of a passing person from, for example, video footage captured by a surveillance camera installed in the store, and extracts facial features. The image data may be multiple images or video data. The detection unit 101 then detects a person who may be committing fraudulent acts by comparing the acquired image data with facial features on a previously stored blacklist. The facial features on the blacklist are, for example, stored in the storage device 505 in advance. The detection unit 101 outputs image data of the detected person under surveillance to the first determination unit 102.

[0026] The first determination unit 102 is a means for determining whether it is necessary to deploy a mobile robot on a monitored person based on at least one of the customer's behavior and belongings. The first determination unit 102 analyzes image data of the monitored person and uses various publicly known techniques to determine whether the monitored person is engaging in suspicious behavior that may indicate fraud. Specifically, the first determination unit 102 determines that it is necessary to deploy the mobile robot 200 when the monitored person is loitering around the sales floor more than necessary, when the monitored person is walking back and forth around the sales floor more than necessary without exhibiting any particular shopping behavior, or when multiple people are visiting the store and each person appears to be acting as a lookout, a blindfolder, an actor, or the like. The first determination unit 102 also determines that it is necessary to deploy the mobile robot 200 when the monitored person is not carrying a dedicated shopping cart or the like, is carrying a large bag or shoulder bag, and is wandering around the store while being overly cautious of their surroundings. However, the conditions for the behavior or belongings of the monitoring target person that are determined by the first determination unit 102 are not limited to these.

[0027] The control unit 103 is a means for placing the mobile robot 200 at the person being monitored when it is determined that it is necessary to place the mobile robot 200. Specifically, the control unit 103 transmits a control signal to the mobile robot 200 indicating a movement command to move to the position of the person being monitored.

[0028] When there are multiple mobile robots 200 in the store, the control unit 103 selects the mobile robot 200 to be deployed near the person being monitored. In this case, the control unit 103 may select the mobile robot 200 to be deployed based on the distance of the mobile robot 200 to the person being monitored or the remaining battery power of the mobile robot 200. For example, the control unit 103 may select the mobile robot 200 closest to the person being monitored, or the mobile robot 200 with a remaining battery power equal to or greater than a predetermined amount. The location information of each mobile robot 200 may be determined by a camera such as a surveillance camera installed in the store, or may be determined by the radio wave intensity of wireless communication between the mobile robot 200 and a communication device installed in the store using short-range wireless communication. The remaining battery power is periodically obtained by, for example, a power sensor provided in each mobile robot 200 and stored in the storage device 505.

[0029] Furthermore, after placing the mobile robot 200 near the person being monitored, the control unit 103 may control the mobile robot 200 or equipment in the store to prevent the person being monitored from engaging in fraudulent behavior. For example, when the control unit 103 detects that the mobile robot 200 has come within a predetermined distance from the person being monitored using the distance measurement unit 204, the control unit 103 switches the mobile robot 200 or equipment in the store to a predetermined monitoring setting. The predetermined distance is, for example, a distance at which the behavior or belongings of the person being monitored can be confirmed from image data captured by the imaging unit 202.

[0030] More specifically, the control unit 103 controls the imaging unit 202 to capture an image of the monitored person after placing the mobile robot 200 at the monitored person. Furthermore, for example, after placing the mobile robot 200 at the monitored person, the control unit 103 may display a message on the display unit 203 of the mobile robot 200 or on a signage installed in the store that shoplifting is a crime. Furthermore, for example, after placing the mobile robot 200 at the monitored person, the control unit 103 may make an announcement about anti-shoplifting measures from the mobile robot 200 or a speaker in the store. The anti-shoplifting announcement is, for example, an announcement informing the customer that measures to prevent shoplifting are being taken in the store. Furthermore, after placing the mobile robot 200 at the monitored person, the control unit 103 may change the music played in the store to predetermined music played during monitoring. This allows the store staff to be notified that the mobile robot 200 has been placed at the monitored person.

[0031] 4 is a flowchart showing an outline of the operation of the person monitoring device 100 according to the present disclosure. Note that the processing according to this flowchart may be executed based on program control by the processor described above. In this embodiment, the flow up to step S104 is essential.

[0032] As shown in FIG. 4 , the detection unit 101 first detects a person under surveillance who may be engaging in fraudulent activities using images captured by a camera installed in the store (step S101). Next, the first determination unit 102 determines whether or not it is necessary to deploy a mobile robot 200 around the person under surveillance based on at least one of the person's behavior and belongings (step S102). Next, if it is determined that it is necessary to deploy the mobile robot 200 (S103; YES), the control unit 103 deploys the mobile robot 200 around the person under surveillance (step S104). After deploying the mobile robot 200, the control unit 103 controls the mobile robot 200 or equipment in the store to deter the person under surveillance from engaging in fraudulent activities (step S105). Note that if it is determined in step S103 that it is not necessary to deploy the mobile robot 200 (S103; NO), the flow ends. The person monitoring device 100 then ends the processing.

[0033] In the person monitoring device 100 according to the present disclosure, the first determination unit 102 determines whether or not it is necessary to deploy a mobile robot 200 around the person being monitored, based on at least one of the behavior and belongings of the person being monitored. Then, the control unit 103 deploys the mobile robot 200 when it is determined that it is necessary to deploy the mobile robot 200 around the person being monitored. Thus, the person monitoring device 100 can deploy the mobile robot 200 around the person being monitored who needs to be deployed. Therefore, it is possible to effectively deter fraudulent behavior using the mobile robot 200.

[0034] In the person monitoring device 100 according to the present disclosure, the control unit 103 places the mobile robot 200 near the person being monitored, and then controls the mobile robot 200 or the equipment in the store to deter the person being monitored from engaging in fraudulent behavior. In this case, fraudulent behavior by the person being monitored can be further deterred.

[0035] [Second Embodiment] Next, a second embodiment of the present disclosure will be described in detail with reference to the drawings. Below, the description of the second embodiment will be omitted to the extent that it does not make the description of the present embodiment unclear.

[0036] Fig. 5 is a block diagram of the person monitoring system 11 according to the present disclosure. Fig. 5 is a diagram for explaining a management location in the second embodiment. As with the computer device shown in Fig. 2, the functions of the person monitoring device 110 and the mobile robot 210 can be realized not only by hardware but also by a computer device and software based on program control.

[0037] 5, the person monitoring device 110 includes a detection unit 111, a first determination unit 112, a control unit 113, an acquisition unit 114, a second determination unit 115, and a notification unit 116. In this embodiment, the configurations of the detection unit 111, the first determination unit 112, and the control unit 113 are similar to the corresponding components in the first embodiment.

[0038] The acquisition unit 114 is a means for acquiring status information of the monitored person using a sensor provided on the mobile robot 210 after placing the mobile robot 210 near the monitored person. The acquisition unit 114 acquires, as status information, for example, biometric information related to the monitored person's psychological state. The thrill of committing a fraudulent act such as shoplifting can increase a person's level of tension, which can change their psychological state from normal. Therefore, the acquisition unit 114 acquires status information that affects the monitored person's psychological state immediately before committing a fraudulent act. The acquisition unit 114 may acquire biometric information measured by a sensor such as an image sensor provided on the mobile robot 210. The acquisition unit 114 acquires, as biometric information, information such as pulse rate, blood pressure, body temperature, and facial expression. For example, the acquisition unit 114 acquires pulse rate or blood pressure by analyzing the blood flow state from an image of the monitored person's face captured by a camera. The acquisition unit 114 also acquires body temperature using a thermosensor and facial expression of the monitored person captured by the image capture unit 212. The acquisition unit 114 outputs the state information to the second determination unit 115 .

[0039] The second determination unit 115 is a means for determining whether or not it is necessary to track the monitoring target based on the status information. For example, the second determination unit 115 determines that it is necessary to track the monitoring target if the value of the biometric information affecting the psychological state of the monitoring target is equal to or greater than a threshold. On the other hand, the second determination unit 115 determines that it is not necessary to track the monitoring target if the value of the biometric information affecting the psychological state of the monitoring target is not equal to or greater than the threshold. The second determination unit 115 may use multiple types of biometric information for its determination. Furthermore, the second determination unit 115 may determine whether or not to track the monitoring target using a trained model in which the biometric information is an explanatory variable and the determination result is an objective variable. The second determination unit 115 may output a determination result indicating whether or not tracking is necessary, or may output some numerical value such as the degree of necessity.

[0040] Based on the determination result, the control unit 113 controls the mobile robot 210. When it is determined that the monitored person needs to be tracked, the control unit 113 switches the current operation mode of the mobile robot 210 to track the monitored person. For example, when the current operation mode of the mobile robot 210 is the cleaning mode, the control unit 113 controls the mobile robot 210 to switch to the tracking mode and start tracking the monitored person.

[0041] When it is not possible to switch the current operation mode, the control unit 113 controls another mobile robot to track the monitored person. When switching is not possible, for example, when the mobile robot 210 is serving a customer or cannot stop other work. For example, when the current operation mode of the mobile robot 210 is inventory replenishment mode, the control unit 113 controls another mobile robot 210 that is within a predetermined distance from the monitored person to switch the operation mode to tracking mode and track the monitored person.

[0042] On the other hand, when it is determined that there is no need to track the monitored person, the control unit 113 continues to monitor the monitored person while maintaining the current operation mode of the mobile robot 210. For example, when the current operation mode of the mobile robot 210 is the cleaning mode, the control unit 113 controls the mobile robot 210 to monitor the monitored person by having the image capturing unit 212 capture an image of the monitored person while cleaning.

[0043] The notification unit 116 is a means for notifying a store clerk when the mobile robot 210 is unable to track the monitored person. For example, when the monitored person is moving faster than the mobile robot 210 and cannot be tracked, the notification unit 116 sends a message requesting tracking to a display device in the backroom or a store terminal device carried by the store clerk. For example, the notification unit 116 notifies a store clerk when the distance to the monitored person detected by the distance measuring unit 214 of the mobile robot 210 exceeds a predetermined distance.

[0044] 6 is a flowchart showing an outline of the operation of the person monitoring device 110 according to the present disclosure. The processing according to this flowchart may be executed based on program control by the processor described above. The flow of steps S201 to S205 in FIG. 6 is the same as the flow of steps S101 to S105 in FIG. 4.

[0045] As shown in FIG. 6A , the detection unit 111 first detects a person under surveillance who may be engaging in fraudulent behavior using images captured by a camera installed in the store (step S201). Next, the first determination unit 112 determines whether or not it is necessary to deploy a mobile robot 210 around the person under surveillance based on at least one of the person's behavior and belongings (step S202). Next, if it is determined that it is necessary to deploy the mobile robot 210 (S203; YES), the control unit 113 deploys the mobile robot 210 around the person under surveillance (step S204). Next, the control unit 113 controls the mobile robot 210 or in-store equipment to deter the person under surveillance from engaging in fraudulent behavior (step S205). Note that if it is determined in step S203 that it is not necessary to deploy a mobile robot (S203; NO), the flow ends.

[0046] Next, the acquisition unit 114 positions the mobile robot 210 near the monitored person and acquires status information of the monitored person using a sensor provided on the mobile robot (step S206). Next, the second determination unit 115 determines whether or not it is necessary to track the monitored person based on the status information (step S207). Next, if it is determined that it is necessary to track the monitored person (S208; YES) and if the current operating mode can be switched (S209; YES), the control unit 113 switches the current operating mode of the mobile robot 210 to track the monitored person (step S210). Next, if the mobile robot 210 cannot track the monitored person (S211; YES), the notification unit 116 notifies a store clerk (step S212). On the other hand, if the mobile robot 210 can track the monitored person (S211; NO), the flow ends.

[0047] On the other hand, if the control unit 113 determines that it is necessary to track the monitored person (S208; YES) and the current operating mode cannot be switched (S209; NO), it controls another mobile robot 210 to track the monitored person (step S213).

[0048] If it is determined in step S208 that there is no need to track the person to be monitored (S208; NO), the control unit 113 controls the mobile robot 210 to monitor the person to be monitored while operating in the current operating mode (step S214).

[0049] In the person monitoring device 110 of the present disclosure, the second determination unit 115 determines whether or not it is necessary to track the person being monitored, based on the status information of the person being monitored. If it is determined that it is necessary to track the person being monitored, the control unit 113 switches the current operation mode of the mobile robot 210 to track the person being monitored. This allows the person being monitored, who is more likely to engage in fraudulent behavior, to be monitored more carefully.

[0050] In addition, in the person monitoring device 110 of the present disclosure, the notification unit 116 notifies a store clerk when the mobile robot 210 is unable to track the person being monitored, thereby preventing the person being monitored from being lost.

[0051] Furthermore, in the person monitoring device 110 of the present disclosure, when it is not possible to switch the current operating mode, the control unit 113 controls another mobile robot 210 to track the person being monitored. This allows the mobile robot 210 to track the person being monitored without stopping the task assigned to it.

[0052] Although the present disclosure has been described above with reference to various embodiments, the present disclosure is not limited to the above embodiments. The configuration and details of each of the present disclosures may include embodiments to which various modifications that would be apparent to those skilled in the art are applied within the scope of the present disclosure. The present disclosure may also include embodiments in which the details described herein are appropriately combined or substituted as necessary. For example, details described using a particular embodiment may also be applied to other embodiments to the extent that no contradiction occurs. For example, although multiple operations are described in sequence in the form of a flowchart, the order of the descriptions does not limit the order in which the multiple operations are performed. Therefore, when implementing each embodiment, the order of the multiple operations may be changed as long as it does not interfere with the content.

[0053] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0054] (Supplementary Note 1) A person monitoring device comprising: a detection means for detecting a person to be monitored who may be committing fraudulent acts using images taken by a camera installed in a store; a first determination means for determining whether or not it is necessary to deploy a mobile robot around the person to be monitored based on at least one of the behavior or belongings of the person to be monitored; and a control means for deploying the mobile robot around the person to be monitored when it is determined that it is necessary to deploy the mobile robot.

[0055] (Supplementary Note 2) The person monitoring device according to Supplementary Note 1, wherein the detection means detects a person who is included in a blacklist of the store in advance.

[0056] (Supplementary Note 3) In the person monitoring device described in Supplementary Note 1 or Supplementary Note 2, when there are multiple mobile robots, the control means selects a mobile robot to be placed at the position of the person to be monitored and places the selected mobile robot.

[0057] (Appendix 4) The person monitoring device described in any one of Appendices 1 to 3, wherein the control means controls the mobile robot or equipment in the store to deter fraudulent behavior by the person being monitored after placing the mobile robot near the person being monitored.

[0058] (Supplementary Note 5) The person monitoring device according to Supplementary Note 4, wherein the control means, after placing the mobile robot near the person to be monitored, displays on a display provided on the mobile robot or in the store that shoplifting is a crime.

[0059] (Appendix 6) The person monitoring device described in Appendix 4 or Appendix 5, wherein the control means changes the music played in the store to music to be played during predetermined monitoring after placing the mobile robot at the person to be monitored.

[0060] (Supplementary Note 7) A person monitoring device as described in any of Supplementary Notes 1 to 6, further comprising: an acquisition means for acquiring status information of the monitored person using a sensor provided on the mobile robot after placing the mobile robot near the monitored person; and a second determination means for determining whether or not it is necessary to track the monitored person based on the status information, wherein the control means controls the mobile robot based on the determination result.

[0061] (Appendix 8) The person monitoring device described in Appendix 7, wherein the mobile robot has a plurality of operating modes, and when the control means determines that it is necessary to track the monitored person, it switches the current operating mode of the mobile robot to track the monitored person.

[0062] (Supplementary Note 9) The person monitoring device according to Supplementary Note 7 or Supplementary Note 8, further comprising a notification means for notifying a store clerk when the mobile robot is unable to track the person to be monitored.

[0063] (Supplementary Note 10) The person monitoring device according to Supplementary Note 8, wherein the control means controls another mobile robot to track the customer if it is not possible to switch the current operating mode.

[0064] (Appendix 11) The person monitoring device described in Appendix 7, wherein the mobile robot has a plurality of operating modes, and when it is determined that there is no need to track the monitored person, the control means controls the mobile robot to monitor the monitored person while operating in the current operating mode.

[0065] (Supplementary Note 12) A people monitoring system comprising a person monitoring device according to any one of Supplementary Note 1 to Supplementary Note 11 and a mobile robot that operates based on commands from the person monitoring device, wherein the mobile robot comprises: a moving means that moves the mobile robot itself to the location of the person being monitored based on commands from the person monitoring device; a photographing means that photographs at least one of the behavior or belongings of the person being monitored; and a display means that displays information to deter the person being monitored from engaging in fraudulent behavior.

[0066] (Supplementary Note 13) A method for monitoring people, in which a computer uses images taken by a camera installed in a store to detect a person to be monitored who may be committing fraudulent acts, determines whether or not it is necessary to deploy a mobile robot around the person to be monitored based on at least one of the behavior or belongings of the person to be monitored, and deploys a mobile robot around the person to be monitored if it is determined that it is necessary to deploy the mobile robot.

[0067] (Supplementary Note 14) The person monitoring method according to Supplementary Note 13, wherein after the mobile robot is placed near the person to be monitored, the mobile robot or equipment in the store is controlled to prevent the person to be monitored from engaging in any fraudulent activity.

[0068] (Supplementary Note 15) A person monitoring method as described in Supplementary Note 13 or Supplementary Note 14, wherein after the mobile robot is placed near the person to be monitored, status information of the person to be monitored is acquired using a sensor provided on the mobile robot, based on the status information, it is determined whether or not it is necessary to track the person to be monitored, and based on the determination result, the mobile robot is controlled.

[0069] (Appendix 16) The person monitoring method described in Appendix 15, wherein the mobile robot has a plurality of operating modes, and when it is determined that the monitored person needs to be tracked, the current operating mode of the mobile robot is switched to track the monitored person.

[0070] (Supplementary Note 17) The person monitoring method according to Supplementary Note 16, wherein if the current operating mode of the mobile robot cannot be switched, another mobile robot is controlled to track the customer.

[0071] (Appendix 18) The person monitoring method described in Appendix 15, wherein the mobile robot has a plurality of operating modes, and when it is determined that there is no need to track the monitored person, the mobile robot is controlled to monitor the monitored person while operating in the current operating mode.

[0072] (Supplementary Note 19) The person monitoring method according to any one of Supplementary Note 16 and Supplementary Note 17, wherein if the mobile robot is unable to track the person being monitored, a store clerk is notified.

[0073] (Appendix 20) A recording medium storing a program that causes a computer to execute a process of detecting a person to be monitored who may be committing fraudulent acts using images taken by a camera installed in a store, determining whether or not it is necessary to deploy a mobile robot around the person to be monitored based on at least one of the behavior or belongings of the person to be monitored, and deploying a mobile robot around the person to be monitored if it is determined that it is necessary to deploy the mobile robot.

[0074] Some or all of the configurations described in Supplements 2 to 11, which are dependent on Supplement 1 described above, are also dependent on Supplement 13 and Supplement 20 in the same dependent relationship as Supplements 2 to 11. Not limited to Supplements 1, 11, and 20, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording devices for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0075] 10, 11 Person monitoring system 100, 110 Person monitoring device 101, 111 Detection unit 102, 112 First determination unit 103, 113 Control unit 114 Acquisition unit 115 Second determination unit 116 Notification unit 200, 210 Mobile robot 201, 211 Movement unit 202, 212 Photography unit 203, 213 Display unit 204, 214 Distance measurement unit 500 Computer device 501 CPU 502 ROM 503 RAM 504 Program 505 Storage device 506 Recording medium 507 Drive device 508 Communication interface 509 Input / output interface 510 Bus

Claims

1. A person monitoring device comprising: a detection means for detecting a person to be monitored who may be committing fraudulent acts using images taken by a camera installed in a store; a first determination means for determining whether or not it is necessary to deploy a mobile robot around the person to be monitored based on at least one of the behavior or belongings of the person to be monitored; and a control means for deploying the mobile robot around the person to be monitored when it is determined that it is necessary to deploy the mobile robot.

2. The person monitoring device according to claim 1, wherein said detection means detects people who are included in a blacklist of people in said store in advance.

3. A person monitoring device as described in claim 1 or claim 2, wherein, when there are multiple mobile robots, the control means selects a mobile robot to be placed at the position of the person being monitored and places the selected mobile robot.

4. A person monitoring device as described in any one of claims 1 to 3, wherein the control means controls the mobile robot or equipment within the store to deter fraudulent behavior by the person being monitored after placing the mobile robot near the person being monitored.

5. The person monitoring device according to claim 4, wherein the control means, after placing the mobile robot near the person to be monitored, displays a message on a display provided on the mobile robot or in the store that shoplifting is a crime.

6. A person monitoring device as described in claim 4 or claim 5, wherein the control means changes the music played in the store to a predetermined music to be played during monitoring after placing the mobile robot at the person to be monitored.

7. A person monitoring device as described in any one of claims 1 to 6, further comprising: an acquisition means for acquiring status information of the monitored person using a sensor provided on the mobile robot after placing the mobile robot near the monitored person; and a second determination means for determining whether or not it is necessary to track the monitored person based on the status information, wherein the control means controls the mobile robot based on the determination result.

8. A person monitoring device as described in claim 7, wherein the mobile robot has a plurality of operating modes, and when the control means determines that it is necessary to track the person being monitored, it switches the current operating mode of the mobile robot to track the person being monitored.

9. The person monitoring device according to claim 7 or 8, further comprising a notification means for notifying a store clerk if the mobile robot is unable to track the person being monitored.

10. The person monitoring device according to claim 8, wherein the control means controls another mobile robot to track the customer if it is not possible to switch the current operating mode.

11. A person monitoring device as described in claim 7, wherein the mobile robot has a plurality of operating modes, and when it is determined that there is no need to track the person being monitored, the control means controls the mobile robot to monitor the person being monitored while operating in the current operating mode.

12. A people monitoring system comprising a people monitoring device according to any one of claims 1 to 11 and a mobile robot that operates based on commands from the people monitoring device, wherein the mobile robot comprises: a moving means that moves the mobile robot itself to the location of the person being monitored based on commands from the people monitoring device; a photographing means that photographs at least one of the actions or belongings of the person being monitored; and a display means that displays information to deter the person being monitored from engaging in fraudulent behavior.

13. A method for monitoring people, in which a computer uses images taken by a camera installed in a store to detect a person to be monitored who may be committing fraud, determines whether or not it is necessary to deploy a mobile robot around the person to be monitored based on at least one of the person's behavior or belongings, and deploys a mobile robot around the person to be monitored if it is determined that it is necessary to deploy the mobile robot.

14. A person monitoring method as described in claim 13, wherein after placing the mobile robot near the person being monitored, the mobile robot or equipment in the store is controlled to prevent the person being monitored from engaging in any fraudulent activity.

15. A person monitoring method as described in claim 13 or claim 14, wherein after placing the mobile robot near the person to be monitored, status information of the person to be monitored is acquired using a sensor provided on the mobile robot, based on the status information, it is determined whether or not it is necessary to track the person to be monitored, and based on the determination result, the mobile robot is controlled.

16. A person monitoring method as described in claim 15, wherein the mobile robot has a plurality of operating modes, and when it is determined that the monitored person needs to be tracked, the current operating mode of the mobile robot is switched to track the monitored person.

17. The person monitoring method according to claim 16, wherein if the current operating mode of the mobile robot cannot be switched, another mobile robot is controlled to track the customer.

18. A person monitoring method as described in claim 15, wherein the mobile robot has a plurality of operating modes, and when it is determined that there is no need to track the monitored person, the mobile robot is controlled to monitor the monitored person while operating in the current operating mode.

19. A method of monitoring people as claimed in claim 16 or claim 17, wherein if the mobile robot is unable to track the monitored person, a store attendant is notified.

20. A recording medium storing a program that causes a computer to execute a process of detecting a person to be monitored who may be committing fraudulent acts using images taken by a camera installed in a store, determining whether or not it is necessary to deploy a mobile robot around the person to be monitored based on at least one of the person's behavior or belongings, and deploying a mobile robot around the person to be monitored if it is determined that it is necessary to deploy the mobile robot.

Images