Robot

Abstract

To provide a robot capable of accurately detecting suspicious persons or objects.SOLUTION: A robot according to an embodiment comprises a robot body, a detection unit, and a suspicious person / object sensing unit. The detection unit detects surrounding situation representing the situation around the robot body. The suspicious person / object sensing unit senses a suspicious person or object present in the surroundings of the robot on the basis of the surrounding situation detected by the detection unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The disclosed embodiments relate to a robot. [Background technology]

[0002] Conventionally, a system has been known in which, when a vehicle is threatened, a drone is launched from the vehicle and an image of the vehicle and its surroundings is captured by a camera mounted on the drone (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-93618 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology leaves room for improvement in terms of accurately detecting suspicious objects in facilities requiring security, such as airports and train stations.

[0005] The present invention has been made in view of the above, and aims to detect suspicious objects with high accuracy. [Means for solving the problem]

[0006] According to one aspect of the embodiment, a robot includes a robot body, a detection unit, and a suspicious object detection unit. The detection unit detects a surrounding situation indicating a situation around the robot body. The suspicious object detection unit detects a suspicious object present around the robot body based on the surrounding situation detected by the detection unit. [Effects of the Invention]

[0007] According to one aspect of the embodiment, it is possible to detect a suspicious object with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an outline of a robot system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an outline of a robot according to an embodiment. [Figure 3] FIG. 3 is a functional block diagram showing an outline of a control device for a robot according to an embodiment. [Figure 4] FIG. 4 is a functional block diagram illustrating an overview of a management device according to an embodiment. [Figure 5] FIG. 5 is a flowchart illustrating a suspicious object detection process executed by the robot according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating a control process for a short-range drone according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating a control process for a long-range drone according to the embodiment. [Figure 8A] FIG. 8A is a diagram showing an example of the flight of a long-range drone before a tracking mark is emitted. [Figure 8B] FIG. 8B is a diagram showing an example of the flight of a long-range drone after a tracking mark has been attached to a suspicious object. [Figure 9] FIG. 9 is a diagram schematically illustrating an example of a computer hardware configuration that functions as a mobile object control device. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described below through embodiments, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] A robot system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an outline of the robot system 1 according to an embodiment.

[0011] The robot system 1 includes a robot 2, a long-range drone 3, and a management device 4. The robot 2, the long-range drone 3, and the management device 4 are connected via a network N. For example, a plurality of robots 2 and a plurality of long-range drones 3 may be provided.

[0012] The network N is, for example, a mobile communication network such as LTE (Long Term Evolution), 5G, etc. The robot 2 and the long-range drone 3 may be connected via the network N.

[0013] Next, the robot 2 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an outline of the robot 2 according to the embodiment.

[0014] The robot 2 is installed, for example, in a facility requiring security. Such a facility may be an airport, train station, bus terminal, commercial facility, office building, or other facility used by an unspecified number of people. Note that although specific facilities have been described above, these are merely examples and are not limiting. Furthermore, the robot 2 does not necessarily need to be installed in a facility requiring security, and may be installed in other facilities or locations.

[0015] The robot 2 includes a robot body 10, a short-range drone 11, a detection unit 12, a control device 13, and an alarm unit 14. The robot 2 is capable of autonomous travel, and when a suspicious object is detected, it tracks the suspicious object. The suspicious object may be, for example, a person behaving suspiciously or the perpetrator of a crime. Furthermore, the suspicious object is not limited to a person, but may also be a vehicle carrying a person behaving suspiciously. In this way, the robot 2 can also be said to be a tracking robot because it tracks the suspicious object. Furthermore, a robot system 1 including such a robot 2 can also be said to be a tracking system.

[0016] Furthermore, the suspicious object detected by the robot 2 may be an object. A suspicious object is an object that is prohibited from being brought into a facility where the robot 2 is installed. For example, suspicious objects include metal objects (knives, scissors, and other cutting tools), lethal weapons (guns, knives, and the like), and illegal drugs (narcotics, and the like), but these are merely examples and are not limiting.

[0017] The robot body 10 is, for example, a humanoid robot. The robot body 10 comprises a torso 10a, legs 10b, arms 10c, and a head 10d. The legs 10b, arms 10c, and head 10d are movably attached to the torso 10a. Note that the robot body 10 is not limited to a humanoid robot. The robot body 10 is provided with a drive mechanism such as a motor that moves each part of the robot, such as the legs 10b, arms 10c, and head 10d.

[0018] A base unit 16 is provided on the back of the body unit 10a, from which the short-range drone 11 can take off and land. The base unit 16 is equipped with a charging device that charges the battery of the short-range drone 11. The base unit 16 may also be equipped with a replacement battery for the short-range drone 11.

[0019] The short-range drone 11 is an example of a moving object. The short-range drone 11 is a multicopter equipped with multiple (for example, four) rotary propellers, and performs unmanned autonomous flight. The short-range drone 11 is also powered by a battery.

[0020] The short-range drone 11 includes a camera 20 and a tracking mark 21. The tracking mark 21 is emitted from the short-range drone 11. The short-range drone 11 includes a launcher that launches the tracking mark 21.

[0021] For example, when the tracking mark 21 is projected at a suspicious object, it adheres to the suspicious object (person or vehicle). For example, the tracking mark 21 is provided with a magnet. For example, if the suspicious object is a vehicle, the tracking mark 21 adheres to the suspicious object by magnetic force. The tracking mark 21 may be adhesive.

[0022] The tracking mark 21 has, for example, a positioning device. The positioning device is, for example, a GNSS (Global Navigation Satellite System), and can receive radio waves from navigation satellites orbiting in the sky to determine position and time. The positioning device also has a communication module that transmits information about its own detected position. The position information of the tracking mark 21 detected by the positioning device is transmitted to the management device 4 via the network N. The position information of the tracking mark 21 detected by the positioning device is transmitted to the control device 13 via the network N. When the tracking mark 21 is attached to a suspicious object, the position of the suspicious object is detected based on the position information transmitted from the positioning device.

[0023] The tracking mark 21 may also be a ball containing fluorescent paint. The ball containing fluorescent paint explodes when it hits a suspicious object, and the fluorescent paint adheres to the suspicious object. The short-range drone 11 may be equipped with multiple types of tracking marks 21.

[0024] The short-range drone 11 includes a communication module for performing wireless communication via the network N. The short-range drone 11 also includes various sensors such as an acceleration sensor, a gyro sensor, and an optical sensor.

[0025] The short-range drone 11 also has a positioning device 22 for measuring its own position. The positioning device 22 is, for example, a GNSS. The position information of the short-range drone 11 detected by the positioning device 22 may be transmitted to the management device 4 via the network N. The position information of the short-range drone 11 detected by the positioning device 22 may be transmitted to the control device 13 via the network N. The short-range drone 11 also has a computer (for example, a microcomputer) that executes a flight control function, an attitude control function for controlling the attitude, and the like.

[0026] The short-range drone 11 acquires information about the first flight path from the control device 13 via the communication module. The information about the first flight path includes position information (e.g., latitude, longitude, and altitude) of the first flight path. The first flight path is a flight path for tracking a suspicious target.

[0027] The short-range drone 11 transmits images captured by the camera 20 to the management device 4 via the communication module. The short-range drone 11 transmits images captured by the camera 20 to the control device 13 via the communication module. The short-range drone 11 acquires information regarding the emission signal of the tracking mark 21 from the control device 13 via the communication module.

[0028] When a suspicious object (person or vehicle) is detected, the short-range drone 11 launches from the base unit 16 of the robot main body 10. The short-range drone 11 acquires information about a first flight path and flies along the first flight path to track the suspicious object. The short-range drone 11 performs image processing based on images captured by the camera 20 and flies along the first flight path while avoiding obstacles. When a signal to launch a tracking mark 21 is acquired, the short-range drone 11 launches the tracking mark 21. The short-range drone 11 launches the tracking mark 21 toward the suspicious object.

[0029] The detection unit 12 is provided, for example, in the head 10d of the robot body 10. The detection unit 12 may also be provided in the trunk 10a or the like of the robot body 10. The detection unit 12 detects the surrounding situation that indicates the situation around the robot body 10.

[0030] The detection unit 12 includes a camera. The camera may include, for example, a high-sensitivity camera capable of 360-degree sensing, a thermal camera, etc. The detection unit 12 captures an image of the surroundings of the robot main body 10 and detects the captured image as the surrounding situation.

[0031] The detection unit 12 also includes a metal detector. Examples of metal detectors that can be used include, but are not limited to, electromagnetic induction, magnetic induction, and X-ray detectors. The detection unit 12 detects metallic objects present around the robot main body 10 as part of the surrounding conditions.

[0032] The detection unit 12 also includes a shape detection sensor capable of detecting the shape of an object. The shape detection sensor may be one that uses infrared rays to detect the shape of an object, but is not limited to this and may be another type of sensor that uses X-rays, for example. The detection unit 12 detects the shapes of objects present around the robot main body 10 as the surrounding conditions.

[0033] The detection unit 12 also includes an odor detection sensor (odor detector) that can detect the odor (smell) of an object. The odor detection sensor may be, for example, a semiconductor type or a quartz oscillator type, but is not limited to these. The detection unit 12 detects the odor of an object present around the robot main body 10 as the surrounding conditions.

[0034] Furthermore, the detection unit 12 detects, as the surrounding situation, identification information that can identify people present around the robot main body 10. The identification information includes, for example, facial information of the person, personal information stored on an identification card (ID card, passport, driver's license, etc.) held by the person, etc. When the identification information is facial information of the person, the detection unit 12 includes a camera and detects the facial information of the person captured by the camera as the identification information. When the identification information is personal information on the identification card, the detection unit 12 includes a reader and detects the personal information on the identification card read by the reader as the identification information.

[0035] The detection unit 12 outputs the various surrounding conditions detected as described above to the control device 13. The detection unit 12 may transmit the various surrounding conditions that it has detected to the management device 4 via the network N.

[0036] The detection unit 12 may be configured to include all or some of the above-mentioned cameras, metal detectors, shape detection sensors, odor detection sensors, and readers. The detection unit 12 may also include sensors for LiDAR (light detection and ranging), radar, vision recognition, minute sounds, ultrasonic waves, vibrations, ultraviolet rays, electromagnetic waves, and the like. A plurality of detection units 12 may be provided. The detection unit 12 may be multiple types of sensors.

[0037] The detection unit 12 also includes a positioning device. The positioning device is, for example, a GNSS. The positioning device detects the position of the robot main body 10. Information about the detected position of the robot main body 10 is transmitted to the management device 4 via the network N.

[0038] When a suspicious object is detected, the notification unit 14 notifies the detection of the suspicious object. The notification unit 14 includes, for example, a speaker, and outputs audio information indicating that a suspicious object has been detected to notify those in the vicinity. The audio information may be an alarm sound such as a buzzer, or may be an audio message indicating that a suspicious object has been detected.

[0039] As shown in Fig. 3, the control device 13 includes a communication unit 30, a storage unit 31, and a control unit 32. Fig. 3 is a functional block diagram showing an outline of the control device 13 of the robot 2 according to the embodiment.

[0040] The communication unit 30 is wirelessly connected to the network N. The communication unit 30 transmits and receives information to and from the management device 4 via the network N. The communication unit 30 transmits various pieces of information detected by the detection unit 12 to the management device 4. The communication unit 30 receives images captured by the camera 20 of the short-range drone 11. The communication unit 30 receives position information of the tracking mark 21 from the positioning device of the tracking mark 21.

[0041] The storage unit 31 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), an optical disk, etc. The storage unit 31 stores suspicious target information 33, non-suspicious target information 34, various programs, various data, etc.

[0042] The suspicious target information 33 is information relating to a suspicious target. The suspicious target information 33 includes suspicious target shape information, suspicious target odor information, and suspicious target person information. The suspicious target shape information is information indicating the shape of a suspicious target, for example, information indicating the shape of an object (a weapon such as a gun or knife) that is prohibited from being brought into a facility where the robot 2 is installed. The suspicious target odor information is information indicating the odor of a suspicious target, for example, information indicating the odor of an object (an illegal substance such as a narcotic) that is prohibited from being brought into a facility where the robot 2 is installed. The suspicious target person information is information indicating a person who is a suspicious target, for example, information indicating a person who is prohibited from entering a facility where the robot 2 is installed. The suspicious target person information includes, for example, facial information of the perpetrator of a crime (a wanted criminal), facial information of suspicious people who have behaved suspiciously in a facility in the past, etc.

[0043] Non-suspicious target information 34 is information about non-suspicious targets, in other words, information about things that do not fall under the category of suspicious targets. Non-suspicious target information 34 includes non-suspicious target person information. Non-suspicious target person information is information indicating people who are not suspicious targets, for example, information indicating people who are permitted to enter the facility where robot 2 is installed. Non-suspicious target person information includes, for example, facial information and personal information of facility employees, and if the facility is an airport, facial information and personal information of airplane passengers.

[0044] The suspicious target information 33 and the non-suspicious target information 34 are predetermined information. The suspicious target information 33 and the non-suspicious target information 34 are stored in the storage unit 31 in advance.

[0045] The control unit 32 is a controller and includes, for example, a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM, input / output ports, etc., and various other circuits. The control unit 32 may also be configured with hardware such as an integrated circuit, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 32 includes a suspicious object detection unit 35, a drone control unit 36, and a robot control unit 37.

[0046] The suspicious object detection unit 35 detects a suspicious object. The suspicious object detection unit 35 detects a suspicious object based on various information detected by the detection unit 12. For example, the suspicious object detection unit 35 performs predetermined image processing on an image captured by a high-sensitivity camera provided on the robot 2 to detect a suspicious object. For example, the suspicious object detection unit 35 may detect a suspicious object based on the detection result of an infrared sensor. The suspicious object detection unit 35 may detect a suspicious object using a suspicious object detection model in AI (Artificial Intelligence).

[0047] Specifically, the suspicious object detection unit 35 detects suspicious objects (objects and people) present around the robot main body 10 based on the surrounding conditions detected by the detection unit 12. For example, when the detection unit 12 including a metal detector detects a metal object (a bladed object such as a knife or scissors) present around the robot main body 10, the suspicious object detection unit 35 detects the object as a suspicious object.

[0048] Furthermore, the suspicious object detection unit 35 compares the shape of an object present around the robot main body 10 detected by the detection unit 12 including a shape detection sensor with the suspicious object shape information of the suspicious object information 33 stored in the memory unit 31. If the shape of the object detected by the detection unit 12 is a suspicious object shape (a weapon such as a gun or knife), the suspicious object detection unit 35 detects the object as a suspicious object.

[0049] Furthermore, the suspicious object detection unit 35 compares the odor of an object present around the robot main body 10 detected by the detection unit 12 including an odor detection sensor with the suspicious object odor information of the suspicious object information 33 stored in the memory unit 31. If the odor of the object detected by the detection unit 12 is a suspicious object odor (the odor of an illegal drug such as a narcotic), the suspicious object detection unit 35 detects the object as a suspicious object.

[0050] Furthermore, the suspicious target detection unit 35 compares the identification information (face information) of people present around the robot main body 10 detected by the detection unit 12 including a camera with the suspicious target person information of the suspicious target information 33 stored in the memory unit 31. Based on the identification information detected by the detection unit 12, if the person in the identification information is a suspicious person (a wanted criminal or a suspicious person), the suspicious target detection unit 35 detects such person as a suspicious target.

[0051] The suspicious object detection unit 35 may also request a person present around the robot main body 10 to present their identification card and authenticate whether the person is a suspicious or non-suspicious person. For example, the suspicious object detection unit 35 requests that the person present around the robot main body 10 have their identification card read by the detection unit 12 including a reader. The suspicious object detection unit 35 compares the identification information (personal information) of the ID card detected by the detection unit 12 with the suspicious object information of the person in the non-suspicious object information 34 stored in the memory unit 31. If the person in the identification information detected by the detection unit 12 is a non-suspicious person (a facility employee or an airplane passenger), the suspicious object detection unit 35 detects the person as a non-suspicious person based on the identification information detected by the detection unit 12. If facial information is registered on the identification card, the suspicious object detection unit 35 may compare the facial information of the person presenting the ID card with the facial information of the person detected by the detection unit 12 including a camera, and if they match, detect the person as a non-suspicious person.

[0052] On the other hand, the suspicious object detection unit 35 may detect a person as a suspicious object if the person present around the robot main body 10 refuses to present their identification card, or if the person identified by the identification information on the identification card is not a non-suspicious object.

[0053] When a suspicious object is detected as described above, the suspicious object detection unit 35 activates the notification unit 14 to notify the surrounding area that a suspicious object has been detected. The suspicious object detection unit 35 also notifies the management device 4 via the network N that a suspicious object has been detected.

[0054] In this embodiment, after a suspicious object is detected, if the suspicious person flees, or if a person carrying a suspicious object flees, the suspicious object is tracked.

[0055] Specifically, the drone control unit 36 ​​controls the short-range drone 11 and the long-range drone 3. The drone control unit 36 ​​sets a first flight path for the short-range drone 11. The drone control unit 36 ​​sets a first flight path for tracking a detected suspicious object. The drone control unit 36 ​​sets the first flight path based on the current position information of the robot 2 and the position information at which the suspicious object was detected. For example, the drone control unit 36 ​​sets the first flight path based on the current position information of the robot 2, the direction in which the suspicious object was detected relative to the robot 2, and the distance from the robot 2 to the suspicious object.

[0056] The drone control unit 36 ​​sets a second flight path for the long-range drone 3. The drone control unit 36 ​​sets a second flight path for tracking a detected suspicious object. The drone control unit 36 ​​sets the second flight path based on the position information of the station where the long-range drone 3 is waiting and the position information of the tracking mark 21. The drone control unit 36 ​​may set the second flight path based on the position information of the long-range drone 3 and the position information of the tracking mark 21.

[0057] The drone control unit 36 ​​may set the first flight path and the second flight path based on the predicted escape route of the suspicious subject. The predicted escape route is generated, for example, by an escape route prediction model. The escape route prediction model predicts the escape route of the suspicious subject based on the position information of the robot 2 when the suspicious subject is detected, the position information of the tracking mark 21, map information, traffic information, etc. The escape route prediction model includes the position information of the tracking mark 21, for example, historical information about the position of the tracking mark 21.

[0058] For example, the escape route prediction model can use MapGPT. For example, the escape route prediction model is a text generation model (so-called AI chat engine), and can be interpreted as an algorithm and calculation for automatic text-based dialogue processing. The text generation model is described, for example, in JP 2018-081444 A and chatGPT (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and therefore a detailed description thereof will be omitted. Such a sentence generation model is configured using a large language model (LLM).

[0059] For example, the drone control unit 36 ​​generates a sentence such as "A suspicious person is fleeing from east to west at the intersection of ABC Town, DD. What escape route do you think they might have taken?" using a language generation model based on the position information of the tracking mark 21. The generated sentence is then input into an escape route prediction model, which generates a sentence such as "The suspicious person is likely heading towards EF in ABC Town." In this case, the drone control unit 36 ​​sets a first flight route and a second flight route toward "EF in ABC Town."

[0060] By setting the first flight path and the second flight path based on the predicted escape route of the suspicious subject, the short-range drone 11 and the long-range drone 3 can get to the suspicious subject's destination in advance.

[0061] The set first flight path is transmitted to the short-range drone 11. Upon receiving the first flight path, the short-range drone 11 takes off from the base unit 16, flies along the first flight path, and tracks the suspicious object.

[0062] The set second flight path is transmitted to the long-range drone 3. Upon receiving the second flight path, the long-range drone 3 takes off from the waiting station, flies along the second flight path, and tracks the suspicious object.

[0063] The drone control unit 36 ​​generates a launch signal for the tracking mark 21 of the short-range drone 11. For example, the drone control unit 36 ​​calculates the distance to the suspicious target from an image captured by the camera 20 of the short-range drone 11. Then, if the distance to the suspicious target is equal to or less than a predetermined distance, the drone control unit 36 ​​generates a launch signal for launching the tracking mark 21 toward the suspicious target. The generated launch signal is transmitted to the short-range drone 11. As a result, the tracking mark 21 is launched toward the suspicious target.

[0064] The robot control unit 37 sets a travel route for the robot body 10. The travel route includes a preset security route. The robot control unit 37 causes the robot body 10 to travel autonomously along the travel route. The robot control unit 37 causes the robot body 10 to travel autonomously in accordance with the situation detected by the detection unit 12. The robot control unit 37 causes the robot body 10 to travel autonomously by controlling the drive of the legs 10b and arms 10c of the robot body 10. The robot control unit 37 controls the drive of the legs 10b and arms 10c of the robot body 10 based on various information detected by the detection unit 12, for example, so that the robot body 10 travels while avoiding obstacles.

[0065] Furthermore, for example, when a suspicious object is detected, the robot control unit 37 sets a travel route to track the suspicious object. The robot control unit 37 sets a travel route to track the suspicious object, for example, based on the position information of the robot main body 10 and the position information at which the suspicious object was detected. For example, the robot control unit 37 sets a travel route to track the suspicious object, based on the current position information of the robot main body 10, the direction in which the suspicious object was detected relative to the robot main body 10, and the distance from the robot main body 10 to the suspicious object.

[0066] In addition, after the short-range drone 11 takes off, the robot control unit 37 sets a travel route to track the suspicious object based on the position information of the robot body 10 and the position information of the tracking mark 21.

[0067] When a suspicious object is detected, the robot control unit 37 may set a travel route to track the suspicious object based on the predicted escape route of the suspicious object. By setting the travel route of the robot main body 10 based on the predicted escape route of the suspicious object, the robot main body 10 can get to the destination of the suspicious object in advance.

[0068] Returning to Figure 1, the long-range drone 3 is an example of a moving object. The long-range drone 3 is, for example, a multicopter equipped with multiple rotary propellers, similar to the short-range drone 11, and performs unmanned autonomous flight. The short-range drone 11 is also powered by a battery.

[0069] The long-range drone 3 has a longer flight distance than the short-range drone 11. The flight distance is the distance that can be flown on a single charge. The long-range drone 3 waits at a preset station. For example, the long-range drone 3 has a larger battery to have a longer flight distance than the short-range drone 11. For example, the long-range drone 3 is larger than the short-range drone 11. The long-range drone 3 may be, for example, an airplane-type drone.

[0070] The long-range drone 3 is equipped with a camera, similar to the short-range drone 11. Unlike the short-range drone 11, the long-range drone 3 is not equipped with a tracking mark. Note that the long-range drone 3 may be equipped with a tracking mark, similar to the short-range drone 11.

[0071] The long-range drone 3 includes a communication module for wireless communication via the network N. The long-range drone 3 also includes various sensors such as an acceleration sensor, a gyro sensor, and an optical sensor.

[0072] The long-range drone 3 also has a positioning device 39 (see FIG. 1) for measuring its own position. The positioning device 39 is, for example, a GNSS. The position information of the long-range drone 3 detected by the positioning device 39 may be transmitted to the management device 4 via the network N. The position information of the long-range drone 3 detected by the positioning device 39 may be transmitted to the control device 13 via the network N. The long-range drone 3 also has a computer (for example, a microcomputer) that executes a flight control function, an attitude control function for controlling its attitude, and the like.

[0073] The long-range drone 3 acquires information about the second flight route from the control device 13 via the communication module. The information about the second flight route includes position information (for example, latitude, longitude, and altitude) of the second flight route.

[0074] The long-range drone 3 may fly to take over the tracking of the suspicious target by the short-range drone 11. The long-range drone 3 may track the suspicious target simultaneously with the short-range drone 11.

[0075] The long-range drone 3 transmits images taken by the camera to the management device 4 via the communication module. The long-range drone 3 transmits images taken by the camera to the control device 13 via the communication module.

[0076] When a suspicious object is detected, the long-range drone 3 takes off from the waiting station. The long-range drone 3 acquires information about the second flight path and flies along the second flight path to track the suspicious object. The long-range drone 3 performs image processing based on the images captured by the camera and flies along the second flight path while avoiding obstacles.

[0077] The ability to track a suspicious target can be improved by tracking the suspicious target using the short-range drone 11 and the long-range drone 3. For example, indoors, tracking can be performed using the short-range drone 11, which is smaller than the long-range drone 3, and if the suspicious target escapes outdoors, tracking can be performed using the long-range drone 3.

[0078] The management device 4 is, for example, a server device. The management device 4 may be a cloud server. As shown in Fig. 4, the management device 4 includes a communication unit 40, a storage unit 41, and a control unit 42. Fig. 4 is a functional block diagram showing an outline of the management device 4 according to the embodiment.

[0079] The management device 4 collects, from the robot 2 that detected the suspicious object, various pieces of information detected by the detection unit 12 of the robot 2. The management device 4 also collects, from a robot 2 different from the robot 2 that detected the suspicious object, various pieces of information detected by the detection unit 12. The management device 4 also collects, from multiple long-range drones 3, position information of each long-range drone 3.

[0080] The management device 4 may generate information regarding the travel path of the robot 2. The management device 4 may also generate information regarding the first flight path of the short-range drone 11. The management device 4 may also generate information regarding the second flight path of the long-range drone 3.

[0081] The communication unit 40 is connected to the network N by wire or wirelessly. The communication unit 40 transmits and receives information to and from the control device 13 of the robot 2, the short-range drone 11, and the long-range drone 3 via the network N.

[0082] The communication unit 40 receives various information detected by the detection unit 12 from the control device 13 of the robot 2. The communication unit 40 receives images captured by the camera 20 from the short-range drone 11. The communication unit 40 receives position information from the short-range drone 11, the tracking mark 21, and the long-range drone 3. The communication unit 40 receives images captured by the camera from the long-range drone 3.

[0083] The storage unit 41 is realized by, for example, a semiconductor memory element such as RAM or flash memory, or a storage device such as an HDD, SSD, or optical disk. Various programs and various data are stored in the storage unit 41. For example, the storage unit 41 stores various pieces of information detected by the detection unit 12 of each robot 2.

[0084] The control unit 42 is a controller and includes, for example, a microcomputer and various circuits having a CPU, ROM, RAM, input / output ports, etc. The control unit 42 may also be configured with hardware such as an integrated circuit, for example, an ASIC or FPGA.

[0085] The control unit 42 may generate information regarding the travel route of the robot body 10. For example, the control unit 42 generates information regarding the travel route of the robot body 10 using a sentence generation model (a so-called AI chat engine). As described above, the sentence generation model may be interpreted as an algorithm and calculation for automatic dialogue processing using text.

[0086] The control unit 42 generates a question about the suspicious object from various pieces of information detected by the detection unit 12 of the robot 2. First, the control unit 42 generates a question using, for example, a language generation model.

[0087] For example, if movement is detected by the infrared sensor of Robot 2, the following question will be generated: "The infrared sensor detected this movement. Is this movement a suspicious person? From a criminal's perspective, what action would they take next?"

[0088] When such a question is input to the sentence generation model, the control unit 42 generates a sentence such as, "There is a high possibility that this person is a suspicious person. If he senses the presence of a person, he will take action to run away." The generated sentence is transmitted to the robot 2 as information regarding the travel route, and the robot 2 sets a travel route so as to approach the person presumed to be a suspicious person at a low speed.

[0089] Also, for example, if a sound is detected by a sensor of robot 2 that detects minute sounds, a question such as "I can hear a beep from 2 meters away. What sound is this?" is generated.

[0090] When such a question is input to the sentence generation model, the control unit 42 generates a sentence such as, for example, "An alarm is sounding." The generated sentence is sent to the robot 2 as information regarding the travel route, and the robot 2 recognizes the object as suspicious, and sets a travel route to approach the suspicious object via, for example, the shortest route.

[0091] The control unit 42 may similarly generate information about the travel route of the robot 2 based on other information detected by the detection unit 12 of the robot 2. The control unit 42 may generate information about the travel route of the robot 2 based on information obtained by a high-sensitivity camera capable of 360-degree sensing, a LiDAR, a thermal camera, a radar, or the like. The control unit 42 may generate information about the travel route of the robot 2 based on information obtained by sensors such as vision recognition, ultrasonic waves, vibrations, ultraviolet rays, and electromagnetic waves.

[0092] Similarly, information regarding the travel route of the robot 2 may be generated based on a plurality of pieces of information detected by the detection unit 12 of the robot 2. The control unit 42 may set the travel route of the robot 2.

[0093] By generating information about the travel path of the robot 2 using the document generation model, the robot 2 can, for example, accurately determine a suspicious object and can move the robot 2 closer to the suspicious object without the suspicious object noticing. This allows the robot 2 to improve the hit rate of the tracking mark 21 emitted from the short-range drone 11, for example. In this way, by generating information about the travel path of the robot 2 using the document generation model, the ability of the robot 2 to track suspicious objects can be improved.

[0094] The control unit 42 may also generate information about at least one of the first flight path of the short-range drone 11 and the second flight path of the long-range drone 3. For example, the control unit 42 generates information about the first flight path using a sentence generation model.

[0095] For example, if the robot 2 detects a suspicious object, the suspicious object is a car, and the license plate number of the suspicious object is detected as "AA-BB," the control unit 42 uses the language generation model to create the sentence "Take a picture of the car with the license plate number AA-BB." The control unit 42 then inputs the generated sentence into the sentence generation model to generate information about the first flight path. For example, the control unit 42 generates a flight program for the short-range drone 11 that captures a picture of the car with the license plate number "AA-BB" using the camera of the short-range drone 11 and tracks the car with the license plate number "AA-BB." The control unit 42 then generates the generated flight program as information about the first flight path. The generated information about the first flight path is transmitted to the short-range drone 11, causing the short-range drone 11 to fly in a manner that tracks the car with the license plate number "AA-BB." The image captured by the camera of the short-range drone 11 is transmitted to the management device 4.

[0096] For example, by using a document generation model to generate information regarding the first flight path of the short-range drone 11, the ability of the short-range drone 11 to track suspicious objects can be improved.

[0097] The generation of information about the travel route of the robot 2 using the sentence generation model may be executed by the robot 2.

[0098] Next, a suspicious object detection process according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart illustrating the suspicious object detection process executed by the robot 2 according to the embodiment. The suspicious object detection process is executed by the control device 13 of the robot 2.

[0099] The control unit 32 acquires various pieces of information detected by the detection unit 12 (S10). Based on the acquired various pieces of information, the control unit 32 determines whether or not a metal object (a bladed object such as a knife or scissors) has been detected around the robot main body 10 (S11). If a metal object has been detected (S11: Yes), the control unit 32 detects the object as a suspicious object (S12). Next, the control unit 32 executes a notification process to notify that a suspicious object has been detected (S13).

[0100] If no metal object is detected (S11: No), the control unit 32 determines whether the shape of the object present around the robot main body 10 is a suspicious object shape (a weapon such as a gun or knife) based on the acquired various information (S14). If the shape of the object is a suspicious object shape (S14: Yes), the control unit 32 proceeds to S12, detects the object as a suspicious object, and executes the notification process of S13.

[0101] If the shape of the object is not a suspicious shape (S14: No), the control unit 32 determines whether the odor of the object present around the robot main body 10 is a suspicious odor (the odor of an illegal drug such as a narcotic) based on the various acquired information (S15). If the odor of the object is a suspicious odor (S15: Yes), the control unit 32 proceeds to S12, detects the object as a suspicious object, and executes the notification process of S13.

[0102] If the odor of the object is not a suspicious odor (S15: No), the control unit 32 determines whether the person with the identification information is a suspicious person (wanted criminal or suspicious person) based on the identification information of the person present around the robot body (S16). If the person with the identification information is a suspicious person (S16: Yes), the control unit 32 proceeds to S12, detects the person as a suspicious person, and executes the notification process of S13. On the other hand, if the person with the identification information is not a suspicious person (S16: No), the control unit 32 ends the process.

[0103] Although the above example shows that the processes are performed in the order of S11, S14, S15, S16, and S17, the present invention is not limited to this. That is, the order of the processes of S11, S14, S15, S16, and S17 can be set arbitrarily, and all or part of the processes may be performed at the same time.

[0104] Next, the control processing of the short-range drone 11 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating the control processing of the short-range drone 11 according to the embodiment. The control processing of the short-range drone 11 is executed by the control device 13 of the robot 2.

[0105] The control unit 32 acquires various pieces of information detected by the detection unit 12 (S100). The control unit 32 determines whether a suspicious object has been detected based on the acquired various pieces of information (S101). For example, the control unit 32 detects a suspicious object by performing the above-described suspicious object detection process or by performing predetermined image processing on an image captured by a high-sensitivity camera.

[0106] If a suspicious object is not detected (S101: No), the control unit 32 ends this processing. If a suspicious object is detected (S101: Yes), the control unit 32 sets a first flight path for the short-range drone 11 (S102).

[0107] The control unit 32 transmits the first flight path to the short-range drone 11 (S103). As a result, the short-range drone 11 takes off from the base unit 16 of the robot 2.

[0108] The control unit 32 acquires the image captured by the camera 20 of the short-range drone 11 (S104).

[0109] The control unit 32 determines whether the distance to the suspicious target is equal to or less than a predetermined distance (S105). If the distance to the suspicious target is greater than the predetermined distance (S105: No), the control unit 32 continues tracking the suspicious target and acquires an image captured by the camera 20 of the short-range drone 11 (S104).

[0110] If the distance to the suspicious target is equal to or shorter than the predetermined distance (S105: Yes), the control unit 32 generates a signal to emit the tracking mark 21 (S106). The control unit 32 transmits the generated emission signal to the short-range drone 11 (S107). As a result, the tracking mark 21 is emitted from the short-range drone 11 toward the suspicious target.

[0111] Next, the control processing of the long-range drone 3 according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart illustrating the control processing of the long-range drone 3 according to the embodiment. The control processing of the long-range drone 3 is executed by the control device 13 of the robot 2.

[0112] The control unit 32 acquires various pieces of information detected by the detection unit 12 (S200). The control unit 32 determines whether or not a suspicious object has been detected based on the acquired various pieces of information (S201).

[0113] If a suspicious object is not detected (S201: No), the control unit 32 ends this processing. If a suspicious object is detected (S201: Yes), the control unit 32 sets a second flight path for the long-range drone 3 (S202).

[0114] The control unit 32 transmits the second flight path to the short-range drone 11 (S203). As a result, the long-range drone 3 takes off from the station.

[0115] The long-range drone 3 flies based on the position information of the tracking mark 21. Therefore, as shown in FIG. 8A, when the short-range drone 11 has not emitted the tracking mark 21 at the suspicious target C, the long-range drone 3 tracks the short-range drone 11 and flies so as to approach the suspicious target C. Then, as shown in FIG. 8B, when the tracking mark 21 is emitted from the short-range drone 11 and attached to the suspicious target C, the long-range drone 3 flies so as to track the suspicious target C. FIG. 8A is a diagram showing an example of the flight of the long-range drone 3 before the tracking mark 21 is emitted. FIG. 8B is a diagram showing an example of the flight of the long-range drone 3 after the tracking mark 21 has attached to the suspicious target C.

[0116] The camera that captures images for detecting and tracking a suspicious object is not limited to a camera installed on the robot 2, etc. For example, if the suspicious object is indoors, a camera on another indoor robot or a surveillance camera installed in the room may be used as the camera that captures images for detecting and tracking a suspicious object. Also, if the suspicious object is outdoors, an outdoor surveillance camera may be used as the camera that captures images for detecting and tracking a suspicious object. That is, for example, the robot 2 may detect a suspicious object based on images taken by a camera on another indoor robot, an indoor surveillance camera, and an outdoor surveillance camera.

[0117] As described above, the robot 2 according to this embodiment includes the robot main body 10, the detection unit 12, and the suspicious object detection unit 35. The detection unit 12 detects the surrounding situation indicating the situation around the robot main body 10. The suspicious object detection unit 35 detects a suspicious object present around the robot main body 10 based on the surrounding situation detected by the detection unit 12. In this way, by using the surrounding situation detected by the detection unit 12 included in the robot 2, it is possible to accurately detect a suspicious object.

[0118] The detection unit 12 detects metallic objects present around the robot main body 10 as surrounding conditions. The suspicious object detection unit 35 detects the objects detected by the detection unit 12 as suspicious objects. This makes it possible to accurately detect metallic objects (e.g., blades) present around the robot main body 10 as suspicious objects.

[0119] The detection unit 12 detects the shapes of objects present around the robot main body 10 as the surrounding situation. If the shape of an object detected by the detection unit 12 is a predetermined suspicious object shape, the suspicious object detection unit 35 detects the object as a suspicious object. This makes it possible to accurately detect objects such as weapons present around the robot main body 10 as suspicious objects.

[0120] The detection unit 12 detects the odor of an object present around the robot main body 10 as the surrounding conditions. If the odor detected by the detection unit 12 is a predetermined suspicious object odor, the suspicious object detection unit 35 detects the object as a suspicious object. This allows objects such as illegal drugs present around the robot main body 10 to be accurately detected as suspicious objects.

[0121] The detection unit 12 detects, as the surrounding situation, identification information that can identify a person present around the robot main body 10. If the person is a predetermined suspicious person based on the identification information detected by the detection unit 12, the suspicious object detection unit 35 detects the person as a suspicious object. This makes it possible to accurately detect people such as wanted criminals and suspicious individuals present around the robot main body 10 as suspicious objects.

[0122] The robot 2 according to this embodiment includes a notification unit 14 that notifies the detection of a suspicious object by the suspicious object detection unit 35. This allows the detection of a suspicious object to be notified to those around, thereby enabling, for example, a security guard or the like to take appropriate action against the suspicious object.

[0123] The robot 2 comprises a short-range drone 11, a robot main body 10, a detection unit 12, and a control unit 32. The short-range drone 11 is capable of tracking a suspicious object and emitting a tracking mark 21 toward the suspicious object. The robot main body 10 is capable of launching and landing the short-range drone 11 and of autonomous travel. The detection unit 12 detects the situation around the robot main body 10. The control unit 32 controls the drone and causes the robot main body 10 to autonomously travel in accordance with the situation detected by the detection unit 12. When a suspicious object is detected, the control unit 32 launches the short-range drone 11 from the robot main body 10 and causes the short-range drone 11 to emit a tracking mark 21 toward the suspicious object.

[0124] This allows the robot 2 to track the suspicious object by tracking the tracking mark 21. Therefore, the robot 2 can easily identify the suspicious object, making it easier to track the suspicious object and enabling accurate identification of the suspicious object. Therefore, the robot 2 can improve the ability to track the suspicious object.

[0125] 9 is a diagram schematically illustrating an example of a computer hardware configuration that functions as the robot 2 or the management device 4. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or can cause the computer 1200 to perform operations associated with the device according to the present embodiment or one or more "parts," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0126] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0127] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0128] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0129] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0130] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0131] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0132] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0133] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0134] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0135] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0136] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0137] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0138] Computer-readable instructions may be provided locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc. to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or programmable circuitry, such that the processor or programmable circuitry executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0139] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0140] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0141] 1. Robot System 2. Robot 3 Long-Range Drone 4 Management device 10 Robot body 11 Short-Range Drone 12 Detector 13 Control device 14. Information Department 20 Camera 21 Tracking Mark 32 Control section 35 Suspicious object detection unit 36 Drone control unit 37 Robot control unit

Claims

1. The robot body, a detection unit that detects a surrounding situation indicating a surrounding situation of the robot body; a suspicious object detection unit that detects a suspicious object present around the robot body based on the surrounding situation detected by the detection unit; A robot equipped with:

2. The detection unit detecting a metal object present around the robot body as the surrounding situation; The suspicious object detection unit Detecting the object detected by the detection unit as the suspicious object; The robot of claim 1 .

3. The detection unit detecting the shapes of objects present around the robot body as the surrounding situation; The suspicious object detection unit If the shape of the object detected by the detection unit is a predetermined suspicious object shape, the object is detected as the suspicious object. The robot of claim 1 .

4. The detection unit Detecting odors from objects present around the robot body as the surrounding conditions; The suspicious object detection unit If the odor detected by the detection unit is a predetermined suspicious odor, the object is detected as the suspicious object. The robot of claim 1 .

5. The detection unit detecting, as the surrounding situation, identification information that can identify a person present around the robot body; The suspicious object detection unit If the person is a predetermined suspicious person based on the identification information detected by the detection unit, the person is detected as the suspicious person. The robot of claim 1 .

6. a notification unit that notifies that the suspicious object has been detected by the suspicious object detection unit; The robot of claim 1 , comprising:

Images