Information processing device, control method, and control program

Abstract

An information processing device (100) communicates with a robot device (200). The information processing device (100) comprises: an acquisition unit (120) that acquires information indicating that a target has been detected; and a control unit (130) that controls the robot device (200) so as to repeatedly image the target.

Description

Information processing device, control method, and control program

[0001] The present disclosure relates to an information processing device, a control method, and a control program.

[0002] 2. Description of the Related Art A robot is known, and a technique for controlling the robot has been proposed (see Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-117695

[0004] In machine learning, scarce data may be used as training data. For example, scarce data may include images of suspicious objects, suspicious people, areas showing signs of aging, and dangerous locations. Machine learning requires a large amount of training data. However, collecting a large amount of scarce data is difficult.

[0005] The goal of this disclosure is to allow for the collection of a large amount of scarce data.

[0006] According to one aspect of the present disclosure, there is provided an information processing device that communicates with a robotic device, the information processing device including an acquisition unit that acquires information indicating that an object has been detected, and a control unit that controls the robotic device to repeatedly capture images of the object.

[0007] According to the present disclosure, a large amount of scarce data can be collected.

[0008] FIG. 1 is a diagram illustrating a control system. FIG. 2 is a diagram illustrating hardware included in an information processing device. FIG. 3 is a block diagram illustrating functions of an information processing device. FIG. 4 is a flowchart illustrating an example of processing executed by an information processing device of Modification 1. FIG. 5 is a flowchart illustrating an example of processing executed by an information processing device of Modification 2. FIG. 6 is a flowchart illustrating an example of processing executed by an information processing device of Modification 3. FIG. 7 is a flowchart illustrating an example of processing executed by an information processing device of Modification 4.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] 1 is a diagram showing a control system. The control system includes an information processing device 100, robot devices 200a and 200b, and a terminal device 300. The information processing device 100, the robot devices 200a and 200b, and the terminal device 300 communicate with each other via a network 10.

[0011] The information processing device 100 is a device that executes a control method. For example, the information processing device 100 is a server. The robot device 200a is a traveling robot device. The robot device 200b is a flying robot device. For example, the robot device 200b is a drone. Here, the robot devices 200a and 200b are collectively referred to as the robot device 200.

[0012] For example, the robot device 200 has an RGB (Red Green Blue) camera, an RGB-D (Depth) camera, a depth sensor, a LiDAR (Light Detection and Ranging), a millimeter wave radar, etc. The robot device 200 transmits its current position, remaining battery power, etc. to the information processing device 100.

[0013] The robot device 200 can also detect rare data objects. For example, the robot device 200 can detect suspicious objects. For example, the robot device 200 captures an image of an object, inputs the captured image into a trained model, and if the trained model outputs information indicating that the object is a suspicious object, detects the object as a suspicious object. The robot device 200 may also detect a suspicious object using the image and image recognition technology. Furthermore, for example, if the object does not move for a long time after being detected, the robot device 200 determines that the object is a suspicious object. The robot device 200 detects a suspicious object by making a determination. For example, when the robot device 200 detects an object, it transmits information indicating that the object has been detected to the information processing device 100.

[0014] The terminal device 300 is a device used by a user. For example, the terminal device 300 is a smartphone, a personal computer (PC), etc. The terminal device 300 can transmit a movement request to the robot device 200 via the information processing device 100.

[0015] Next, the hardware of the information processing device 100 will be described. Fig. 2 is a diagram showing the hardware of the information processing device. The information processing device 100 is a computer. The information processing device 100 has a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0016] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0017] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The nonvolatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the nonvolatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0018] Next, a description will be given of the functions of the information processing device 100. Fig. 3 is a block diagram showing the functions of the information processing device. The information processing device 100 has a storage unit 110, an acquisition unit 120, and a control unit 130.

[0019] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. The acquisition unit 120 and the control unit 130 may be partly or entirely realized by a processing circuit. Furthermore, the acquisition unit 120 and the control unit 130 may be partly or entirely realized as a program module executed by the processor 101. For example, the program executed by the processor 101 is also referred to as a control program. For example, the control program is recorded on a recording medium.

[0020] The storage unit 110 stores map information, the current position of the robot device 200, the remaining battery level of the robot device 200, information indicating the status of the robot device 200, etc. For example, the status of the robot device 200 may be stopped, moving, or under monitoring.

[0021] The acquisition unit 120 acquires information indicating that a target has been detected. For example, the acquisition unit 120 acquires the information from the robot device 200. The acquisition unit 120 may acquire the information from the robot device 200 via an external device. For example, the external device is a cloud server. The external device is not shown in the drawing.

[0022] The control unit 130 controls the robot device 200 to repeatedly capture images of the target. Specifically, the control unit 130 transmits an instruction to the robot device 200 to repeatedly capture images of the target. As a result, the robot device 200 repeatedly captures images of the target. As the robot device 200 repeatedly captures images, a large amount of rare data is collected. The robot device 200 may transmit a large amount of rare data to the information processing device 100, or may transmit a large amount of rare data to the terminal device 300. In this way, the information processing device 100 can cause the robot device 200 to collect a large amount of rare data.

[0023] Next, a process executed by the information processing device 100 will be described using a flowchart. In the following description, the target is a suspicious object. FIG. 4 is a flowchart showing an example of a process executed by the information processing device. First, it is assumed that the robot device 200 is performing surveillance. It is assumed that the robot device 200 has detected a suspicious object. Then, the robot device 200 transmits information indicating that a suspicious object has been detected.

[0024] (Step S11) The acquisition unit 120 acquires information indicating that a suspicious object has been detected. (Step S12) The control unit 130 controls the robotic device 200 to repeatedly capture images while moving around the suspicious object. Specifically, the control unit 130 transmits an instruction to the robotic device 200 to repeatedly capture images while moving around the suspicious object. The control unit 130 may also use map information to generate route information for moving around the suspicious object. Then, the control unit 130 may transmit the instruction and the route information to the robotic device 200. Based on the instruction, the robotic device 200 repeatedly captures images while moving around the suspicious object. Thus, many images of the suspicious object are collected.

[0025] Modification 1 of the embodiment In the above embodiment, an example was described in which the target was a suspicious object. In Modification 1, a case in which the target is a suspicious person will be described.

[0026] FIG. 5 is a flowchart showing an example of processing executed by the information processing device of Modification 1. First, it is assumed that the robot device 200 is performing surveillance. It is assumed that the robot device 200 has detected a suspicious individual. For example, the robot device 200 captures an image of a person behaving suspiciously and detects the suspicious individual using the captured image and a trained model. Furthermore, for example, the robot device 200 detects the suspicious individual using the image and image recognition technology. Furthermore, for example, if the robot device 200 detects a person speaking loudly, it detects the person as a suspicious individual. Then, the robot device 200 transmits information indicating that a suspicious individual has been detected.

[0027] (Step S21) The acquisition unit 120 acquires information indicating that a suspicious individual has been detected. (Step S22) The control unit 130 controls the robot device 200 to repeatedly capture images while tracking the suspicious individual. Specifically, the control unit 130 sends an instruction to the robot device 200 to repeatedly capture images while tracking the suspicious individual. As a result, the robot device 200 repeatedly captures images while tracking the suspicious individual. Thus, many images of the suspicious individual are collected.

[0028] Modification 2 of the Embodiment. Modification 2 describes a case where the target is a location where deterioration over time has occurred. FIG. 6 is a flowchart illustrating an example of processing executed by the information processing device of Modification 2. First, it is assumed that the robot device 200 is performing monitoring. It is assumed that the robot device 200 has detected a location where deterioration over time has occurred. For example, when the robot device 200 detects a crack in a building, it captures an image of the location where the crack exists, inputs the captured image into a trained model, and if the trained model outputs information indicating that deterioration over time has occurred, it detects the location as a location where deterioration over time has occurred. The robot device 200 may detect the location where deterioration over time has occurred using the image and image recognition technology. Then, the robot device 200 transmits information indicating that a location where deterioration over time has occurred has been detected.

[0029] (Step S31) The acquisition unit 120 acquires information indicating that a location where deterioration has occurred has been detected. (Step S32) The control unit 130 controls the robot device 200 to periodically capture images of the location where deterioration has occurred. Specifically, the control unit 130 sends an instruction to the robot device 200 to periodically capture images of the location where deterioration has occurred. As a result, the robot device 200 periodically captures images of the location where deterioration has occurred. Thus, many images of the location where deterioration has occurred are collected.

[0030] Furthermore, when the robot device 200 captures an image of a location where deterioration has occurred over time, the control unit 130 may transmit an instruction to the robot device 200 to capture the image using any of a method using an RGB-D camera, a method using an RGB camera and a depth sensor, and a method of capturing images of the location from various angles. For example, when the robot device 200 transmits an image obtained by capturing the image to the information processing device 100, the control unit 130 can generate three-dimensional information of the location using the image. The control unit 130 may output the three-dimensional information of the location. For example, the control unit 130 outputs the three-dimensional information of the location to the terminal device 300. This allows the user to recognize the location in three dimensions.

[0031] Modification 3 of the embodiment. Modification 3 describes a case where the target is a site where danger is occurring. FIG. 7 is a flowchart illustrating an example of processing executed by an information processing device according to Modification 3. First, it is assumed that the robot device 200 is performing monitoring. It is assumed that the robot device 200 has detected a site where danger is occurring. For example, after an earthquake occurs, the robot device 200 captures an image of a bridge, inputs the captured image into a trained model, and if the trained model outputs information indicating that the bridge is in a state where it is about to collapse, it detects the site where the bridge is present as a site where danger is occurring. Alternatively, the robot device 200 may use image recognition technology to detect the site where danger is occurring. Then, the robot device 200 transmits information indicating that a site where danger is occurring has been detected.

[0032] (Step S41) The acquisition unit 120 acquires information indicating that a site where a danger is occurring has been detected. (Step S42) The control unit 130 controls the robot device 200 to periodically capture images of the site where a danger is occurring. Specifically, the control unit 130 sends an instruction to the robot device 200 to periodically capture images of the site where a danger is occurring. As a result, the robot device 200 periodically captures images of the site where a danger is occurring. Thus, many images of the site where a danger is occurring are collected.

[0033] Furthermore, when the robot device 200 captures an image of a dangerous site, the control unit 130 may transmit an instruction to the robot device 200 to capture the image using any of the following methods: a method using an RGB-D camera, a method using an RGB camera and a depth sensor, and a method of capturing images of the site from various angles. For example, when the robot device 200 transmits an image obtained by capturing the image to the information processing device 100, the control unit 130 can generate three-dimensional information of the site using the image. The control unit 130 may output the three-dimensional information of the site. For example, the control unit 130 outputs the three-dimensional information of the site to the terminal device 300. This allows the user to recognize the site in three dimensions.

[0034] Variation 4 of the embodiment. In Variation 4, a case where the target is an unknown obstacle present on the travel path of the robot device 200 will be described. FIG. 8 is a flowchart illustrating an example of processing executed by the information processing device of Variation 4. First, it is assumed that the robot device 200 is moving to a destination while performing monitoring. It is assumed that the robot device 200 detects an unknown obstacle on the travel path. For example, the robot device 200 captures or measures the unknown obstacle on the travel path, and detects the unknown obstacle using the captured image or distance data obtained by a sensor such as LiDAR and a trained model. Then, the robot device 200 transmits information indicating that an unknown obstacle has been detected on the travel path.

[0035] (Step S51) The acquisition unit 120 acquires information indicating that an unknown obstacle has been detected on the travel path. (Step S52) The control unit 130 controls the robotic device 200 to move to the destination while avoiding the unknown obstacle and to continuously capture images of the obstacle while avoiding the obstacle. For example, the control unit 130 transmits to the robotic device 200 an instruction to move to the destination while avoiding the obstacle, and an instruction to continuously capture images of the obstacle while avoiding the obstacle while avoiding the obstacle. This allows many unknown obstacles to be collected. Furthermore, the robotic device 200 may measure the obstacle.

[0036] Furthermore, when the robot device 200 captures or measures an unknown obstacle, the control unit 130 may transmit an instruction to the robot device 200 to perform the image capture or measurement using any of the following methods: a method using an RGB-D camera, a method using an RGB camera and a depth sensor, a method using an image capture of the obstacle from various angles, and a method using LiDAR. For example, when the robot device 200 transmits data obtained by image capture or measurement to the information processing device 100, the control unit 130 can generate three-dimensional information of the obstacle using the data. The control unit 130 may output the three-dimensional information of the obstacle. For example, the control unit 130 outputs the three-dimensional information of the obstacle to the terminal device 300. This allows the user to recognize the obstacle in three dimensions.

[0037] REFERENCE SIGNS LIST 10 network, 100 information processing device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 110 storage unit, 120 acquisition unit, 130 control unit, 200 robot device, 200a, 200b robot device, 300 terminal device.

Claims

1. An information processing device that communicates with a robot device, an acquisition unit that acquires information indicating that a rare data or unknown data target has been detected; a control unit that controls the robot device so as to repeatedly capture images of the object to collect learning data; An information processing device having the above.

2. The target is a suspicious object, the acquisition unit acquires information indicating that the suspicious object has been detected, the control unit controls the robot device to repeatedly take images while moving around the suspicious object. The information processing device according to claim 1 .

3. The target is a suspicious person, the acquisition unit acquires information indicating that the suspicious individual has been detected, the control unit controls the robot device to repeatedly capture images while tracking the suspicious person. The information processing device according to claim 1 .

4. The target is a location where deterioration over time has occurred, the acquisition unit acquires information indicating that the portion where aging deterioration has occurred has been detected, the control unit controls the robot device to periodically capture an image of the area where aging deterioration has occurred. The information processing device according to claim 1 .

5. When the robot device captures an image of the location where deterioration has occurred over time, the control unit transmits an instruction to the robot device to capture the image using one of a method using an RGB (Red Green Blue)-D (Depth) camera, a method using an RGB camera and a depth sensor, and a method of capturing images of the location from various angles. The information processing device according to claim 4 .

6. The target is a site where a danger is occurring, the acquisition unit acquires information indicating that the scene where a danger is occurring has been detected, the control unit controls the robot device to periodically capture an image of the scene where a danger is occurring. The information processing device according to claim 1 .

7. When the robot device captures an image of the site where a danger is occurring, the control unit transmits an instruction to the robot device to capture the image using any one of a method using an RGB (Red Green Blue)-D (Depth) camera, a method using an RGB camera and a depth sensor, and a method of capturing images of the site from various angles. The information processing device according to claim 6 .

8. the target is an unknown obstacle present on a travel path of the robot device, the acquisition unit acquires information indicating that the obstacle has been detected on the travel route; the control unit controls the robot device to move to the destination while avoiding the obstacle, and to continuously capture images of the obstacle while avoiding the obstacle. The information processing device according to claim 1 .

9. When the robot device captures an image of the obstacle, the control unit transmits an instruction to the robot device to capture the image using any one of a method using an RGB (Red Green Blue)-D (Depth) camera, a method using an RGB camera and a depth sensor, and a method of capturing the image of the obstacle from various angles. The information processing device according to claim 8 .

10. An information processing device that communicates with the robot device, obtaining information indicating that a rare or unknown data subject has been detected; controlling the robotic device to repeatedly capture images of the object to collect learning data of the object; Control method.

11. A computer communicating with the robotic device obtaining information indicating that a rare or unknown data subject has been detected; controlling the robotic device to repeatedly capture images of the object to collect learning data of the object; A control program that executes processing.