Data processing device, data processing method, and mobile body

Abstract

The present technique relates to a data processing apparatus, a data processing method, and a robot with which it is possible to perform environment sensing using an appropriate algorithm. The data processing apparatus according to an aspect of the present technique includes a sensing control unit which adaptively selects and executes an environment sensing program in accordance with environment sensing conditions, the environment sensing program specifying an environment sensing algorithm that performs environment sensing on the basis of sensor data output from a sensor installed on a robot. The present technique is applicable to sensor devices mounted on various apparatuses.

Description

Technical Field

[0001] This technology particularly relates to a data processing device and a data processing method that can perform environmental sensing using an appropriate algorithm. Moving body It relates to.

Background Art

[0002] Conventionally, various technologies for updating the software of a device have been proposed from the viewpoints of adding functions and ensuring compatibility with other devices.

[0003] For example, Patent Document 1 discloses a technology for determining a service that can be realized by a combination of a camera and a communication device and installing software for providing the service.

[0004] Also, Patent Document 2 discloses a technology for updating the firmware between a photographing device and a host system when it is detected that the firmware of the photographing device is incompatible with the host system.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] Although various technologies for changing the software of a photographing device such as a camera have been disclosed, there is no disclosure regarding adaptively changing the software of a device that performs sensing according to the situation of the sensing target and the like.

[0007] This technology has been made in view of such a situation, and it enables environmental sensing to be performed using an appropriate algorithm.

Means for Solving the Problems

[0008] The data processing device according to the first aspect of the present technology determines an environmental sensing algorithm for sensing the environment based on sensor data output from sensors mounted on a moving body having a movable operation unit with respect to the moving body main body, according to environmental sensing conditions, Determined the environmental sensing algorithm Select and execute a defined environmental sensing program from an environmental sensing program set that is a combination of a plurality of the environmental sensing programs and includes a combination of information representing the types of the environmental sensing algorithms and information representing the execution order of the environmental sensing programs, controls the movement of the moving body and the operation of the operation unit.

[0009] The data processing device according to the second aspect of the present technology determines an environmental sensing algorithm for sensing the environment based on sensor data output from sensors mounted on a moving body having a movable operation unit with respect to the moving body main body, according to environmental sensing conditions, and the determined environmental sensing algorithm Select a defined environmental sensing program from an environmental sensing program set that is a combination of a plurality of the environmental sensing programs and includes a combination of information representing the types of the environmental sensing algorithms and information representing the execution order of the environmental sensing programs, includes a data processing unit that transmits data to the moving body.

[0010] In the third aspect of the present technology Moving body includes a sensor that outputs sensor data representing a sensing result, a sensing control unit that adaptively selects and executes an environmental sensing program in which an environmental sensing algorithm for sensing the environment based on the sensor data output from the sensor is defined, according to environmental sensing conditions, an operation plan setting unit that sets an operation plan based on the execution result of the environmental sensing program by the sensing control unit, and an operation unit that operates according to the operation plan set by the operation plan setting unit The operating part is movable with respect to the moving body main body .

[0011] In the first aspect of the present technology, an environmental sensing algorithm for sensing the environment based on sensor data output from sensors mounted on a moving body having a movable operation unit with respect to the moving body main body is determined according to environmental sensing conditions, DecidedThe environmental sensing algorithm The defined environmental sensing program is selected and executed from an environmental sensing program set that is a combination of a plurality of the environmental sensing programs and includes a combination of information representing the types of the environmental sensing algorithms and information representing the execution order of the environmental sensing programs, The movement of the moving body and the operation of the operating unit are controlled.

[0012] In a second aspect of the present technology, an environmental sensing algorithm for sensing the environment based on sensor data output from a sensor mounted on a moving body having a movable operating unit with respect to the moving body main body is determined according to environmental sensing conditions, and the determined environmental sensing algorithm The defined environmental sensing program is selected from an environmental sensing program set that is a combination of a plurality of the environmental sensing programs and includes a combination of information representing the types of the environmental sensing algorithms and information representing the execution order of the environmental sensing programs, Is transmitted to the moving body.

[0013] In a third aspect of the present technology, an environmental sensing program in which an environmental sensing algorithm for sensing the environment based on sensor data output from a sensor that outputs sensor data representing a sensing result is defined is adaptively selected and executed according to environmental sensing conditions, and based on the execution result of the environmental sensing program, an operation plan is set, and according to the set operation plan Of the operating part that is movable with respect to the moving body main body Operations are performed.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Embodiments for Carrying Out the Invention

[0015] <Summary of the Present Technology> The present technology focuses on the fact that it is difficult to perform sensing using an optimal algorithm in sensor devices mounted on various devices such as robots, mobile bodies, and smartphones, and realizes a solution method therefor.

[0016] Factors · There are many types of elemental technologies. · The maturity levels of elemental technologies vary. · There are many cost variations. · There are many application variations. · The difficulty levels of system design and implementation are high. · There are many types of user requirements. · There are many implementation constraints such as the power, power consumption, and circuit scale of the processor. · There are many types of sensing targets.

[0017] In particular, in the present technology, in a sensor device that performs environmental sensing, as a sensing algorithm related to sensing, it enables sensing to be performed using an optimal algorithm.

[0018] Hereinafter, embodiments for carrying out the present technology will be described. The description will be made in the following order. 1. Program providing system 2. Use case of sensing program 3. Configuration of transport robot 4. Operation of transport robot 5. Modification example

[0019] <Program providing system> · System configuration FIG. 1 is a diagram showing a configuration example of a program providing system according to an embodiment of the present technology.

[0020] In the program providing system of FIG. 1, various devices such as a mobile terminal 2-1, an arm robot 2-2, a mobile body 2-3, a cooking robot 2-4, and a transport robot 2-5 are connected to a program management server 1 via a network 11 composed of the Internet or the like.

[0021] The mobile terminal 2-1 is a smartphone.

[0022] The arm robot 2-2 is a dual-arm robot. A carriage is provided on the housing of the arm robot 2-2. The arm robot 2-2 is a movable robot.

[0023] The mobile body 2-3 is an automobile. The mobile body 2-3 is equipped with an automatic driving function and the like.

[0024] The cooking robot 2-4 is a kitchen-type robot. The cooking robot 2-4 has a function of performing cooking by driving a plurality of cooking arms. The cooking arms reproduce the same operations as those performed by a person.

[0025] The transport robot 2-5 is a robot that can place an object to be transported on a top plate prepared as a mounting table and move to a target position in that state. Wheels are provided on the base portion of the transport robot 2-5.

[0026] Each device shown in FIG. 1 is equipped with a sensor device used for environmental sensing, object sensing, human sensing, etc.

[0027] FIG. 2 is a diagram showing a configuration example of the sensor device.

[0028] As shown in FIG. 2, the sensor device 21 is provided with a controller 31 and a sensor group 32.

[0029] The controller 31 controls each sensor constituting the sensor group 32 to perform sensing of various targets such as environmental sensing, object sensing, and human sensing. The sensing by the controller 31 is performed based on the sensor data output by each sensor constituting the sensor group 32.

[0030] The controller 31 outputs the sensing result to the device on the host side. Based on the sensing result by the controller 31, various processes are performed on the device on the host side. When the sensor device 21 is mounted on the mobile terminal 2-1, the CPU (Central Processing Unit) of the smartphone becomes the device on the host side. The controller 31 is also provided with a function for communicating with the device on the host side.

[0031] The sensor group 32 is composed of a plurality of sensors that perform sensing of various targets. In the example of FIG. 2, the sensor group 32 is composed of an RGB camera 32A, a stereo camera 32B, a ToF sensor 32C, a structured-light sensor 32D, and a LiDAR 32E.

[0032] The RGB camera 32A has an image sensor for RGB images. The RGB camera 32A drives the image sensor to photograph the surrounding situation and outputs the RGB image obtained by the photographing as sensor data.

[0033] The stereo camera 32B is a distance sensor using the stereo camera method and has two image sensors for distance images. The stereo camera 32B outputs a distance image representing the distance to an object as sensor data.

[0034] The ToF sensor 32C is a distance sensor using the ToF (Time Of Flight) method. The ToF sensor 32C measures the distance to an object by the ToF method and outputs the distance information as sensor data.

[0035] The structured-light sensor 32D is a distance sensor using the structured-light method. The structured-light sensor 32D measures the distance to an object by the structured-light method and outputs the distance information as sensor data.

[0036] LiDAR (Light Detection and Ranging) 32E measures the three-dimensional position of each point of an object and outputs information representing the measurement result as sensor data.

[0037] Sensors different from the sensors shown in FIG. 2, such as a positioning sensor, a gyro sensor, an acceleration sensor, a temperature sensor, and an illuminance sensor, may be included in the sensor group 32.

[0038] The type of sensors constituting the sensor group 32 is appropriately changed by the device on which the sensor device 21 is mounted. The sensor group 32 may be constituted by one sensor.

[0039] The sensor device 21 may be constituted by a substrate on which the controller 31 and the sensor group 32 are arranged, or may be constituted as a device in which a substrate on which each sensor is arranged is housed in the housing 21A as shown in FIG. 3.

[0040] In the sensor device 21 having such a configuration, the controller 31 executes a sensing program which is a program for sensing, and realizes sensing functions for various targets such as the environment, objects, and people. The sensing function of the controller 31 is realized based on the output of one sensor constituting the sensor group 32, or based on a combination of the outputs of a plurality of sensors.

[0041] Environmental sensing (sensing of the environment) includes, for example, the following. · Shooting of RGB images using the RGB camera 32A · Measurement of the distance to the target object using the outputs of the stereo camera 32B, the ToF sensor 32C, and the structured-light sensor 32D · Generation of a three-dimensional map using the output of the LiDAR 32E · Estimation of the self-position using the three-dimensional map

[0042] The environment to be sensed by the sensor device 21 includes various physical states that are external to the sensor device 21 or external to the device on which the sensor device 21 is mounted and can be expressed as quantitative data by performing sensing.

[0043] Object sensing (sensing of objects) includes, for example, the following. · Recognition and identification of target objects using the RGB images captured by the RGB camera 32A · Measurement of characteristics of target objects such as shape, size, color, and temperature

[0044] The objects to be sensed by the sensor device 21 include various stationary objects and moving objects around the sensor device 21 or around the device on which the sensor device 21 is mounted.

[0045] Human sensing (sensing of people) includes, for example, the following. · Recognition of people, recognition of human faces, and identification of people using the RGB images captured by the RGB camera 32A ·Recognition of specific parts of a person, such as the head, arms, hands, eyes, nose, etc. ·Estimation of the position of a specific part, including bone estimation ·Estimation of a person's physical characteristics, such as height and weight ·Estimation of a person's attributes, such as age and gender

[0046] The person to be sensed by the sensor device 21 includes a person around the sensor device 21 or around the device on which the sensor device 21 is mounted.

[0047] The controller 31 has a plurality of programs with different algorithms as sensing programs for realizing respective sensing functions.

[0048] Figure 4 is a diagram showing an example of a sensing program prepared for the sensor device 21.

[0049] In the example of Figure 4, a distance measurement program A, a distance measurement program B, and a distance measurement program C are prepared as firmware operating on an OS (Operating System). The distance measurement program A, the distance measurement program B, and the distance measurement program C are sensing programs that realize a distance measurement function as an environmental sensing function.

[0050] The distance measurement program A, the distance measurement program B, and the distance measurement program C are sensing programs that realize the same distance measurement function by different sensing algorithms. The distance measurement program A, the distance measurement program B, and the distance measurement program C define different sensing algorithms.

[0051] The distance measurement program A is a sensing program that performs distance measurement by the distance measurement algorithm A. The distance measurement program B is a sensing program that performs distance measurement by the distance measurement algorithm B. The distance measurement program C is a sensing program that performs distance measurement by the distance measurement algorithm C.

[0052] For example, ranging algorithms A to C are sensing algorithms that perform ranging using different parameters, such as setting different parameters for the same sensor and calculating the distance by performing the same calculation based on the output of the sensor.

[0053] Also, ranging algorithms A to C are sensing algorithms that perform ranging using different calculation methods, such as setting the same parameters for the same sensor and calculating the distance by performing different calculations based on the output of the sensor.

[0054] When a plurality of distance sensors such as a stereo camera 32B, a ToF sensor 32C, and a structured-light sensor 32D are provided, ranging algorithms A to C may be sensing algorithms that perform ranging using different distance sensors.

[0055] In this case, for example, ranging algorithm A performs ranging based on the output of the stereo camera 32B, ranging algorithm B performs ranging based on the output of the ToF sensor 32C, and ranging algorithm C performs ranging based on the output of the structured-light sensor 32D.

[0056] In this way, a plurality of programs with different sensing algorithms are provided in the sensor device 21 as sensing programs for realizing the same ranging function. At least one of each sensing algorithm and the sensing program that defines each sensing algorithm is associated with a sensor used for ranging. When the sensing program is executed, the operation of the associated sensor is controlled in conjunction with that.

[0057] For example, in the sensor device 21 mounted on the transport robot 2-5, ranging is performed by selecting a sensing algorithm according to the sensing conditions. The sensing conditions are the conditions for selecting a sensing algorithm determined according to the situation of the transport robot 2-5.

[0058] For example, when the situation is suitable for the ranging algorithm A, the ranging program A is executed, and ranging is performed using the ranging algorithm A. Also, when the situation is suitable for the ranging algorithm B, the ranging program B is executed, and ranging is performed using the ranging algorithm B. When the situation is suitable for the ranging algorithm C, the ranging program C is executed, and ranging is performed using the ranging algorithm C.

[0059] Since the sensing algorithm (sensing program) corresponding to the sensing conditions is adaptively selected for ranging, ranging can be performed with the optimal sensing algorithm. The same applies when the sensing target is other than distance.

[0060] One sensing program defines one sensing algorithm. Selecting a sensing program is equivalent to selecting a sensing algorithm.

[0061] Note that adaptively selecting a sensing algorithm means selecting the sensing algorithm associated with the sensing conditions when it is detected that the conditions are met. For each assumed sensing condition corresponding to the situation, a sensing algorithm considered to be suitable is associated. The association between the sensing conditions and the sensing algorithm may be changed dynamically.

[0062] Figure 5 is a diagram showing another example of the sensing program.

[0063] In the example of Figure 5, the food recognition program A, the food recognition program B, and the food recognition program C are prepared as firmware that operates on the OS. The food recognition programs A to C are sensing programs that realize the food recognition function as an object sensing function.

[0064] For example, in the sensor device 21 mounted on the cooking robot 2-4, the recognition of food ingredients is performed by selecting a sensing algorithm according to the sensing conditions determined by the cooking process or the like.

[0065] Figure 6 is a diagram showing still another example of the sensing program.

[0066] In the example of Figure 6, a face recognition program A, a face recognition program B, and a face recognition program C are prepared as firmware that operates on the OS. The face recognition programs A to C are sensing programs that realize the face recognition function as a human sensing function.

[0067] For example, in the sensor device 21 mounted on the cooking robot 2-4, the recognition of a face is performed by selecting a sensing algorithm according to the sensing conditions determined by the cooking process or the like.

[0068] · Update of the sensing program In the program providing system of Figure 1, the update of the sensing program prepared as firmware for the sensor device 21 of each device is enabled.

[0069] Figure 7 is a diagram showing an example of the update of the sensing program.

[0070] As shown by the arrow in Figure 7, the program management server 1 provides a sensing program to each device. The program management server 1 has a DB (Data Base) of the sensing program to be provided to each device.

[0071] In the example of Figure 7, a ranging program D that performs ranging by a ranging algorithm D is provided to the mobile terminal 2-1, and a face recognition program H that performs face recognition by a face recognition algorithm H is provided to the arm robot 2-2.

[0072] In addition, a self-position estimation program J that performs self-position estimation by a self-position estimation algorithm J is provided to the moving body 2-3, and an object recognition program K that performs object recognition by an object recognition algorithm K is provided to the cooking robot 2-4. A person recognition program M that performs person recognition by a person recognition algorithm M is provided to the transport robot 2-5.

[0073] Figure 8 is a diagram showing an example of updating the sensing program.

[0074] In the sensor device 21 of each device, as shown in A of FIG. 8, it is possible to add a sensing program. In the example of A in FIG. 8, a ranging program D that performs ranging by a ranging algorithm D is added to the ranging programs A to C that perform ranging by the ranging algorithms A to C.

[0075] In the sensor device 21 of each device, in the default state, a sensing program that defines a sensing algorithm according to a general situation is prepared. Even when the sensor device 21 of each device is in a situation that cannot be handled by the pre-prepared sensing program, it is possible to add a sensing program that defines a sensing algorithm according to such a special situation, so that it is possible to handle special situations.

[0076] Also, as shown in B of FIG. 8, it is also possible to delete (uninstall) an unnecessary program. In the example of B in FIG. 8, as shown by the dashed frame, the ranging program C among the ranging programs A to C is deleted.

[0077] Figure 9 is a diagram showing another example of updating the sensing program.

[0078] As shown in FIG. 9, it is also possible to update in units of a sensing program set composed of a plurality of sensing programs. In the example of FIG. 9, a sensing program set including a ranging program D that performs ranging by a ranging algorithm D, a ranging program E that performs ranging by a ranging algorithm E, and a ranging program F that performs ranging by a ranging algorithm F is provided and added from the program management server 1.

[0079] As shown in FIG. 10, in the DB of the program management server 1, a plurality of sensing program sets that group a plurality of sensing programs according to usage conditions such as location, situation, and purpose are prepared.

[0080] In the example of FIG. 10, a sensing program set for indoor ranging and a sensing program set for outdoor ranging are prepared. These sensing program sets are sensing program sets according to location.

[0081] The sensing program set according to location is, for example, a set used in the sensor device 21 mounted on a device having a moving function. Even within the same indoor area, sensing program sets may be prepared in more detailed location units, such as a sensing program set for the kitchen and a sensing program set for the dining room.

[0082] It is also possible to prepare sensing program sets for various locations, such as a sensing program set for the sea, a sensing program set for the mountains, and a sensing program set for inside a train.

[0083] Also, in the example of FIG. 10, a sensing program set for ranging on sunny days and a sensing program set for ranging on rainy days are prepared. These sensing program sets are sensing program sets according to weather.

[0084] The sensing program set according to the weather is, for example, a set used in a sensor device 21 mounted on a device that has a moving function and may move outdoors. It is possible to prepare sensing program sets for various changing situations, such as a sensing program set for each time zone like morning, day, and night, a sensing program set for each brightness level, and a sensing program set for each temperature.

[0085] It is also possible to prepare sensing program sets for various purposes, such as a sensing program set for running, a sensing program set for baseball, a sensing program set for cooking curry, and a sensing program set for cooking salad.

[0086] The sensor device 21 of each device can add sensing programs collectively by specifying the ID of the sensing program set according to the usage conditions. An ID as identification data is set for each sensing program set. An ID as identification data is also set for each sensing program that constitutes the sensing program set.

[0087] Instead of a set of sensing programs that realize the same distance measurement function by different sensing algorithms, as shown in FIG. 11, a set of sensing programs that realize different functions may be added.

[0088] In the example of FIG. 11, a sensing program set is composed of a distance measurement program D, a face recognition program H, and an object recognition program K. The distance measurement program D is a sensing program that performs distance measurement by the distance measurement algorithm D, the face recognition program H is a sensing program that performs face recognition by the face recognition algorithm H, and the object recognition program K is a sensing program that performs object recognition by the object recognition algorithm K.

[0089] FIG. 12 is a diagram showing an example of a sensing program set.

[0090] The sensing program set shown in FIG. 12 includes an algorithm manager which is a program for controlling the adaptive selection of algorithms.

[0091] The sensor device 21 executes the algorithm manager and selects a sensing algorithm according to the sensing conditions. Information representing the type of sensing program for controlling the execution and information representing the execution order of the sensing program are set in the algorithm manager.

[0092] FIG. 13 is a diagram showing an example of the update of a sensing program.

[0093] The sensing program may be executed in each of the sensor device 21 and the controller 51 which is a host-side device, so that a predetermined function is realized. In this case, the sensing program of the controller 51 can also be updated in the same way as the sensing program of the sensor device 21. The controller 51 is a host-side data processing device such as, for example, the CPU of the mobile terminal 2-1 or the CPU of the PC installed in the arm robot 2-2.

[0094] The sensing program for updating the firmware of the sensor device 21 and the sensing program for updating the firmware of the controller 51 may be included in and provided as one sensing program set.

[0095] The provision of the sensing program and the sensing program set may be paid or free. A paid sensing program and a free sensing program may be mixed and included in one sensing program set.

[0096] When updating the sensing program performed as described above, the program management server 1 may perform authentication of the sensor device 21 based on the key information for authentication, and perform the update when it is confirmed that the device is a legitimate device. For each sensor device 21, key information for authentication is prepared as unique information.

[0097] Instead of performing authentication of the sensor device 21 using the key information for authentication at the time of updating the sensing program, it may be performed at the time of executing the sensing program.

[0098] · Provider of the sensing program FIG. 14 is a diagram showing an example of the provider of the sensing program.

[0099] As shown in FIG. 14, the sensing program provided from the program management server 1 to each device is developed, for example, by a developer who has performed user registration of a service in the program providing system. For each developer, information regarding the specifications of the sensor device 21 and development tools such as an SDK (Software Development Kit) are provided by a service provider who operates a service using the program providing system.

[0100] Each developer develops a sensing program or a sensing program set by using the SDK or the like, and uploads it from his / her own computer to the program management server 1. The uploaded sensing program and sensing program set are stored and managed in the sensing program DB.

[0101] The program management server 1 manages the usage status of each sensing program and sensing program set, such as the number of installations and the number of executions in each device. A service provider may provide a predetermined incentive such as payment of an amount according to the usage status and issuance of points to the developer.

[0102] FIG. 15 is a diagram showing an example of generation of a sensing program set.

[0103] A sensing program set may be generated by any user collecting the sensing programs developed and uploaded by each developer.

[0104] In the example of FIG. 15, among the ranging programs A to G, an indoor ranging program set is generated by collecting three sensing programs: ranging program D, ranging program E, and ranging program F.

[0105] The indoor ranging program set generated in this way is published in the program management server 1 as an installable sensing program set and is installed in a predetermined device as appropriate, similar to the sensing program set developed by the developer.

[0106] An incentive may also be provided to the user who generated the program set by collecting a plurality of sensing programs.

[0107] <Use case of sensing program> · Use case of transport robot Here, the use case of environmental sensing will be described.

[0108] When the transport robot 2-5 transports an object to be transported, in the sensor device 21 mounted on the transport robot 2-5, environmental sensing is performed by executing a sensing program. In order to move safely to the destination, detection of obstacles, measurement of the distance to the obstacles, estimation of the direction of the obstacles, estimation of the self-position, etc. are performed as environmental sensing.

[0109] FIG. 16 is a diagram showing the state of transportation by the transport robot 2-5.

[0110] FIG. 16 shows the state of the transport robot 2-5 moving in the kitchen inside the building. Cooked dishes are placed on the top plate prepared as the placement table for the object to be transported. In this example, the transport robot 2-5 is used for the purpose of serving dishes.

[0111] Based on the results of environmental sensing by the sensor device 21, the transport robot 2-5 will plan the movement path, avoid obstacles, etc., and move to the destination to serve the dishes.

[0112] FIG. 17 is a diagram showing an enlarged view of the appearance of the transport robot 2-5.

[0113] As shown in FIG. 17, the transport robot 2-5 is composed of an annular base portion 101 and a circular thin plate-shaped top plate 102 connected by a thin rod-shaped support arm 103. A plurality of tires are provided on the bottom surface side of the base portion 101. The base portion 101 functions as a moving part for realizing the movement of the transport robot 2-5.

[0114] The radial length of the base portion 101 and the radial length of the top plate 102 are substantially the same length. When the top plate 102 is substantially directly above the base portion 101, the support arm 103 is in an inclined state as shown in FIG. 17.

[0115] The support arm 103 is composed of an arm member 103-1 and an arm member 103-2. The diameter of the arm member 103-1 on the top plate 102 side is slightly smaller than the diameter of the arm member 103-2 on the base portion 101 side. By the arm member 103-1 being housed in the telescopic portion 103A inside the arm member 103-2, the length of the support arm 103 is adjusted as shown by the double-headed arrow.

[0116] The support arm 103 can adjust the angle at each of the connecting portion between the base portion 101 and the support arm 103 and the connecting portion between the top plate 102 and the support arm 103.

[0117] FIG. 18 is a diagram showing an example of the posture of the transport robot 2-5 when placing a dish.

[0118] In the example of FIG. 18, by making the support arm 103 substantially vertical and having the maximum length, the height of the top plate 102 is adjusted to be substantially the same as the height of the top plate of the cooking robot 2-4.

[0119] When the transport robot 2-5 is in such a state, the dish is placed on the top plate 102 by the cooking arm of the cooking robot 2-4. In the example of FIG. 18, the dish completed by the cooking operation of the cooking robot 2-4 is placed by the cooking arm.

[0120] As shown in FIG. 18, the cooking robot 2-4 is provided with a plurality of cooking arms for performing various cooking operations such as cutting food ingredients and cooking food ingredients. The cooking operations by the cooking arms are performed according to cooking data that defines the content and order of the cooking operations. The cooking data includes information on each cooking process until the dish is completed.

[0121] Thus, the dish served by the transport robot 2-5 is the dish made by the cooking robot 2-4. The dish made by a person may be placed on the top plate 102 by the person and served.

[0122] FIG. 19 is a plan view showing the layout of the space where the transport robot 2-5 moves.

[0123] As shown in FIG. 19, in the building where the transport robot 2-5 moves, each room such as Kitchen #1, Dining Room #2, and Party Room #3 is prepared. There is a corridor #11 between Kitchen #1 and Dining Room #2, and there is a corridor #12 between Kitchen #1 and Party Room #3.

[0124] Outside the building shown in Fig. 19 with the area indicated by the dashed line, a courtyard #21 is provided facing the dining room #2 and the party room #3. As shown by the hatching, large windows are provided in the wall of the party room #3 including the wall on the courtyard #21 side. Windows are also provided on both sides of the corridor #12.

[0125] The following describes the sensing algorithm used when the transport robot 2-5 moves in such a space to serve meals.

[0126] · Specific examples of the sensing algorithm Fig. 20 is a diagram showing an example of the sensing algorithm defined by the sensing program prepared for the sensor device 21 of the transport robot 2-5.

[0127] As shown in Fig. 20, programs defining algorithms A1 to A9 are prepared for the sensor device 21 of the transport robot 2-5. Algorithms A1 to A9 are sensing algorithms for environmental sensing used when serving meals.

[0128] In the following, for the sake of convenience of explanation, the case where environmental sensing is mainly performed using the stereo camera 32B provided in the sensor device 21 will be mainly described. Depending on the algorithm, the output of the ToF sensor 32C is used.

[0129] As shown in Fig. 20, algorithm A1 is divided into algorithm A1-1 and algorithm A1-2. Algorithm A1 is used, for example, as the default sensing algorithm.

[0130] Algorithm A1-1 is a sensing algorithm that can only be used in narrow places but has high accuracy. The responsiveness of algorithm A1-1 is slower than the reference speed. Responsiveness is the time required for environmental sensing.

[0131] Environmental sensing by algorithm A1-1 is performed by setting the shooting resolution of the stereo camera 32B higher than the reference resolution. Since environmental sensing is performed based on an image with a high resolution and a large amount of data, although the processing speed decreases and the responsiveness deteriorates, the accuracy increases.

[0132] In addition, the output of the ToF sensor 32C is also used for environmental sensing by algorithm A1-1. When performing environmental sensing by algorithm A1-1, the detection range of the ToF sensor 32C is set to be narrower than the reference width. Also, the power for driving the ToF sensor 32C is set to be stronger than the reference power.

[0133] Algorithm A1-2 can be used in a wider area than algorithm A1-1, but it is a sensing algorithm with lower accuracy than algorithm A1-1.

[0134] Environmental sensing by algorithm A1-2 is performed by setting the shooting resolution of the stereo camera 32B lower than the shooting resolution of algorithm A1-1. Since environmental sensing is performed based on an image with a slightly lower resolution and a smaller amount of data, compared with algorithm A1-1, the processing speed is faster and the responsiveness is better, but the accuracy deteriorates.

[0135] In addition, environmental sensing by algorithm A1-2 is performed by setting the baseline length of the stereo camera 32B longer than the baseline length in algorithm A1-1. Distance measurement using the stereo camera 32B is performed by utilizing the parallax between the two cameras. By increasing the baseline length, which is represented as the distance between the two cameras, it becomes possible to measure the distance to an object far away.

[0136] Algorithm A2 is a sensing algorithm that can be used in a wide area and has sufficient responsiveness to capture moving objects.

[0137] Environmental sensing by Algorithm A2 is performed by setting the baseline length of the stereo camera 32B longer than the baseline length in Algorithm A1-2.

[0138] Algorithm A3 is a sensing algorithm that is robust against noise such as rain.

[0139] Environmental sensing by Algorithm A3 is performed by performing image processing for noise removal on the distance image captured by the stereo camera 32B and then calculating the distance based on the distance image after noise removal. Specifically, since image processing for removing raindrops as noise is added, although the processing speed decreases and the responsiveness deteriorates, it becomes possible to cope with rain.

[0140] Known techniques are used for noise removal. Regarding the techniques of noise removal, they are disclosed in, for example, "https: / / digibibo.com / blog-entry-3422.html" and "http: / / www.robot.t.u-tokyo.ac.jp / ~yamashita / paper / A / A025Final.pdf".

[0141] Algorithm A4 is a sensing algorithm that is robust against direct sunlight.

[0142] Environmental sensing by Algorithm A4 is performed by adjusting the shooting parameters so as to increase the shutter speed of the stereo camera 32B and decrease the sensitivity.

[0143] Algorithm A5 is a sensing algorithm that can adapt to dim places.

[0144] Environmental sensing by Algorithm A5 is performed by adjusting the shooting parameters so as to decrease the shutter speed and increase the sensitivity. Although it becomes difficult to measure the distance to moving objects, accuracy is ensured by decreasing the moving speed of the transport robot 2-5.

[0145] It is also possible to perform environmental sensing by the algorithm A5 using other sensors. For example, other sensors such as the ToF sensor 32C that does not rely on visible light are used. By using the ToF sensor 32C, it becomes difficult to measure distances to far away places, but it becomes possible to measure distances even in dark places. Regarding the inability to measure distances to far away places, the risk of collision can be avoided by reducing the moving speed of the transport robot 2-5.

[0146] The algorithm A6 is a sensing algorithm that is strong against shadows.

[0147] Environmental sensing by the algorithm A6 is performed by adjusting the shooting parameters so as to expand the dynamic range of the brightness of the stereo camera 32B. By expanding the dynamic range of the brightness, it becomes possible to measure distances to bright places and dark places simultaneously.

[0148] The algorithm A7 is a sensing algorithm that is strong at measuring distances to reflectors such as mirrors.

[0149] Environmental sensing by the algorithm A7 is performed using sensors that do not rely on visible light. For example, a sensor that measures distances using sound waves is used.

[0150] The algorithm A8 is a sensing algorithm that can detect transparent objects.

[0151] Environmental sensing by the algorithm A8 is performed by setting the shooting resolution of the stereo camera 32B higher than the reference resolution.

[0152] The algorithm A9 is a sensing algorithm that can handle completely dark places.

[0153] Environmental sensing by the algorithm A9 is performed by measuring distances using active sensors such as the ToF sensor 32C and the LiDAR 32E.

[0154] Referring to FIGS. 21 to 25, the environmental sensing conditions, which are the conditions for selecting the above sensing algorithm, will be described.

[0155] As shown in the balloon in FIG. 21, Kitchen #1 is a space with characteristics such as many objects and narrow, many stationary objects, and many dangerous objects such as knives and glasses. There are few moving objects, but due to its narrowness, there is a high possibility of colliding with obstacles. For the transport robot 2-5 to move safely, even if the responsiveness is poor, the accuracy needs to be guaranteed.

[0156] The sensing algorithm suitable for environmental sensing in Kitchen #1 having such characteristics is Algorithm A1-1. When the transport robot 2-5 is in Kitchen #1, as shown at the tip of the white arrow, Algorithm A1-1 is used for environmental sensing.

[0157] When the transport robot 2-5 detects the situation that it itself is in Kitchen #1, according to the environmental sensing condition of performing environmental sensing in Kitchen #1, it will select and execute the sensing program that defines Algorithm A1-1.

[0158] The environmental sensing condition is the condition for selecting the sensing algorithm, that is, the condition for selecting the sensing program.

[0159] As shown in the balloon in FIG. 22, Corridor #11 is a space with characteristics such as being sometimes completely dark for power saving and having reflectors such as mirrors placed. There are no moving objects, but due to being a completely dark place, there is a high possibility of colliding with the wall or misidentifying the transport robot 2-5 itself reflected in the mirror as an obstacle. For the transport robot 2-5 to move safely, it needs to be resistant to the reflection of reflectors such as mirrors and be able to handle completely dark places.

[0160] The sensing algorithms suitable for performing environmental sensing in the corridor #11 having such characteristics are algorithms A7 and A9. When the transport robots 2-5 are in the corridor #11, as shown at the tip of the white arrow, algorithms A7 and A9 are used for environmental sensing.

[0161] When the transport robots 2-5 detect the situation that the transport robots 2-5 themselves are in the corridor #11, they will select and execute the sensing program that defines algorithms A7 and A9 according to the environmental sensing condition of performing environmental sensing in the corridor #11.

[0162] For example, environmental sensing using algorithm A7 and environmental sensing using algorithm A9 are executed in parallel or alternately. In this way, when multiple sensing algorithms are set as sensing algorithms suitable for the environmental sensing conditions, environmental sensing is performed by switching between the multiple sensing algorithms.

[0163] As shown in the balloon of FIG. 23, the party room #3 is a space with characteristics such as being visible far away, wide, having few objects, having many people, and having large windows. Although it is wide, due to the large number of people and moving objects, there is a high possibility of colliding with obstacles or colliding without being able to recognize the large windows. In addition, direct sunlight entering through the large windows is highly likely to cause misrecognition. In order for the transport robots 2-5 to move safely, it is necessary to be usable in a wide area, be resistant to direct sunlight, and be able to detect transparent obstacles.

[0164] The sensing algorithms suitable for performing environmental sensing in the party room #3 having such characteristics are algorithms A2, A4, and A8. When the transport robots 2-5 are in the party room #3, as shown at the tip of the white arrow, algorithms A2, A4, and A8 are used for environmental sensing.

[0165] When the transport robot 2-5 detects the situation that it is in the party room #3, it will select and execute a sensing program that defines algorithms A2, A4, and A8 according to the environmental sensing condition of performing environmental sensing in the party room #3.

[0166] As shown in the balloon in Figure 24, the garden #21 is a space with characteristics such as being exposed to direct sunlight, sometimes getting dim, sometimes having rain, and sometimes having wild animals such as birds flying in. There is a high possibility of misrecognition due to direct sunlight, rain, or getting dim. Also, there is a high possibility of colliding with wild animals that fly in suddenly. In order for the transport robot 2-5 to move safely, it is necessary to be able to handle rain, direct sunlight, strong shadows, and dim places.

[0167] Sensing algorithms suitable for performing environmental sensing in the garden #21 with such characteristics are algorithms A3, A4, A5, and A6. When the transport robot 2-5 is in the garden #21, as shown at the tip of the white arrow, algorithms A3, A4, A5, and A6 are used for environmental sensing.

[0168] When the transport robot 2-5 detects the situation that it is in the garden #21, it will select and execute a sensing program that defines algorithms A3, A4, A5, and A6 according to the environmental sensing condition of performing environmental sensing in the garden #21.

[0169] Figure 25 is a diagram showing an example of the correspondence between the food serving situation and the sensing algorithm.

[0170] For example, the sensing algorithms that can be used when transporting an object to be transported from Kitchen #1 to another space are algorithms A1-1, A1-2, A2, A4, A7, A8, and A9. From among the algorithms A1-1, A1-2, A2, A4, A7, A8, and A9, a sensing algorithm corresponding to the environmental sensing condition of performing environmental sensing in each use case is selected.

[0171] When moving from Kitchen #1 to Party Room #3, the transition of the sensing algorithm is from algorithm A1-1 to algorithm A2.

[0172] When moving from Kitchen #1 to Garden #21, the transition of the sensing algorithm is from algorithm A1-1 to algorithm A4.

[0173] When moving from Kitchen #1 to Party Room #3 through the glassed-in corridor #12, the transition of the sensing algorithm is from algorithm A1-1 to algorithm A8, and further from algorithm A8 to algorithm A2.

[0174] When moving from Kitchen #1 to Party Room #3 through the corridor with a mirror, the transition of the sensing algorithm is from algorithm A1-1 to algorithm A7, and further from algorithm A7 to algorithm A2.

[0175] When moving from Kitchen #1 to Dining Room #2 through the dark corridor #11 without electricity, the transition of the sensing algorithm is from algorithm A1-1 to algorithm A9, and further from algorithm A9 to algorithm A2.

[0176] The sensing algorithms that can be used when serving meals in courtyard #21 are algorithms A3, A4, A5, and A6. Similarly, in the situation of serving meals in courtyard #21, a sensing algorithm corresponding to the environmental sensing condition of performing environmental sensing in each use case is selected.

[0177] In this way, in the sensor device 21, various situations of the transport robots 2-5, such as location and actions, are detected, and a sensing algorithm is selected according to the environmental sensing condition of performing environmental sensing in such situations.

[0178] <Configuration of Transport Robot> FIG. 26 is a block diagram showing a configuration example of the hardware of the transport robots 2-5.

[0179] The transport robots 2-5 are configured by connecting a top plate lifting drive unit 122, a tire drive unit 123, a sensor group 124, and a communication unit 125 to a controller 121. A sensor device 21 is also connected to the controller 121.

[0180] The controller 121 has a CPU, ROM, RAM, flash memory, etc. The controller 121 executes a predetermined program and controls the overall operation of the transport robots 2-5, including the sensor device 21. The controller 121 corresponds to the host-side controller 51 (FIG. 13).

[0181] The top plate lifting drive unit 122 is composed of motors provided at the connecting parts of the base part 101 and the support arm 103, the connecting parts of the top plate 102 and the support arm 103, etc. The top plate lifting drive unit 122 drives each connecting part.

[0182] Also, the top plate lifting drive unit 122 is composed of rails and motors provided inside the support arm 103. The top plate lifting drive unit 122 expands and contracts the support arm 103.

[0183] The tire drive unit 123 is composed of a motor that drives a tire provided on the bottom surface of the base unit 101.

[0184] The sensor group 124 is composed of various sensors such as a positioning sensor, a gyro sensor, an acceleration sensor, a temperature sensor, and an illuminance sensor. Sensor data representing the detection results by the sensor group 124 is output to the controller 121.

[0185] The communication unit 125 is a wireless communication module such as a wireless LAN module and a mobile communication module compatible with LTE (Long Term Evolution). The communication unit 125 communicates with an external device such as a program management server 1.

[0186] FIG. 27 is a block diagram showing a functional configuration example of the transport robot 2-5.

[0187] Among the functional units shown in FIG. 27, at least a part is realized by a CPU constituting the controller 121 and a CPU constituting the controller 31 of the sensor device 21 executing a predetermined program.

[0188] In the controller 121, a path information acquisition unit 151, a positioning control unit 152, a movement control unit 153, an attitude control unit 155, an environmental data acquisition unit 156, and a surrounding state recognition unit 157 are realized.

[0189] On the other hand, in the controller 31 of the sensor device 21, a situation detection unit 201 and a sensing control unit 202 are realized. The sensor device 21 is a data processing device that controls a sensing algorithm.

[0190] The path information acquisition unit 151 of the controller 121 controls the communication unit 125 and receives information on the destination and the movement path transmitted from a control device (not shown). The information received by the path information acquisition unit 151 is output to the movement control unit 153.

[0191] At the timing when the object to be transported is prepared, etc., the movement route may be planned by the route information acquisition unit 151 based on the destination and the current position of the transport robot 2-5.

[0192] In this case, the route information acquisition unit 151 functions as an operation plan setting unit that plans the operation of the transport robot 2-5 and sets the operation plan.

[0193] The positioning control unit 152 detects the current position of the transport robot 2-5. For example, the positioning control unit 152 generates a map of the space where the transport robot 2-5 is installed based on the detection results of the distance sensors that make up the sensor device 21. The sensor data output from the sensor device 21 is acquired by the environment data acquisition unit 156 and supplied to the positioning control unit 152.

[0194] The positioning control unit 152 detects the current position by identifying its own position in the generated map. The information on the current position detected by the positioning control unit 152 is output to the movement control unit 153. The detection of the current position by the positioning control unit 152 may be performed based on the output of the positioning sensors that make up the sensor group 124.

[0195] The movement control unit 153 controls the movement of the transport robot 2-5 based on the information supplied from the route information acquisition unit 151 and the current position detected by the positioning control unit 152 by controlling the tire drive unit 123.

[0196] In addition, when information about surrounding obstacles is supplied from the surrounding state recognition unit 157, the movement control unit 153 controls the movement to avoid the obstacles. The obstacles include various moving objects and stationary objects such as people, furniture, and household appliances. In this way, the movement control unit 153 controls the movement of the transport robot 2-5 accompanying the transport of the object to be transported based on the results of environmental sensing by the sensor device 21.

[0197] The attitude control unit 155 controls the top plate lifting drive unit 122 to control the attitude of the transport robot 2-5. Further, the attitude control unit 155 controls the attitude of the transport robot 2-5 during movement so as to keep the top plate 102 horizontal in conjunction with the control by the movement control unit 153.

[0198] The attitude control unit 155 controls the attitude of the transport robot 2-5 according to the surrounding state recognized by the surrounding state recognition unit 157. For example, the attitude control unit 155 controls the attitude of the transport robot 2-5 so as to bring the height of the top plate 102 closer to the height of the top plate of the cooking robot 2-4 or the top plate of the dining table recognized by the surrounding state recognition unit 157.

[0199] The environment data acquisition unit 156 controls the sensor device 21 to perform environmental sensing and acquires sensor data representing the result of the environmental sensing. The sensor data acquired by the environment data acquisition unit 156 is supplied to the positioning control unit 152 and the surrounding state recognition unit 157.

[0200] The surrounding state recognition unit 157 recognizes the surrounding state based on the sensor data representing the result of the environmental sensing supplied from the environment data acquisition unit 156. The information representing the recognition result by the surrounding state recognition unit 157 is supplied to the movement control unit 153 and the attitude control unit 155.

[0201] When the sensor device 21 performs environmental sensing such as detection of an obstacle, measurement of the distance to the obstacle, estimation of the direction of the obstacle, and estimation of the self-position, the surrounding state recognition unit 157 outputs information regarding the obstacle as information representing the recognition result of the surrounding state.

[0202] Detection of an obstacle, measurement of the distance to the obstacle, estimation of the direction of the obstacle, estimation of the self-position, etc. may be performed by the surrounding state recognition unit 157 based on the result of the environmental sensing by the sensor device 21. In this case, the sensor data used for each process performed by the surrounding state recognition unit 157 will be detected by the environmental sensing by the sensor device 21.

[0203] Thus, the content of the processing performed as environmental sensing in the sensor device 21 is arbitrary. That is, the raw data detected by the sensors provided in the sensor device 21 may be directly supplied to the controller 121 as sensor data, or the raw data may be processed and analyzed on the sensor device 21 side, and the results of the processing and analysis may be supplied to the controller 121 as sensor data.

[0204] The situation detection unit 201 on the sensor device 21 side detects the situation of the transport robot 2-5. The situation of the transport robot 2-5 is detected based on, for example, the sensor data output by the sensors constituting the sensor group 124, or the sensor data output by the sensors provided in the sensor device 21.

[0205] The situation of the transport robot 2-5 includes, for example, the operation of the transport robot 2-5 such as what operation it is performing, the location where the transport robot 2-5 is located, the weather, temperature, humidity, and brightness at the location where the transport robot 2-5 is located. In addition, the situation of the transport robot 2-5 also includes external situations such as the situation of the person with whom the transport robot 2-5 is communicating and the situation of obstacles around the transport robot 2-5.

[0206] The situation detection unit 201 outputs information representing such a situation of the transport robot 2-5 to the sensing control unit 202.

[0207] The sensing control unit 202 selects a sensing algorithm according to the environmental sensing condition of performing environmental sensing in the situation detected by the situation detection unit 201, and executes a sensing program that defines the selected sensing algorithm.

[0208] For example, for each environmental sensing condition, a sensing algorithm or a sensing program is associated. The sensing control unit 202 selects a sensing algorithm or a sensing program according to the environmental sensing condition, using an ID as identification data. A sensing program set may be selected according to the environmental sensing condition.

[0209] By executing the sensing program, the sensing control unit 202 drives each sensor provided in the sensor device 21, and outputs sensor data to the controller 121 based on the output of each sensor.

[0210] <Operation of the transport robot> Here, the operation of the transport robot 2-5 having the above configuration will be described.

[0211] · Basic processing First, with reference to the flowchart of FIG. 28, the basic sensing process of the transport robot 2-5 will be described.

[0212] In step S1, the situation detection unit 201 of the sensor device 21 senses the environment using a default sensing algorithm.

[0213] In step S2, the situation detection unit 201 detects the situation of the transport robot 2-5 based on the sensor data from the sensor group 124 or the sensor data from the sensor device 21.

[0214] In step S3, the sensing control unit 202 determines whether to switch the sensing algorithm. For example, when the situation detection unit 201 detects that the situation of the transport robot 2-5 has changed, it is determined to switch the sensing algorithm.

[0215] When it is determined in step S3 that the sensing algorithm is switched, in step S4, the sensing control unit 202 selects a sensing algorithm according to the sensing condition of performing environmental sensing in the changed situation, and executes a sensing program that defines the selected sensing algorithm.

[0216] Thereafter, the process returns to step S2, the process of detecting the situation of the transport robot 2-5 is performed, and the above-described process is repeated. Similarly, when it is determined in step S3 that the sensing algorithm is not switched, the process returns to step S2 and the above-described process is repeated.

[0217] ·Specific processing Next, with reference to the flowchart of FIG. 29, the food delivery process of the transport robot 2-5 will be described.

[0218] In step S11, the situation detection unit 201 detects the location of the transport robot 2-5 based on the sensor data from the sensor group 124 or the sensor data from the sensor device 21.

[0219] In step S12, the sensing control unit 202 determines whether the location of the transport robot 2-5 is indoors based on the detection result by the situation detection unit 201.

[0220] If it is determined in step S12 that the location of the transport robot 2-5 is indoors, in step S13, the sensing control unit 202 performs environmental sensing using the basic algorithm for indoors.

[0221] The basic algorithm for indoors is a sensing algorithm that adjusts the shooting parameters of the stereo camera 32B, such as the shutter speed and sensitivity, according to the intensity of the ambient light, and performs environmental sensing. The shutter speed is set to a slower speed than the standard, and the sensitivity is set to a higher level than the standard.

[0222] In step S14, the sensing control unit 202 performs indoor processing. In the indoor processing, a sensing algorithm is selected according to the indoor situation, and environmental sensing is performed. The sensing algorithm used for environmental sensing is appropriately switched from the basic indoor algorithm to other sensing algorithms. Details of the indoor processing will be described later with reference to the flowchart of FIG. 30.

[0223] On the other hand, if it is determined in step S12 that the location of the transport robot 2-5 is not indoors, that is, outdoors, in step S15, the sensing control unit 202 performs environmental sensing using the basic outdoor algorithm.

[0224] The basic outdoor algorithm is a sensing algorithm that adjusts shooting parameters of the stereo camera 32B such as shutter speed and sensitivity according to the intensity of ambient light and performs environmental sensing. The shutter speed is set to a faster speed than the standard, and the sensitivity is set to a lower level than the standard.

[0225] In step S16, the sensing control unit 202 performs outdoor processing. In the outdoor processing, a sensing algorithm is selected according to the outdoor situation, and environmental sensing is performed. The sensing algorithm used for environmental sensing is appropriately switched from the basic outdoor algorithm to other sensing algorithms. Details of the outdoor processing will be described later with reference to the flowchart of FIG. 31.

[0226] After the indoor processing is performed in step S14, or after the outdoor processing is performed in step S16, the process returns to step S11, and the subsequent processing is repeated.

[0227] Here, with reference to the flowchart of FIG. 30, the indoor processing performed in step S14 of FIG. 29 will be described.

[0228] In step S21, the sensing control unit 202 determines whether the location of the transport robot 2-5 is Kitchen #1.

[0229] If it is determined in step S21 that the location of the transport robot 2-5 is Kitchen #1, then in step S22, the sensing control unit 202 selects algorithm A1-1 according to the environmental sensing condition of performing environmental sensing in Kitchen #1 and performs environmental sensing.

[0230] If it is determined in step S21 that the location of the transport robot 2-5 is not Kitchen #1, then in step S23, the sensing control unit 202 determines whether the location of the transport robot 2-5 is Dinning Room #2.

[0231] If it is determined in step S23 that the location of the transport robot 2-5 is Dinning Room #2, then in step S24, the sensing control unit 202 selects algorithm A1-2 according to the environmental sensing condition of performing environmental sensing in Dinning Room #2 and performs environmental sensing.

[0232] If it is determined in step S23 that the location of the transport robot 2-5 is not Dinning Room #2, then in step S25, the sensing control unit 202 determines whether the location of the transport robot 2-5 is Party Room #3.

[0233] If it is determined in step S25 that the location of the transport robot 2-5 is Party Room #3, then in step S26, the sensing control unit 202 selects algorithm A2 according to the environmental sensing condition of performing environmental sensing in Party Room #3 and performs environmental sensing.

[0234] If it is determined in step S25 that the location of the transport robot 2-5 is not Party Room #3, then in step S27, the sensing control unit 202 determines whether the location of the transport robot 2-5 is Corridor #11.

[0235] When it is determined in step S27 that the location of the transport robot 2-5 is corridor #11, in step S28, the sensing control unit 202 selects algorithm A5 and performs environment sensing according to the environment sensing condition of performing environment sensing in corridor #11.

[0236] After environment sensing is performed using any one of algorithms A1-1, A1-2, A2, or algorithm A5 according to the location of the transport robot 2-5, or when it is determined in step S27 that the location of the transport robot 2-5 is not corridor #11, the process proceeds to step S29.

[0237] In step S29, the sensing control unit 202 determines whether there is a transparent object near the transport robot 2-5. The determination of whether there is a transparent object is made based on the situation detection result by the situation detection unit 201.

[0238] When it is determined in step S29 that there is a transparent object near the transport robot 2-5, in step S30, the sensing control unit 202 selects algorithm A8 and performs environment sensing according to the environment sensing condition of performing environment sensing at the location where there is a transparent obstacle.

[0239] After environment sensing is performed in step S30, or when it is determined in step S29 that there is no transparent object near the transport robot 2-5, the process proceeds to step S31.

[0240] In step S31, the sensing control unit 202 determines whether there is an object that reflects near the transport robot 2-5. The determination of whether there is an object that reflects is made based on the situation detection result by the situation detection unit 201.

[0241] If it is determined in step S31 that there is an object reflecting near the transport robot 2-5, in step S32, the sensing control unit 202 selects algorithm A7 and performs environment sensing according to the environment sensing condition of performing environment sensing at the location where the reflector is located.

[0242] After the environment sensing is performed in step S32, or if it is determined in step S31 that there is no object reflecting near the transport robot 2-5, the process proceeds to step S33.

[0243] In step S33, the sensing control unit 202 determines whether the brightness of the location of the transport robot 2-5 is sufficient. The determination of whether the brightness is sufficient is made based on the situation detection result by the situation detection unit 201.

[0244] If it is determined in step S33 that the brightness of the location of the transport robot 2-5 is not sufficient, in step S34, the sensing control unit 202 selects algorithm A9 and performs environment sensing according to the environment sensing condition of performing environment sensing in a completely dark place.

[0245] After the environment sensing is performed in step S34, or if it is determined in step S33 that the brightness of the location of the transport robot 2-5 is sufficient, the process proceeds to step S35.

[0246] In step S35, the sensing control unit 202 determines whether the location of the transport robot 2-5 has changed. The determination of whether the location of the transport robot 2-5 has changed is made based on the situation detection result by the situation detection unit 201.

[0247] If it is determined in step S35 that the location of the transport robot 2-5 has not changed, the process returns to step S29 and the above-described process is repeated.

[0248] On the other hand, if it is determined in step S35 that the location of the transport robot 2-5 has changed, the process returns to step S14 in FIG. 29, and subsequent processing is performed.

[0249] Next, referring to the flowchart of FIG. 31, the outdoor processing performed in step S16 of FIG. 29 will be described.

[0250] In step S51, the sensing control unit 202 determines whether the weather at the location of the transport robot 2-5 is clear. The determination of whether the weather is clear is made based on the situation detection result by the situation detection unit 201.

[0251] If it is determined in step S51 that the weather at the location of the transport robot 2-5 is clear, then in step S52, the sensing control unit 202 determines whether it is a place where shadows are likely to occur.

[0252] If it is determined in step S52 that it is a place where shadows are likely to occur, then in step S53, the sensing control unit 202 selects algorithm A6 and performs environmental sensing according to the environmental sensing condition of performing environmental sensing in a place where shadows are likely to occur.

[0253] If it is determined in step S52 that it is not a place where shadows are likely to occur, then in step S54, the sensing control unit 202 selects algorithm A4 and performs environmental sensing according to the environmental sensing condition of performing environmental sensing under direct sunlight.

[0254] On the other hand, if it is determined in step S51 that the weather is not clear, the process proceeds to step S55.

[0255] In step S55, the sensing control unit 202 determines whether it is raining. The determination of whether it is raining is made based on the situation detection result by the situation detection unit 201.

[0256] If it is determined in step S55 that it is raining, in step S56, the sensing control unit 202 performs environment sensing by selecting algorithm A3 according to the environment sensing condition of performing environment sensing at the place where it is raining.

[0257] If it is determined in step S55 that it is not raining, in step S57, the sensing control unit 202 performs environment sensing by selecting algorithm A5 according to the environment sensing condition of performing environment sensing in a dim place.

[0258] After environment sensing is performed using any one of algorithms A3 to A6, in step S58, the sensing control unit 202 determines whether the location of the transport robot 2-5 has changed.

[0259] If it is determined in step S58 that the location of the transport robot 2-5 has not changed, in step S59, the sensing control unit 202 determines whether the weather has changed.

[0260] If it is determined in step S59 that the weather has changed, the process returns to step S51, and the above-described process is repeated.

[0261] If it is determined in step S59 that the weather has not changed, in step S60, the sensing control unit 202 performs environment sensing while keeping the selected sensing algorithm. Then, the process returns to step S58, and the above-described process is repeated.

[0262] If it is determined in step S58 that the location has changed, the process returns to step S16 in FIG. 29, and the subsequent processes are performed. Through the above processes, the transport robot 2-5 can adaptively select a sensing algorithm according to the situation and perform highly accurate environment sensing.

[0263] When it is time to perform environmental sensing using a certain sensing algorithm and the sensing program that defines the sensing algorithm is not prepared, access may be made to the program management server 1 so that the sensing program is obtained from the program management server 1.

[0264] Although the processing performed in the sensor device 21 mounted on the transport robot 2-5 has been described, processing beyond switching the sensing algorithm to perform environmental sensing is performed in each device equipped with the sensor device 21, such as the mobile terminal 2-1.

[0265] <Modification Example> · Example of the case where the selection of the sensing algorithm is performed from the outside Although the selection of the sensing algorithm according to the environmental sensing conditions is performed within the sensor device 21, it may be performed by a device external to the device equipped with the sensor device 21.

[0266] FIG. 32 is a diagram showing an example of the control of the sensing algorithm.

[0267] In the example of FIG. 32, the selection of the sensing algorithm according to the environmental sensing conditions is performed by the program management server 1 which is an external device. In this case, the configuration of the controller 31 in FIG. 27 is realized in the program management server 1. The program management server 1 is a data processing device that controls the sensing program executed by the sensor device 21 mounted on the transport robot 2-5.

[0268] As indicated by the arrow #1, sensor data used for situation detection is transmitted from the transport robot 2-5 to the program management server 1, and a sensing program is requested.

[0269] In the situation detection unit 201 of the program management server 1, the situation of the transport robot 2-5 is detected based on the sensor data transmitted from the transport robot 2-5. Further, the environmental sensing conditions according to the situation of the transport robot 2-5 are determined by the sensing control unit 202, and a sensing algorithm is selected.

[0270] As shown by the arrow #2, the sensing control unit 202 of the program management server 1 transmits a sensing program that defines a sensing algorithm according to the environmental sensing conditions to the sensor device 21 mounted on the transport robot 2-5 and causes it to execute. In this case, the sensing control unit 202 of the program management server 1 functions as a data processing unit that transmits a sensing program that defines a sensing algorithm according to the environmental sensing conditions to the transport robot 2-5.

[0271] In this way, the control of the sensing algorithm may be performed by a device external to the sensor device 21. For example, it is also possible to use the controller 121 of the transport robot 2-5 on which the sensor device 21 is mounted as an external device and perform the control of the sensing algorithm by the controller 121.

[0272] A sensing program that defines a sensing algorithm according to the environmental sensing conditions may be executed by an external device such as the program management server 1 or the controller 121, and information representing the execution result may be transmitted to the sensor device 21.

[0273] FIG. 33 is a block diagram showing a configuration example of the hardware of a computer that realizes the program management server 1.

[0274] A CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0275] The bus 1004 is further connected to an input / output interface 1005. The input / output interface 1005 is connected to an input unit 1006 composed of a keyboard, a mouse, etc., and an output unit 1007 composed of a display, a speaker, etc. Further, the input / output interface 1005 is connected to a storage unit 1008 composed of a hard disk, a non-volatile memory, etc., a communication unit 1009 composed of a network interface, etc., and a drive 1010 for driving a removable medium 1011.

[0276] The control of the sensing algorithm as described above is realized by the CPU 1001 executing a predetermined program.

[0277] · Example of program The above-described series of processes can be executed by hardware or by software. When the series of processes are executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware, or a general-purpose personal computer, etc.

[0278] The program to be installed is recorded and provided on a removable medium 1011 shown in FIG. 33 composed of an optical disk (CD-ROM (Compact Disc - Read Only Memory), DVD (Digital Versatile Disc), etc.), a semiconductor memory, etc. Further, it may be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital broadcasting. The program can be installed in advance in the ROM 1002 or the storage unit 1008.

[0279] Note that the program executed by the computer may be a program in which processing is performed in time series in accordance with the order described in this specification, or a program in which processing is performed in parallel or at a necessary timing such as when a call is made.

[0280] In addition, in this specification, the term "system" means a collection of a plurality of components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device in which a plurality of modules are housed in one housing are both systems.

[0281] The effects described in this specification are merely illustrative and not limiting, and there may be other effects.

[0282] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

[0283] For example, the present technology can adopt a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.

[0284] In addition, each step described in the above flowchart can be executed by one device or can be shared and executed by a plurality of devices.

[0285] Furthermore, when a single step includes a plurality of processes, the plurality of processes included in that single step can be executed by one device or can be shared and executed by a plurality of devices.

Description of Reference Numerals

[0286] 1 Program management server, 2-1 Mobile terminal, 2-2 Arm robot, 2-3 Mobile body, 2-4 Cooking robot, 2-5 Transport robot, 21 Sensor device, 31 Controller, 32 Sensor group, 121 Controller, 124 Sensor group, 201 Situation detection unit, 202 Sensing control unit

Claims

1. An environment sensing algorithm for sensing an environment based on sensor data output from a sensor implemented in a mobile body having a movable operation unit with respect to the mobile body main body is determined according to environment sensing conditions, and the determined environment sensing algorithm is selected from an environment sensing program set that is a combination of a plurality of the environment sensing programs and includes a combination of information indicating the type of the environment sensing algorithm and information indicating the execution order of the environment sensing programs, and is executed, to control the movement of the mobile body and the operation of the operation unit Data processing device.

2. The environment sensing program defined by the environment sensing algorithm is acquired via a network The data processing device according to claim 1.

3. The environment sensing program set is acquired via a network The data processing device according to claim 1.

4. The environment sensing program set is selected using identification data for identifying the environment sensing program set The data processing device according to claim 3.

5. The environment sensing algorithms defined in the plurality of the environment sensing programs are algorithms applied to sensor data output by setting different parameters for the same sensor The data processing device according to claim 1.

6. The environment sensing algorithms defined in the plurality of the environment sensing programs are algorithms applied to sensor data output by setting the same parameters for the same sensor The data processing device according to claim 1.

7. The environment sensing algorithms defined in the plurality of the environment sensing programs are algorithms applied to sensor data output from different sensors The data processing device according to claim 1.

8. At least any one of the environment sensing program and the environment sensing algorithm defined in the environment sensing program is associated with a sensor, and in conjunction with selecting and executing the environment sensing program, the operations of a plurality of sensors are controlled The data processing device according to claim 7.

9. Based on the execution result of the environmental sensing program, control the state of movement by the moving part during the transportation of the object to be transported The data processing device according to claim 2.

10. The operation part includes a top plate on which the object to be transported is placed, and a telescopic support part that supports the top plate. The moving part is connected to the support part. Based on the execution result of the environmental sensing program, control the posture state including the state of the top plate and the state of the support part, and the state of movement by the moving part. The data processing device according to claim 9.

11. The top plate places the cooking arm of the cooking system that is driven according to the cooking process or the object to be transported placed by a person. The data processing device according to claim 10.

12. A data processing device Determine an environmental sensing algorithm for sensing the environment based on sensor data output from a sensor mounted on a moving body having a movable operation part with respect to the moving body main body according to environmental sensing conditions, Select and execute the determined environmental sensing program defined by the environmental sensing algorithm from an environmental sensing program set including a combination of a plurality of the environmental sensing programs, which is a combination of information representing the type of the environmental sensing algorithm and information representing the execution order of the environmental sensing program, Control the movement of the moving body and the operation of the operation part Data processing method.

13. Determine an environmental sensing algorithm for sensing the environment based on sensor data output from a sensor mounted on a moving body having a movable operation part with respect to the moving body main body according to environmental sensing conditions, Select the determined environmental sensing program defined by the environmental sensing algorithm from an environmental sensing program set including a combination of a plurality of the environmental sensing programs, which is a combination of information representing the type of the environmental sensing algorithm and information representing the execution order of the environmental sensing program, and transmit it to the moving body. Data processing device.

14. In response to a request from the moving body, transmit the environmental sensing program defined by the environmental sensing algorithm. The data processing device according to claim 13.

15. A data processing device An environmental sensing algorithm for sensing the environment based on sensor data output from a sensor implemented in a mobile body having a movable operation unit with respect to the mobile body main body is determined according to environmental sensing conditions. The determined environmental sensing algorithm is a prescribed environmental sensing program, which is a combination of a plurality of the environmental sensing programs and includes a combination of information representing the type of the environmental sensing algorithm and information representing the execution order of the environmental sensing programs. The environmental sensing program is selected from a set of environmental sensing programs and transmitted to the mobile body. Data processing method.

16. A sensor that outputs sensor data representing a sensing result, A sensing control unit that adaptively selects and executes an environmental sensing program in which an environmental sensing algorithm for sensing the environment based on the sensor data output from the sensor is prescribed, according to environmental sensing conditions. An operation plan setting unit that sets an operation plan based on the execution result of the environmental sensing program by the sensing control unit. An operation unit that operates according to the operation plan set by the operation plan setting unit and the operation unit is movable with respect to the mobile body main body Mobile body.

Images