Control system and behavior generation method

Abstract

To make an autonomous system perform an appropriate action considering uncertainty of a peripheral environment.SOLUTION: A control system for generating an action for controlling a control object device comprises: a reception unit which receives sensor data obtained by observing a state of a peripheral environment of the control object device; a self-recognition unit which derives a self-recognition block defining a self-range from the sensor data by using a self-recognition prediction model that predicts the self-range being the range having the prediction property and the action property by the control object device; a target action prediction unit which derives a target action from the sensor data by using a target action prediction model that predicts the target action of the control object device; and a switching unit which selects the self-recognition block or the target action in order to generate the action of the control object device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control system, and more particularly to a behavior generation method for generating a behavior for controlling a controlled device. [Background technology]

[0002] Autonomous systems that coexist with humans are expected to be able to act in situations where the uncertainty of the environment (including people) around the system cannot be resolved. For example, a robot may be required to perform picking tasks near a person it has met for the first time, whose behavior is difficult to predict.

[0003] The following prior art exists as background art in this technical field: Patent Document 1 (JP 2009-131940 A) describes a mobile device that is equipped with a control device, and whose operation is controlled by the control device to move autonomously according to a target trajectory that represents a change in a target position defined in a two-dimensional model space, wherein the control device is equipped with a first processing unit, a second processing unit, and a third processing unit, and the first processing unit recognizes a passable area of ​​the mobile device as an element passing area in the model space, and the mobile device and the trajectory that represents a change in the position of the mobile device are respectively defined as a first spatial element and a first position element. recognizes the object and the trajectory representing the change in position of the object as a first spatial element and a second trajectory representing the change in the second position, respectively, and recognizes the second spatial element continuously or intermittently expanded according to the change in the second position as a second expanded spatial element, and the second processing unit determines, based on the recognition result by the first processing unit, whether a first safety condition indicating that there is a low possibility of contact between the first spatial element and the second spatial element in the element passage area is satisfied. The third processing unit searches for a first target trajectory that can prevent the first spatial element from coming into contact with the second extended spatial element in the element passage area based on the recognition result by the first processing unit, with the requirement that the second processing unit has determined that the first safety condition is not satisfied; the second processing unit determines whether a second safety condition indicating that the first target trajectory has been searched for by the third processing unit is satisfied; the third processing unit searches for a second target trajectory that brings the first spatial element closer to the boundary of the element passage area based on the recognition result by the first processing unit, with the requirement that the second safety condition has been determined that the second safety condition is not satisfied; and the control device controls the operation of the mobile device using the first target trajectory as the target trajectory when the third processing unit determines that the second safety condition is satisfied, while when the third processing unit has searched for the second target trajectory, the control device sets the second target trajectory as the provisional target trajectory and controls the operation of the mobile device using a position corresponding to the end point of the second target trajectory as a stopping position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-131940 Summary of the Invention [Problem to be solved by the invention]

[0005] Conventional autonomous systems explore the environment around the system to optimize it toward its goal, assuming that it will act after environmental uncertainty has been fully resolved. This creates the problem that it cannot act until the environmental uncertainty is resolved.

[0006] The present invention aims to enable an autonomous system to take appropriate action taking into account the uncertainty of the surrounding environment. [Means for solving the problem]

[0007] A representative example of the invention disclosed in the present application is as follows: That is, a control system for generating an action for controlling a controlled device, comprising: a receiving unit that receives sensor data observing a state of an environment surrounding the controlled device; a self-recognition unit that derives a self-recognition block that defines a self-range from the sensor data using a self-recognition prediction model that predicts a self-range, which is a range in which the controlled device has predictability and actionability; a target action prediction unit that derives a target action from the sensor data using a target action prediction model that predicts a target action of the controlled device; and a switching unit that selects the self-recognition block or the target action to generate an action of the controlled device. The switching unit selects the self-recognition block when at least one of the following conditions is satisfied: the size of the self-recognition block is larger than the sum of the size of the object on which the controlled device acts and the size of a predetermined surrounding area; the estimated execution time derived by the desired behavior prediction unit is longer than a predetermined threshold; and the current time is before the desired behavior start time. It is characterized by the following. [Effects of the Invention]

[0008] According to one aspect of the present invention, an autonomous system can take appropriate action taking into account the uncertainty of the surrounding environment. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a logical configuration of a control system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing a physical configuration of a control system according to a first embodiment. [Figure 3A] 10A and 10B are diagrams illustrating an example of the self and another person before grasping an object to be grasped. [Figure 3B] 10A and 10B are diagrams illustrating an example of a self and another person immediately after grasping an object to be grasped. [Figure 3C] FIG. 10 is a diagram showing an example of the self and the other after a certain period of time has elapsed since the object to be grasped was moved. [Figure 4A] FIG. 10 is a diagram showing an example of a self-recognition block of a grasp target before being grasped. [Figure 4B] FIG. 10 is a diagram showing an example of a self-recognition block of a grasp target immediately after being grasped. [Figure 4C] FIG. 10 shows an example of a self-recognition block of a graspable object after a period of movement. [Figure 4D] FIG. 10 is a diagram showing an example of a self-recognition block of a grasped object in a storage stage. [Figure 5] 3 is a flowchart of a process executed by the control system of the first embodiment. [Figure 6A] 10 is a flowchart of a process (pattern 1) executed by a switching unit according to the first embodiment. [Figure 6B] 10 is a flowchart of a process (pattern 2) executed by a switching unit according to the first embodiment. [Figure 6C] 10 is a flowchart of a process (pattern 3) executed by a switching unit according to the first embodiment. [Figure 7] FIG. 10 is a block diagram showing the logical configuration of a control system according to a second embodiment. [Figure 8] 10 is a flowchart of a process executed by a control system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] First, an overview of a control system 100 according to an embodiment of the present invention will be described. The entire environment including the controlled device by the control system 100 is explored so as to be separated from each other in terms of predictability, which indicates the degree to which the controlled device can predict the behavior of the object on which the controlled device acts, and actionability, which indicates whether the controlled device can act on the object, and an action is generated that takes into account the uncertainty of the environment. For this reason, the control system 100 has the function of recognizing itself and others separately, and the function of generating an action based on the result of its own recognition.

[0011] The function of recognizing the self and other separately moves parts that have already been recognized as the self to confirm the agency and predictability of objects whose self and other are unclear (i.e., both predictability and agency are unclear) or of a relatively ambiguous self (agency is clear but predictability is unclear). The function of generating behavior based on the results of self-recognition generates clear self-behavior by taking into account the predictability of the ambiguous self. This makes it possible, for example, to generate a trajectory with ample room when grasping and storing an object whose behavior is difficult to predict, preventing interference between the object and objects in the environment.

[0012] The control system 100 of this embodiment generates the behavior of an autonomous system that is a controlled device (e.g., a robot, a self-driving car, etc.), but it may also be a control device implemented in a controlled device that behaves autonomously, or a control device configured separately from the autonomous system that is the controlled device.

[0013] Example 1 FIG. 1 is a block diagram showing the logical configuration of a control system 100 according to the first embodiment.

[0014] The control system 100 includes a receiving unit 10, a self-recognition unit 20, a desired behavior prediction unit 30, a switching unit 40, and a behavior generation unit 50.

[0015] The receiving unit 10 receives sensor data indicating the status of the surrounding environment of the control system 100. The sensor data received by the receiving unit 10 includes, for example, information on the position and shape of an object (e.g., an object to be grasped) or surrounding objects observed by a camera, LiDAR, radar, etc., and the running state and arm (joint) movement observed by an encoder provided in the robot.

[0016] The self-recognition unit 20 determines the self-range from the sensor data using a self-recognition prediction model that predicts the self-range, which is the range within which predictions or actions by the control system 100 will affect. The self-recognition prediction model is generated for each object that predicts a self-recognition block, and can be configured from a neural network model that has been trained on the sensor data and the range (self-recognition block) that the object recognizes as its own. For example, the self-recognition unit 20 uses sensor data that observes the position and posture of the robot as Self-awareness prediction model to derive a self-recognition block, which is then output to the desired behavior prediction unit 30 and the switching unit 40. The self-recognition block output from the self-recognition unit 20 indicates the predicted position of an object (e.g., an object to be grasped) on which the controlled device acts.

[0017] The desired behavior prediction unit 30 derives a desired behavior from the observed sensor data and the self-recognition block using a desired behavior prediction model that predicts a desired behavior of the control system 100, and outputs the derives the desired behavior to the switching unit 40. The desired behavior prediction model can be configured using the free energy principle. According to a desired behavior prediction model that utilizes the free energy principle, a future desired behavior is determined so as to minimize a cost function that represents free energy. For example, the desired behavior prediction unit 30 derives a future arm movement from the movement of a robot arm. The desired behavior prediction unit 30 may output multiple desired behaviors with probabilities.

[0018] The switching unit 40 selects whether the behavior generation unit 50 will use the self-recognition block or the target behavior to generate a behavior, and outputs a prediction result based on the selection result.

[0019] The behavior generation unit 50 uses a behavior generation model to generate behavior from the prediction result (self-recognition block or target behavior) output from the switching unit 40. The behavior generation unit 50 generates, for example, a behavior in which the controlled device grasps a graspable object and moves to a predetermined location, or a behavior in which the controlled device guides a person at a predetermined distance so as not to interfere with the person. The behavior generation model is preferably created in advance based on rules. The behavior generation model generates behavior in which the self-recognition block does not interfere with surrounding objects, or generates behavior in accordance with the target behavior. The behavior generation unit 50 may be provided outside the control system 100, and the control system 100 may output the prediction result St to the controlled device, which then generates a behavior.

[0020] FIG. 2 is a block diagram showing the physical configuration of the control system 100 of this embodiment.

[0021] The control system 100 of this embodiment is configured by a computer having a processor (CPU) 1, a memory 2, an auxiliary storage device 3, and a communication interface 4. The control system 100 may also have an input interface 5 and an output interface 8.

[0022] The processor 1 is a computing device that executes programs stored in the memory 2. The processor 1 executes various programs to realize the functions of each functional unit (e.g., the receiving unit 10, the self-recognition unit 20, the desired behavior prediction unit 30, the switching unit 40, the behavior generation unit 50, etc.) of the control system 100. Note that some of the processing performed by the processor 1 by executing the programs may be executed by another computing device (e.g., hardware such as an ASIC or FPGA).

[0023] The memory 2 includes a ROM, which is a non-volatile storage element, and a RAM, which is a volatile storage element. The ROM stores unchanging programs (e.g., BIOS), etc. The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by the processor 1 and data used when the programs are executed.

[0024] The auxiliary storage device 3 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD). The auxiliary storage device 3 also stores data used by the processor 1 when executing a program, and the program executed by the processor 1. That is, the program is read from the auxiliary storage device 3, loaded into the memory 2, and executed by the processor 1 to realize each function of the control system 100.

[0025] The communication interface 4 is a network interface device that controls communication with other devices in accordance with a predetermined protocol.

[0026] The input interface 5 is an interface to which input devices such as a keyboard 6 and a mouse 7 are connected and which receives input from an operator. The output interface 8 is an interface to which output devices such as a display device 9 and a printer (not shown) are connected and which outputs the results of program execution in a format that can be viewed by the user. Note that a user terminal connected to the control system 100 via a network may provide the input and output devices. In this case, the control system 100 may have a web server function, and the user terminal may access the control system 100 using a predetermined protocol (for example, http).

[0027] The program executed by the processor 1 is provided to the control system 100 from a removable medium (such as a CD-ROM or flash memory) or via a network, and is stored in a non-volatile auxiliary storage device 3, which is a non-transitory storage medium. For this reason, the control system 100 may have an interface for reading data from removable media.

[0028] The control system 100 is a computer system configured on a single physical computer or on multiple logically or physically configured computers, and may operate on a virtual computer constructed on multiple physical computer resources. For example, the receiving unit 10, self-recognition unit 20, target behavior prediction unit 30, switching unit 40, and behavior generation unit 50 may each operate on separate physical or logical computers, or multiple units may be combined to operate on a single physical or logical computer.

[0029] 3A to 3C are diagrams showing examples of self and other devices in a controlled device by control system 100. FIG.

[0030] The entire environment, including the controlled device (robot) by the control system 100, is divided into self and other from the perspective of agency and predictability. Agency means that it is possible to control parts already known as "self" to change their shape and movement, and predictability means that it is possible to predict changes in shape and movement. The self is considered to be not only the robot itself but also an extended self.

[0031] The self and the other will be explained using the example of a grasping and storing task performed by a robot 80. The link lengths and range of motion of the robot 80 are known, and the robot itself is already known as the "self" 70. The object to be grasped 90 is a group of objects connected by faces or edges in a rosary-like fashion, and its shape is unknown until it is grasped. As shown in Figure 3A, before grasping the object to be grasped 90, the robot's actions do not change the position or shape of the object to be grasped, so the object to be grasped is inactive. Furthermore, since the object to be grasped 90 has been located in the same place for a certain period of time, it is predicted to remain in the same place in the future, and the object to be grasped 90 is predictable. Therefore, at the stage shown in Figure 3A, the object to be grasped 90 is determined to be the "other" 72.

[0032] As shown in Figure 3B, immediately after grasping the graspable object 90, the graspable object 90 has agentivity because the position and shape of the graspable object 90 may change depending on the actions of the robot 80. Also, since it is not known how the position and shape of the graspable object 90 will change depending on the actions of the robot 80, the graspable object 90 has low predictability. For this reason, at the stage shown in Figure 3B, the graspable object 90 is determined to be an "ambiguous self" 71.

[0033] As shown in Figure 3C, after a certain period of time has passed since the robot 80 was controlled to move the object to be grasped 90, it is known that the position and shape of the object to be grasped 90 will change due to the actions of the robot 80, and therefore the object to be grasped 90 is agentive. Furthermore, it is known how the position and shape of the object to be grasped 90 will change due to the actions of the robot 80, and therefore the object to be grasped 90 is highly predictable. For this reason, at the stage shown in Figure 3C, the object to be grasped 90 is determined to be "self" 70.

[0034] 4A to 4D are diagrams showing examples of self-recognition blocks of a grasp target 90 grasped by a controlled device (robot 80) by the control system 100. FIG.

[0035] In a grasping and storing task by the robot 80, the self-aware block 95 is generated when the object to be grasped 90 has an action, and the size of the self-aware block 95 is determined based on the predictability of the object.

[0036] For ease of explanation, only the self-recognition block 95 corresponding to the grasped object 90 is shown, and the self-recognition block corresponding to the robot 80 is omitted. As shown in FIG. 4A, before grasping the grasped object 90, the grasped object 90 is an "other" that has no agency and is predictable, so the self-recognition block 95 is not generated. As shown in FIG. 4B, immediately after grasping the grasped object 90, the grasped object 90 has agency, so the self-recognition block 95 is generated. The size of the self-recognition block 95 is calculated based on predictability. For example, the size of the self-recognition block 95 can be calculated using the accuracy (inverse of the variance) of the inference distribution of the position and orientation of the grasped object 90 relative to the position and orientation of the robot 80. As shown in FIG. 4C, after controlling the robot 80 to move the grasped object 90 for a certain period of time, the variance of the inference distribution becomes smaller, so predictability becomes higher than immediately after grasping the grasped object 90, and the size of the self-recognition block 95 becomes smaller than immediately after grasping the grasped object 90. Considering the self-aware block 95 as the actual object to be grasped 90 can prevent interference with other objects. For example, by notifying other mobile objects (or remote control devices) of the self-aware block 95, unexpected collisions with other mobile objects can be prevented. Furthermore, as shown in FIG. 4D , when storing the object to be grasped 90, the storing trajectory is calculated by considering the self-aware block 95 as the actual object to be grasped 90. When predictability is low, the self-aware block 95 is large, so the storing trajectory has room for the storage box. Calculating the trajectory by considering the self-aware block 95 as the actual object to be grasped 90 is equivalent to exposing the position and orientation of the object relative to the position and orientation of the robot 80, which were previously in a hidden state, as observed values. By exposing the hidden state as observed values, the control system 100 can determine actions by taking into account environmental uncertainty at each point during task execution.

[0037] FIG. 5 is a flowchart of the process executed by the control system 100 of this embodiment.

[0038] First, the receiving unit 10 receives sensor data (101). The self-recognition unit 20 uses a self-recognition prediction model to calculate a self-recognition block from the sensor data and outputs it (102). The desired behavior prediction unit 30 uses the desired behavior prediction model to calculate a desired behavior from the observed sensor data and outputs it (103). For example, in the case of a robot grasping and storing task, the desired behavior for storing the object to be grasped is output. Thereafter, the self-recognition unit 20 updates the self-recognition prediction model, and the desired behavior prediction unit 30 updates the desired behavior prediction model (104). The observed sensor data and the self-recognition block are used to update the self-recognition prediction model, and the observed sensor data and the desired behavior are used to update the desired behavior prediction model. The switching unit 40 selects whether to use the self-recognition block or the desired behavior (105). Details of the processing by the switching unit 40 will be described with reference to Figures 6A to 6C. Thereafter, when the switching unit 40 selects a self-recognition block, the behavior generation unit 50 uses the behavior generation model to Self-awareness prediction model The behavior generating unit 50 generates and outputs a behavior (self-aware behavior that controls the robot to change the position, shape, and movement of the object to be grasped) from the target action prediction unit 30 (107). On the other hand, when a target action is selected by the switching unit 40, the behavior generating unit 50 outputs a behavior in accordance with the target action output from the target action prediction unit 30 (108).

[0039] 6A to 6C are flowcharts of the processing executed by the switching unit 40. FIG.

[0040] The following shows three typical patterns of processing executed by the switching unit 40. The processing executed by the switching unit 40 is not limited to these patterns, and these patterns may also be combined.

[0041] These patterns are: (1) user ofOne may be selected according to the setting, (2) the self-recognition block may be selected when it is determined that the self-recognition block should be selected in all patterns by the logical product of the judgment results of all patterns, or (3) the judgment results of multiple patterns may be scored, and either the self-recognition block or the target behavior may be selected based on the total score (for example, a weighted sum).

[0042] 6A is a flowchart of the process (pattern 1) executed by the switching unit 40. In pattern 1, the switching unit 40 receives a prediction result (self-recognition block) from the self-recognition unit 20 and receives a desired action from the desired action prediction unit 30 (1051). The switching unit 40 compares the size of the self-recognition block, the actual size of the object to be grasped, and the size of a preset surrounding area. θσ Then, the switching unit 40 compares the size of the self-recognition block with the sum of the actual size of the object to be grasped and the size of the preset peripheral area (1052). θσ If the sum of the size of the object to be grasped and the size of the surrounding area is larger than the sum of the size of the object to be grasped, the self-recognition block is selected and output to the behavior generation unit 50 (1055). θσ If the sum is less than or equal to the sum of the target action and the target action is output to the action generation unit 50 (1056), Pattern 1 is effective when there is sufficient time until the time when the object to be grasped should be put away, the predictability is low at the current time, and it is desired to improve it.

[0043] FIG. 6B is a flowchart of processing (pattern 2) executed by the switching unit 40. In pattern 2, the switching unit 40 receives a prediction result (self-recognition block) from the self-recognition unit 20 and receives a desired action from the desired action prediction unit 30 (1051). The switching unit 40 compares the estimated execution time of the desired action predicted by the desired action prediction unit 30 with a preset threshold θT (1053). Then, the switching unit 40 selects the self-recognition block if the estimated execution time is longer than the threshold θT (1055). The behavior generation unit 50 generates a behavior that abandons putting away the object to be grasped and improves the accuracy of the self-recognition block. On the other hand, the switching unit 40 selects the desired action if the estimated execution time is equal to or less than the threshold θT, and outputs the desired action to the behavior generation unit 50 (1056). The estimated execution time is estimated by the desired action prediction unit 30. Pattern 2 is effective when you want to limit the time it takes to store the object to a certain amount of time (for example, when storing the object in a storage box moving on a belt conveyor).

[0044] FIG. 6C is a flowchart of processing (pattern 3) executed by the switching unit 40. In pattern 3, the switching unit 40 receives a prediction result (self-recognition block) from the self-recognition unit 20 and receives a target action from the target action prediction unit 30 (1051). The switching unit 40 compares the current time with the target action start time (1054). If the current time is before the target action start time, the switching unit 40 selects a self-recognition block and outputs the self-recognition block to the action generation unit 50 (1055). The action generation unit 50 generates a self-recognition action to improve the accuracy of the self-recognition block until the target action start time. On the other hand, if the current time is after the target action start time, the switching unit 40 selects a target action and outputs the target action to the action generation unit 50 (1056). Pattern 3 is effective when the time to put away the object to be grasped is determined and predictability until the target action start time is to be improved.

[0045] As described above, according to the control system 100 of the first embodiment, the input to the behavior generation model of the controlled device can be changed by selecting the self-recognition block or the target behavior of the switching unit 40, and behavior can be generated based on the self-recognition block with a defined self-range as needed. This allows appropriate behavior that takes into account the uncertainty of the surrounding environment.

[0046] <Example 2> In the second embodiment, a target action is requested from the switching unit 40, and the target action prediction unit 30 generates an action according to the request for the target action. In the second embodiment, differences from the first embodiment will be mainly described, and descriptions of the same configurations and functions as those in the first embodiment will be omitted.

[0047] FIG. 7 is a block diagram showing the logical configuration of a control system 100 according to the second embodiment.

[0048] The control system 100 includes a receiving unit 10, a self-recognition unit 20, a desired behavior prediction unit 30, a switching unit 40, and a behavior generation unit 50. The functions and configurations of the receiving unit 10, the self-recognition unit 20, and the behavior generation unit 50 are the same as those in the first embodiment described above.

[0049] The desired action prediction unit 30 derives a desired action from the observed sensor data and the self-recognition block using a desired action prediction model that predicts the desired action of the control system 100 in accordance with a desired action request from the switching unit 40, and outputs the derive target action to the switching unit 40. The desired action prediction model can be configured using the free energy principle. According to a desired action prediction model that utilizes the free energy principle, a future desired action is determined so as to minimize a cost function that represents free energy. For example, the desired action prediction unit 30 derives a future arm movement from the movement of a robot arm. The desired action prediction unit 30 may output multiple target actions with probabilities.

[0050] The switching unit 40 selects whether the behavior generation unit 50 will use the self-recognition block or the target behavior to generate a behavior. When the switching unit 40 selects the target behavior, it requests the target behavior prediction unit 30 to generate the target behavior.

[0051] FIG. 8 is a flowchart of the process executed by the control system 100 of this embodiment.

[0052] First, the receiving unit 10 receives sensor data (101). The self-recognition unit 20 calculates and outputs a self-recognition block from the sensor data using a self-recognition prediction model (102). Thereafter, the self-recognition unit 20 updates the self-recognition prediction model (111). The observed sensor data and the self-recognition block are used to update the self-recognition prediction model. The switching unit 40 selects whether to use the self-recognition block or the target behavior (105). Details of the processing by the switching unit 40 have been described with reference to Figures 6A to 6C. Thereafter, when the switching unit 40 selects a self-recognition block, the behavior generation unit 50 uses the behavior generation model to: Self-awareness prediction model The switching unit 40 generates and outputs a behavior (self-aware behavior that controls the robot to change the position, shape, and movement of the grasped object) from the target object (107). On the other hand, if the switching unit 40 selects a desired behavior, it requests the desired behavior prediction unit 30 to generate a desired behavior (113). Upon receiving the desired behavior request, the desired behavior prediction unit 30 updates the desired behavior prediction model (114). The desired behavior prediction unit 30 uses the observed sensor data and the desired behavior to update the desired behavior prediction model. Then, the desired behavior prediction unit 30 calculates and outputs a desired behavior from the observed sensor data using the desired behavior prediction model, and if a desired behavior is selected by the switching unit 40, the behavior generation unit 50 outputs a behavior in accordance with the desired behavior output from the desired behavior prediction unit 30 (115).

[0053] As described above, according to the control system 100 of the second embodiment, when the switching unit 40 selects a target behavior, it requests the target behavior prediction unit 30 to perform the target behavior. This reduces the calculation load on the target behavior prediction unit 30, and allows appropriate behavior to be derived with fewer calculation resources.

[0054] The present invention is not limited to the above-described embodiments, but includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0055] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0056] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0057] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0058] 1 processor 2. Memory 3 Auxiliary storage 4. Communication Interface 5 Input Interface 8 Output Interfaces 10 Receiving unit 20 Self-awareness section 30 Target behavior prediction unit 40 Switching section 50 Behavior generation part 70 self 71 Ambiguous Self 72 Others 80 Robot 90 Grasped object 95 Self-Awareness Block 100 Control System

Claims

1. A control system that generates a behavior for controlling a controlled device, a receiving unit that receives sensor data that observes a state of an environment surrounding the controlled device; a self-recognition unit that derives a self-recognition block that defines a self-range from the sensor data using a self-recognition prediction model that predicts a self-range, which is a range that has predictability and actionability by the controlled device; a desired behavior prediction unit that derives the desired behavior from the sensor data using a desired behavior prediction model that predicts a desired behavior of the controlled device; a switching unit that selects the self-recognition block or the target behavior to generate a behavior of the controlled device; The switching unit The size of the self-recognition block is greater than the sum of the size of the object on which the controlled device acts and the size of a predetermined surrounding area. The estimated execution time derived by the desired behavior prediction unit is longer than a predetermined threshold, and the current time is before the target time for starting the action, the control system selects the self-recognition block when at least one of the following is satisfied.

2. A control system for generating an action for controlling a controlled device, comprising: a receiving unit that receives sensor data that observes a state of an environment surrounding the controlled device; a self-recognition unit that derives a self-recognition block that defines a self-range from the sensor data using a self-recognition prediction model that predicts a self-range, which is a range that has predictability and actionability by the controlled device; a desired behavior prediction unit that derives the desired behavior from the sensor data using a desired behavior prediction model that predicts a desired behavior of the controlled device; a switching unit for selecting the self-recognition block or the target behavior to generate a behavior of the controlled device; A control system characterized by comprising: a behavior generation unit that uses a behavior generation model to generate behavior that guides a person from the self-recognition block or the target behavior selected by the switching unit so that the controlled device does not interfere with the person.

3. A control system according to claim 1, The predictability means that the controlled device can predict changes in the shape and movement of an object that it acts on, A control system characterized in that the actionability means that the shape or movement is changed depending on the action of the controlled device.

4. A control system according to claim 1, a behavior generation unit that generates a behavior from the self-recognition block or the target behavior selected by the switching unit using a behavior generation model; The control system is characterized in that, when the self-recognition block is selected because the estimated execution time derived by the target behavior prediction unit is longer than a predetermined threshold, the behavior generation unit abandons the original behavior regarding the object on which the controlled device acts and generates an behavior that improves the accuracy of the self-recognition block.

5. A control system according to claim 1, a behavior generation unit that generates a behavior from the self-recognition block or the target behavior selected by the switching unit using a behavior generation model; The control system is characterized in that, when the self-recognition block is selected because the current time is before the target action start time, the behavior generation unit generates an action that improves the accuracy of the self-recognition block until the target action start time.

6. A control system according to claim 1, A control system characterized by comprising a behavior generation unit that uses a behavior generation model to generate behavior in which the controlled device grasps a graspable object and moves the graspable object from the self-recognition block or the target behavior selected by the switching unit.

7. The control system of claim 1, the self-awareness unit updates the self-awareness prediction model using the sensor data; The control system is characterized in that the desired behavior prediction unit updates the desired behavior prediction model using the sensor data.

8. A behavior generation method executed by a control system that generates behavior for controlling a controlled device, comprising: the control system includes a computing device that executes predetermined computational processing and a storage device that is connected to the computing device; The behavior generation method includes: a receiving step in which the arithmetic device receives sensor data that observes a state of an ambient environment of the controlled device; a self-awareness procedure in which the computing device derives a self-awareness block that defines a self-range from the sensor data using a self-awareness prediction model that predicts a self-range, which is a range that has predictability and actionability by the controlled device; a desired behavior prediction step in which the computing device derives the desired behavior from the sensor data using a desired behavior prediction model that predicts a desired behavior of the controlled device; a switching procedure in which the computing device selects the self-aware block or the target behavior to generate a behavior for the controlled device; In the switching step, the arithmetic unit The size of the self-recognition block is greater than the sum of the size of the object on which the controlled device acts and the size of a predetermined surrounding area. The estimated execution time derived in the desired behavior prediction step is longer than a predetermined threshold; and the current time is before the target time for starting the behavior, the self-recognition block is selected when at least one of the following is satisfied.

9. A behavior generation method executed by a control system that generates behavior for controlling a controlled device, comprising: the control system includes a computing device that executes predetermined computational processing and a storage device that is connected to the computing device; The behavior generation method includes: a receiving step in which the arithmetic device receives sensor data that observes a state of an ambient environment of the controlled device; a self-awareness procedure in which the computing device derives a self-awareness block that defines a self-range from the sensor data using a self-awareness prediction model that predicts a self-range, which is a range that has predictability and actionability by the controlled device; a desired behavior prediction step in which the computing device derives the desired behavior from the sensor data using a desired behavior prediction model that predicts a desired behavior of the controlled device; a switching procedure in which the computing device selects the self-aware block or the target behavior to generate a behavior for the controlled device; A behavior generation method characterized by comprising: a behavior generation procedure in which the computing device uses a behavior generation model to generate a behavior that guides a person from the self-recognition block or the target behavior selected in the switching procedure so that the controlled device does not interfere with the person.

Images