Robotic device

Abstract

To provide a robotic device that can operate in a more versatile manner. [Solution] The robot device 20 comprises a plurality of sensor devices 50(1) to 50(n) that output detection signals, actuator devices 300 to 305 that drive movable parts, and a higher-level control device 110 that controls the robot body. The higher-level control device 110 is configured to perform calculations on a graph structure of a predetermined shape using a plurality of nodes and a plurality of edges, and uses a basic neural network in which the state of a plurality of nodes is sequentially updated by the reflection of feature quantities input to a predetermined node to other nodes connected to that predetermined node via edges. The higher-level control device 110 controls the driving of the actuator devices based on information obtained by inputting the detection signals of each of the plurality of sensor devices, or information generated from the detection signals, to a plurality of nodes of the basic neural network.

Description

Technical Field

[0001] The present invention relates to a robot device.

Background Art

[0002] Conventionally, there is a robot described in Patent Document 1 below. This robot includes a tactile detection unit, a generation unit, a storage unit, an analysis unit, and a control unit. The tactile detection unit detects contact from the outside. The generation unit generates contact information based on the detection result. The storage unit accumulates and stores the contact information for a predetermined period. The analysis unit analyzes the tendency for each user based on the accumulated contact information. The control unit controls a mechanism for causing the robot to perform an action and dynamically changes the action based on the analysis result.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the robot device described in Patent Document 1, there is room for improvement in the execution of general-purpose operations.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide a robot device that can operate more generally.

Means for Solving the Problems

[0006] The robot device includes a plurality of sensor units that detect external conditions or the condition of the robot body and output detection signals corresponding to the detected external conditions or the condition of the robot body, an actuator unit that drives the movable parts of the robot body, and a control unit that controls the robot body. The control unit is configured to perform calculations on a graph structure of a predetermined shape obtained by topologically transforming the outline of the robot body using a plurality of nodes and a plurality of edges that indicate the connection relationships between the plurality of nodes, and uses a basic neural network in which the state of the plurality of nodes is sequentially updated by reflecting the feature quantities input to the predetermined node to other nodes connected to the predetermined node via edges, and controls the driving of the actuator unit based on information obtained by inputting the detection signals of the plurality of sensor units, or information generated from the detection signals of the plurality of sensor units, to a plurality of nodes of the basic neural network.

[0007] With the above configuration, a basic neural network having a spherical graph structure can process detection signals from multiple sensor units on a graph structure that reflects their spatial relationships, thereby enabling more appropriate control of the actuator unit. As a result, the robot body can be operated in a more versatile manner. [Effects of the Invention]

[0008] According to the robotic device of the present invention, it is possible to operate the robot body in a more versatile manner. [Brief explanation of the drawing]

[0009] [Figure 1] A front view showing a schematic configuration of a robot device according to an embodiment. [Figure 2] A block diagram showing the electrical configuration of the robot device according to this embodiment. [Figure 3] A diagram showing the connection relationships of a communication network according to the embodiment. [Figure 4] A schematic diagram showing the topological representation of the robot body according to this embodiment. [Figure 5] A schematic diagram showing the outline of a spherical graph structure according to the embodiment. [Figure 6] A schematic diagram illustrating the general flow of operations between nodes in a spherical graph structure according to the embodiment. [Figure 7] A block diagram showing the internal configuration of the signal processing device according to the embodiment. [Figure 8] A schematic diagram showing the layer structure of the first neural network according to the embodiment. [Figure 9] A block diagram showing the internal configuration of the first intermediate processing unit according to the embodiment. [Figure 10] A block diagram showing the internal configuration of the second intermediate processing unit according to the embodiment. [Figure 11] A block diagram showing the internal configuration of the higher-level control device according to this embodiment. [Figure 12] A diagram schematically illustrating the relationship between the simulation space and the spherical graph structure according to the embodiment. [Figure 13] A block diagram showing the internal configuration of the third intermediate processing apparatus according to the embodiment. [Figure 14] A block diagram showing the internal configuration of the drive control device according to the embodiment. [Figure 15] A schematic diagram showing the size of the model structures of the basic neural network and the first to fifth neural networks according to the embodiment. [Figure 16] A diagram schematically illustrating the relationship between the parent model and the first to fifth neural networks according to the embodiment, along with the common latent representation space. [Figure 17] A block diagram showing an example of a method for constructing a parent model according to the embodiment. [Figure 18] A block diagram showing an example of the procedure for lightweighting a child model by distillation learning according to the embodiment. [Modes for carrying out the invention]

[0010] Hereinafter, an embodiment of a robot device will be described with reference to the drawings. For ease of understanding the description, the same reference numerals are given to the same components in each drawing as much as possible, and redundant descriptions are omitted.

[0011] <Embodiment> First, the schematic configuration of the robot device according to the embodiment will be described.

[0012] (Schematic Configuration of Robot Device) As shown in FIG. 1, the robot device 20 of the present embodiment is, for example, a humanoid robot device. The robot device 20 includes a robot body 30, a camera device 40, a distance measuring sensor 41, a microphone device 42, a speaker device 43, and a plurality of sensor devices 50(1), 50(2), ···, 50(n). Here, n is an arbitrary integer.

[0013] The robot body 30 includes a body part 31, a head part 32, shoulders 33a, 33b, arms 34a, 34b, hands 35a, 35b, a waist part 36, and legs 37a, 37b. The robot body 30 is driven by the power of a battery 38 built in the waist part 36.

[0014] Actuator devices 300, 301 are respectively built in the shoulders 33a, 33b. The actuator devices 300, 301 respectively operate the arms 34a, 34b. Actuator devices 302, 303 are respectively built in the hands 35a, 35b. The actuator devices 302, 303 respectively operate the hands 35a, 35b. Actuator devices 304, 305 are respectively built in the legs 37a, 37b. The actuator devices 304, 305 respectively operate the legs 37a, 37b. In the present embodiment, the actuator devices 300 to 305 are an example of an actuator part, and the shoulders 33a, 33b, the arms 34a, 34b, the hands 35a, 35b, and the legs 37a, 37b are an example of movable parts of the robot body 30.

[0015] In addition, actuator devices for operating other parts of the robot body 30 are also provided, but these are not shown in Figure 1.

[0016] The camera device 40, distance sensor 41, microphone device 42, and speaker device 43 are mounted on the head 32. The camera device 40 captures images of the area in front of the robot body 30 and acquires image data of the area in front of the robot body 30. The distance sensor 41 is a sensor that measures the distance to an object in front of the robot body 30. The microphone device 42 is a sensor that detects sounds in the vicinity of the robot body 30. The speaker device 43 functions as the mouth of the robot body 30 and emits voices, music, etc. In this embodiment, the camera device 40, distance sensor 41, and microphone device 42 are examples of external sensor units, and the information detected by them is an example of external detection information.

[0017] Multiple sensor devices 50(1), 50(2), ..., 50(n) are distributed across the surface of the robot body 30. Specifically, sensor devices 50(1) to 50(n) are evenly distributed across the surface of each part of the robot body 30, such as the hands 35a, 35b, arms 34a, 34b, body 31, and legs 37a, 37b. Sensor devices 50(1) to 50(n) detect external conditions or the state of the robot body 30. Examples of sensor devices 50(1) to 50(n) include tactile sensors that detect contact conditions with an object, such as the presence or absence of contact and contact pressure; thermal sensors that detect the temperature and heat transfer state of an object; humidity sensors that detect the humidity of an object; and acceleration sensors and gyroscopes that detect the posture and movement of the robot body 30. The tactile sensors and thermal sensors correspond to sensory organs provided on the surface of the robot body 30. Sensory organs such as touch and heat receptors are distributed on the surface of the human body, and the sensor devices 50(1) to 50(n) are modeled after these sensory organs. In this embodiment, the sensor devices 50(1) to 50(n) are examples of sensor units.

[0018] (Electrical configuration of the robotic device) Next, we will describe the electrical configuration of the robot device 20.

[0019] As shown in Figure 2, the robot device 20 includes a plurality of signal processing devices 70(1), 70(2), ..., 70(n), a plurality of first intermediate processing devices 90~95, a second intermediate processing device 100, a higher-level control device 110, a third intermediate processing device 120, and a plurality of drive control devices 130~134. The signal processing devices 70(1)~70(n) are provided in correspondence with each of the sensor devices 50(1)~50(n). As shown in Figure 3, each device is connected to each other so as to be able to communicate via a communication network Na provided inside the robot body 30. In the following, one or more sensor devices included in the sensor devices 50(1)~50(n) may be referred to as "sensor device 50". Also, one or more signal processing devices included in the signal processing devices 70(1)~70(n) may be referred to as "signal processing device 70". In this embodiment, the first intermediate processing units 90-95 are an example of the first intermediate processing unit, the second intermediate processing unit 100 is an example of the second intermediate processing unit, the third intermediate processing unit 120 is an example of the third intermediate processing unit, and the drive control devices 130-134 are an example of the drive control unit.

[0020] The higher-level control unit 110 has a basic neural network NN0 configured to perform computational processing on a spherical graph structure Sg using multiple nodes and multiple edges indicating the connection relationships between the multiple nodes. The basic neural network NN0 is configured so that the state of multiple nodes is sequentially updated when a feature quantity input to a predetermined node is reflected in other nodes connected to that predetermined node via edges. The higher-level control unit 110 functions as a control unit that controls the driving of actuator devices 300 to 305 based on information obtained by inputting the detection signals of the multiple sensor devices 50(1) to 50(n), or information generated from the detection signals of the multiple sensor devices 50(1) to 50(n), to multiple nodes of the basic neural network NN0.

[0021] (Topological representation and spherical graph structures) Here, the spherical graph structure Sg in this embodiment will be described in detail.

[0022] The robot body 30 is humanoid and has an external shape similar to that of a human body. From a topological standpoint, the human body, which has a digestive tract, can be represented as a donut-shaped (torus), but since the robot body 30 does not have a digestive tract, it can be represented as a spherical shape when topologically deformed. Figure 4 is a schematic diagram showing the topological representation of the robot body 30. As shown in Figure 4, the complex external shape of the robot body 30 is simplified to a spherical shape through topological transformation.

[0023] Figure 5 is a schematic diagram illustrating the spherical graph structure Sg. As shown in Figure 5, the spherical graph structure Sg is composed of multiple nodes Nd arranged on the surface of a sphere and multiple edges Eg connecting adjacent nodes Nd.

[0024] (Method for generating spherical lattices) The arrangement of multiple nodes Nd in a spherical graph structure Sg is defined by a spherical lattice that divides the sphere into multiple faces. For example, the following methods may be used to generate the spherical lattice.

[0025] The first method is a recursive partitioning method based on a regular icosahedron. A regular icosahedron is a polyhedron composed of 20 equilateral triangular faces, and its vertices are evenly distributed on the surface of a sphere. By recursively partitioning each triangular face of the regular icosahedron into four smaller triangles with vertices at the midpoints of each edge, and projecting each generated vertex onto the surface of the sphere, an arrangement of nodes Nd that are approximately evenly distributed on the surface of the sphere is obtained. The more times this recursive partitioning is performed, the more nodes Nd are produced, resulting in finer spatial resolution. For example, if the number of partitions is 0, 1, 2, and 3, the number of nodes Nd will be 12, 42, 162, and 642, respectively. The shape of the grid obtained by this method is triangular.

[0026] The second method is a division based on longitude and latitude. The sphere is divided into equal intervals in the longitude and latitude directions, and each intersection is placed as a node Nd. This method is easy to implement, but it has the characteristic that the density of nodes Nd is high near the poles of the sphere. The shape of the grid obtained by this method is quadrilateral. In order to reduce the uneven distribution of node density near the poles, an equal-area division known as HEALPix (Hierarchical Equal Area isoLatitude Pixelization) may be used, in which the number of divisions in the longitude direction is adjusted according to the number of divisions in the latitude direction.

[0027] The third method is based on Goldberg polyhedra. Goldberg polyhedra are polyhedra based on regular dodecahedrons that divide a sphere into approximately equal parts using combinations of pentagons and hexagons. The shape of the lattice obtained by this method is a combination of pentagons and hexagons, that is, a so-called soccer ball shape.

[0028] The method for generating the spherical grid is not limited to the method described above; any method that allows for a nearly uniform distribution of nodes Nd on the sphere may be used. Furthermore, the fineness of the division of the spherical grid, i.e., the number of nodes Nd, may be appropriately set according to the number n of sensor devices 50(1) to 50(n).

[0029] (Method of constructing edges) Multiple edges Eg are configured to connect adjacent nodes Nd in a spherical lattice. Specifically, the edges that make up each face of the spherical lattice correspond to edges Eg. For example, in the case of a triangular lattice based on the icosahedron described above, the three edges of each triangle become edges Eg, and each node Nd is connected to six adjacent nodes Nd via six edges Eg in principle. However, a node Nd originating from an original vertex of the icosahedron is connected to five adjacent nodes Nd via five edges Eg.

[0030] Furthermore, the method for constructing edges Eg is not limited to methods based on the edges of a spherical grid. For example, a method may be used in which nodes Nd whose geodesic distance on the sphere is less than or equal to a predetermined threshold are connected by edges Eg, or a method based on a so-called k-nearest neighbor graph may be used in which a predetermined number of nodes Nd are connected by edges Eg to each node Nd, starting with those closest in geodesic distance on the sphere.

[0031] Each node Nd is associated with one of the sensor devices 50(1) to 50(n) distributed across the surface of the robot body 30. Specifically, the position of each sensor device on the surface of the robot body 30 is mapped to a position on a sphere by topological transformation, and each sensor device is associated with the node Nd closest to that position on the sphere. In other words, the position of each node Nd on the spherical graph structure Sg corresponds to the position of the sensor devices 50(1) to 50(n) on the surface of the robot body 30. As a result, nodes Nd corresponding to sensor devices that are placed close to each other on the surface of the robot body 30 are also placed close to each other on the spherical graph structure Sg, making it possible to input the detection signals of each sensor device 50(1) to 50(n) to the corresponding node Nd on the spherical graph structure Sg, which reflects the spatial relationships on the surface of the robot body 30.

[0032] Figure 6 schematically illustrates the operation between nodes Nd in a spherical graph structure Sg. As shown in Figure 6, in the basic neural network NN0, features input to a given node Nd are propagated to other adjacent nodes Nd via edges Eg. Specifically, each node Nd updates its own features based on the features it holds and the features received from adjacent nodes Nd via edges Eg. This update process is performed sequentially or in parallel across multiple nodes Nd. This operation, called message passing, enables the propagation of information throughout the entire spherical graph structure Sg.

[0033] In this way, by using a spherical graph structure Sg, it becomes possible to process detection signals from numerous sensor devices 50(1) to 50(n) distributed on the surface of the robot body 30 while maintaining their spatial relationships in three-dimensional space. Furthermore, because the complex shape of the robot body 30 is simplified to a spherical shape through topological transformation, the amount of computation can be reduced. In addition, since sensory information including noise components contained in the detection signals from sensor devices 50(1) to 50(n) can be handled as is, it becomes possible to utilize information including the noise component in the signal-to-noise ratio.

[0034] In this embodiment, the basic neural network NN0 is preferably a graph neural network (GNN). A GNN is a neural network that performs computational processing on a graph structure Sg consisting of nodes and edges. Specific structures of the GNN may include, for example, a graph convolutional network (GCN), a graph attention network (GAT), a message-passing neural network (MPNN), or a temporal graph neural network (T-GNN) capable of handling information in the time direction. However, the basic neural network NN0 is not limited to a GNN; any neural network capable of performing computational processing on a graph structure Sg consisting of nodes and edges is acceptable.

[0035] (Configuration of signal processing device) Next, the configuration of the signal processing units 70(1) to 70(n) will be described. As shown in Figure 2, the signal processing units 70(1) to 70(n) are provided in correspondence to each of the sensor devices 50(1) to 50(n) and process the detection signals output from the sensor devices 50(1) to 50(n). Since the basic configuration of each of the signal processing units 70(1) to 70(n) is the same or similar, the configuration of the signal processing unit 70(1) corresponding to the sensor device 50(1) will be described as representative below.

[0036] As shown in Figure 7, the signal processing device 70(1) includes a communication device 700, an input interface 701, a storage device 702, and a control device 703. The communication device 700 communicates with the first intermediate processing device 90 via the communication network Na. The detection signal output from the sensor device 50(1) is input to the input interface 701.

[0037] The storage device 702 is a non-volatile storage medium such as a hard disk drive, flash memory, or SSD. The storage device 702 stores various programs for controlling the signal processing device 70(1). The storage device 702 also stores the first neural network NN1. The first neural network NN1 is a trained neural network constructed by inheriting the signal processing capabilities of the basic neural network NN0 through lightweight processing such as distillation learning, which will be described later. The first neural network NN1 has a lighter structure than the basic neural network NN0, while being configured to generate feature representations in a latent representation space common to the basic neural network NN0. The first neural network NN1 receives the detection signal from the sensor device 50(1) as input and generates first detection information corresponding to the detection signal.

[0038] As shown in Figure 8, the first neural network NN1 comprises a plurality of input layers 200, a plurality of hidden layers 201, and a plurality of output layers 202. The input layer 200 receives the detection signal from the sensor device 50(1). The hidden layer 201 performs predetermined calculations on the information acquired from the input layer 200, including weighting calculations, nonlinear transformations, or feature extraction processing. The output layer 202 generates output information based on the feature representation generated by the hidden layer 201. For example, the output layer 202 outputs a latent feature vector corresponding to the detection signal from the sensor device 50(1) as first detection information.

[0039] The control device 703 is primarily composed of a microcomputer with a processor and RAM, and comprehensively controls the signal processing device 70(1). The control device 703 includes a signal processing unit 703a, which is a functional configuration realized by the processor executing a program stored in the storage device 702.

[0040] The signal processing unit 703a acquires the detection signal from the sensor device 50(1) at a predetermined interval and inputs the acquired detection signal from the sensor device 50(1) to the first neural network NN1 to generate first detection information corresponding to the detection signal from the sensor device 50(1). The signal processing unit 703a transmits the generated first detection information to the first intermediate processing unit 90 via the communication device 700.

[0041] (Configuration of the first intermediate processing unit) Next, the configuration of the first intermediate processing units 90-95 will be described.

[0042] As shown in Figure 2, the first intermediate processing units 90 to 95 are provided to correspond to each part of the robot body 30. For example, the first detection information output from the signal processing unit 70 corresponding to the sensor device 50 located on the surface of the right hand 35a is input to the first intermediate processing unit 90. The first detection information output from the signal processing unit 70 corresponding to the sensor device 50 located on the surface of the right arm 34a is input to the first intermediate processing unit 91. The first detection information from the signal processing unit 70 corresponding to the sensor devices 50 located on the surface of the left hand 35b, left arm 34b, right leg 37a, and left leg 37b is input to the first intermediate processing units 92 to 95, respectively.

[0043] Since the basic configurations of the first intermediate processing units 90 to 95 are the same or similar, the configuration of the first intermediate processing unit 90 will be described as representative below.

[0044] As shown in Figure 9, the first intermediate processing unit 90 includes a communication device 900, a storage device 901, and a control device 902.

[0045] The communication device 900 communicates with the signal processing device 70 and the second intermediate processing device 100 via the communication network Na.

[0046] The storage device 901 is a non-volatile storage medium such as a hard disk drive, flash memory, or SSD. The storage device 901 stores various programs for controlling the first intermediate processing unit 90. The storage device 901 also stores the second neural network NN2. The second neural network NN2 is a trained neural network capable of generating second detection information by integrating multiple first detection information outputs from corresponding signal processing units. The second neural network NN2 is constructed by inheriting the information integration capabilities of the basic neural network NN0 through lightweight processing such as distillation learning, which will be described later. The second neural network NN2 is also constructed using a neural network as shown in Figure 8, for example.

[0047] The control device 902 is primarily composed of a microcomputer with a processor and RAM, and comprehensively controls the first intermediate processing unit 90. The control device 902 includes an information processing unit 902a, which is a functional configuration realized by the processor executing a program stored in the storage device 901.

[0048] The information processing unit 902a inputs multiple first detection information outputs from the corresponding signal processing units 70 to the second neural network NN2, thereby generating a second detection information by integrating these first detection information. The information processing unit 902a transmits the generated second detection information to the second intermediate processing unit 100 via the communication device 900.

[0049] (Configuration of the second intermediate processing unit) Next, the configuration of the second intermediate processing unit 100 will be described.

[0050] As shown in Figure 10, the second intermediate processing unit 100 includes a communication device 1000, a storage device 1001, and a control device 1002.

[0051] The communication device 1000 communicates with the first intermediate processing units 90-95 and the higher-level control unit 110 via the communication network Na.

[0052] The storage device 1001 is a non-volatile storage medium such as a hard disk drive, flash memory, or SSD. The storage device 1001 stores various programs for controlling the second intermediate processing unit 100. The storage device 1001 also stores the third neural network NN3. The third neural network NN3 is a trained neural network capable of generating third detection information by further integrating multiple second detection information outputs from the first intermediate processing units 90 to 95. The third neural network NN3 is constructed by inheriting the ability of the basic neural network NN0 to integrate sensory information from the whole body through lightweight processing such as distillation learning, which will be described later. The third neural network NN3 is also constructed by a neural network as shown in Figure 8, for example.

[0053] The control device 1002 is primarily composed of a microcomputer with a processor and RAM, and comprehensively controls the second intermediate processing unit 100. The control device 1002 includes an information processing unit 1002a, which is a functional configuration realized by the processor executing a program stored in the storage device 1001.

[0054] The information processing unit 1002a inputs multiple second detection information outputs from the first intermediate processing units 90 to 95 to the third neural network NN3, thereby generating a third detection information that further integrates these second detection information. The information processing unit 1002a transmits the generated third detection information to the higher-level control unit 110 via the communication device 1000.

[0055] (Configuration of the higher-level control unit) Next, the configuration of the higher-level control device 110 will be described.

[0056] As shown in Figure 11, the higher-level control device 110 includes a communication device 1100, an input I / F 1101, a storage device 1102, and a control device 1103.

[0057] The communication device 1100 communicates with the second intermediate processing unit 100 and the third intermediate processing unit 120 via the communication network Na.

[0058] The input interface 1101 receives image data acquired by the camera device 40, distance data detected by the distance measuring sensor 41, and sound data acquired by the microphone device 42, among other things.

[0059] The storage device 1102 is a non-volatile storage medium such as a hard disk drive, flash memory, or SSD. The storage device 1102 stores various programs for controlling the higher-level control device 110. The storage device 1102 also stores the basic neural network NN0. The basic neural network NN0 is a trained neural network configured to perform arithmetic processing on the spherical graph structure Sg described above. The basic neural network NN0 can generate first instruction information that instructs the operation of the entire robot body 30 based on the third detection information output from the second intermediate processing device 100, image data acquired by the camera device 40, distance data detected by the distance measuring sensor 41, and sound data acquired by the microphone device 42.

[0060] The generation of the first instruction information by the basic neural network NN0 will be explained in detail below, relating it to the nodes Nd and edges Eg of the spherical graph structure Sg.

[0061] (Input to the node) In the initial stages of inference processing by the basic neural network NN0, corresponding input information is set as features for each of the multiple nodes Nd that make up the spherical graph structure Sg. Specifically, the node Nd corresponding to each of the sensor devices 50(1) to 50(n) is input with a feature vector corresponding to each sensor device, which is included in the third detection information output from the second intermediate processing unit 100. The third detection information is integrated detection information generated through hierarchical processing by the signal processing units 70(1) to 70(n), the first intermediate processing units 90 to 95, and the second intermediate processing unit 100, and therefore includes both local features of the detection signals of individual sensors and global features that integrate the detection signals of multiple sensors.

[0062] If the number of nodes Nd on the spherical graph structure Sg does not match the number n of sensor devices 50(1) to 50(n), then a zero vector or a predetermined initial value may be set as a feature for nodes Nd that are not associated with a sensor device 50, or a value interpolated from the feature of a node Nd corresponding to a neighboring sensor device 50 may be set.

[0063] (Integration of multimodal information into a graph structure) The basic neural network NN0 integrates multimodal information, such as image data acquired by the camera device 40, distance data detected by the distance measuring sensor 41, and sound data acquired by the microphone device 42, into the feature quantities of each node Nd on the spherical graph structure Sg. For example, the following methods may be used for integration.

[0064] The first method involves adding multimodal information as a global feature vector to all nodes Nd of a spherical graph structure Sg. Specifically, image data, distance data, and sound data are each converted into feature vectors by a predetermined encoder, and these feature vectors are combined or added to generate a global feature vector. This global feature vector is then combined or added to the feature quantities of all nodes Nd on the spherical graph structure Sg. As a result, each node Nd retains both local feature quantities based on the detection signal of the corresponding sensor device and global feature quantities based on multimodal information from the camera device 40, etc., and the update process by message passing, described later, is then performed.

[0065] The second method involves selectively adding multimodal information to specific nodes Nd on a spherical graph structure Sg. For example, if object position information is extracted from image data acquired by a camera device 40, the object's feature vector is added to the node Nd corresponding to the nearest position of the object on the surface of the robot body 30. This method achieves local information integration that reflects spatial relationships.

[0066] Furthermore, the method for integrating multimodal information into the graph structure Sg is not limited to the method described above. Any method may be used, such as a method in which each node Nd selectively retrieves the portion of the multimodal information relevant to itself using a graph attention mechanism.

[0067] (Inference processing via message passing) After features are set for each node Nd as described above, the basic neural network NN0 repeatedly performs message passing multiple times on the spherical graph structure Sg. In each round, each node Nd receives a message from a neighboring node Nd via an edge Eg and updates its own features.

[0068] In one round of message passing, the feature quantities of each node Nd are updated based only on the feature quantities of neighboring nodes Nd that are directly connected to that node Nd by an edge Eg. However, by repeating message passing multiple times, the feature quantities of each node Nd indirectly reflect information from more distant nodes Nd. For example, if k rounds of message passing are performed, the feature quantities of each node Nd will reflect information from nodes Nd whose shortest path on the spherical graph structure Sg is within k edges. In this way, by appropriately setting the number of message passing rounds, the propagation of information across the entire spherical graph structure Sg is controlled. That is, sensory information detected locally on the surface of the robot body 30 is integrated into global information covering the entire body through repeated message passing.

[0069] (Generation of the first instruction information through readout processing) After multiple rounds of message passing are complete, the basic neural network NN0 performs a readout process to aggregate information from the entire spherical graph structure Sg. In the readout process, pooling operations are applied to the final feature quantities of all nodes Nd on the spherical graph structure Sg. Pooling operations include, for example, global average pooling, which takes the average of the feature quantities of all nodes Nd; global maximum pooling, which takes the maximum feature quantity of all nodes Nd; or attention pooling, which dynamically determines the weighting of the feature quantities of each node Nd using an attention mechanism.

[0070] The graph-level feature vectors generated by the readout process are further input to an output layer such as a fully connected layer and converted into first instruction information that instructs the movement of the entire robot body 30. The first instruction information includes information that instructs actions such as the direction of movement, movement speed, posture change of the robot body 30, or operation on a specific object. The basic neural network NN0 (parent model LM) is configured to perform a series of processes end-to-end, from the above-mentioned readout process and output layer, to the distribution of the first instruction information to individual instruction information corresponding to each of the multiple actuator devices 300 to 305, and the generation of control signals corresponding to each actuator device 300 to 305. However, in this embodiment, the information actually output by the higher-level control device 110 is the first instruction information, the distribution from the first instruction information to individual instruction information is handled by the third intermediate processing device 120, and the generation of control signals is handled by the drive control devices 130 to 134.

[0071] Here, we will provide further details regarding the relationship between the spherical graph structure Sg described above and the physical structure of the robot body 30. The basic neural network NN0 has model data, including shape data and joint structure data representing the three-dimensional dimensions of the entire robot body 30, pre-stored as training information in the memory device 1102. Each node Nd of the spherical graph structure Sg is pre-associated with corresponding three-dimensional positional information on the surface of the robot body 30, based on the model data.

[0072] Furthermore, the basic neural network NN0 generates three-dimensional spatial data representing the external environment around the robot body 30 based on various detection information acquired from the camera device 40, the distance sensor 41, and the microphone device 42. Based on the above model data and the above three-dimensional spatial data, the basic neural network NN0 generates a simulation space that virtually reproduces the robot body 30 and its surrounding environment. Figure 12 is a schematic diagram showing the relationship between the simulation space of the higher-level control device 110 and the spherical graph structure Sg. As shown in Figure 12, the spherical graph structure Sg is associated with the three-dimensional positional information corresponding to the surface of the robot body 30 within this simulation space.

[0073] The simulation space is used in the integration of the multimodal information described above into the graph structure Sg. Specifically, image data acquired by the camera device 40 and distance data detected by the distance measuring sensor 41 are converted into three-dimensional arrangement information of the external environment within the simulation space and then integrated into each node Nd of the spherical graph structure Sg. For example, if an object exists near the surface of the robot body 30 within the simulation space, a feature vector indicating the attributes of that object is added to the node Nd closest to that object. In this way, the simulation space functions as a coordinate reference system for spatially associating and integrating multimodal information into appropriate nodes Nd on the spherical graph structure Sg.

[0074] As described above, the basic neural network NN0 integrates the third detection information and multimodal information into each node Nd of the spherical graph structure Sg under spatial correspondence based on the simulation space, and generates first instruction information that instructs the movement of the entire robot body 30 by performing inference processing including message passing and readout processing.

[0075] (The world model inherent in basic neural networks) Here, we will explain the information structure inherent within the basic neural network NN0.

[0076] In large-scale language models (LLMs), it is known that knowledge about the world described by language is encoded as a distributed representation in the numerous weight parameters of the trained model. For example, this knowledge is statistically held in the set of weight parameters of the self-attention layer and feedforward layer in the Transformer architecture. In other words, it is inferred that the weight parameters of an LLM function as a model of the world expressed through language (a world model).

[0077] Similarly, in the basic neural network NN0 of this embodiment, at least the following information is distributedly encoded within the entire set of learned weight parameters. Firstly, information regarding the topological body structure of the robot body 30 is encoded. Specifically, the connection relationships of each node Nd in the spherical graph structure Sg and the weight parameters of the associated edges Eg intrinsically reflect the shape of the body surface of the robot body 30, the relative positional relationships of each part, and the kinematic constraints between each part derived from the joint structure. Secondly, information regarding the contactable range (range that can be touched) of each sensor device 50(1) to 50(n) is encoded. Specifically, the spatial receptive field of each sensor device 50(1) to 50(n), that is, the range on the body surface where each sensor device 50(1) to 50(n) can detect physical contact, is intrinsically reflected in the weight parameters that define the update rules for the feature quantities of each node Nd in message passing. Thirdly, information regarding the observable range (the range in which it can be seen or heard) of external sensor units such as the camera device 40, the distance measuring sensor 41, and the microphone device 42 is encoded. Specifically, information regarding the observation characteristics of each external sensor unit, such as the field of view, detection distance range, and directivity, is intrinsically reflected in the weight parameters when integrating multimodal information into each node Nd of the graph structure Sg.

[0078] The information described above is not necessarily localized to a specific location in the network structure of the basic neural network NN0, but rather is distributed across the weight parameters and message passing update rules across multiple nodes Nd. This is similar to the fact that in LLM, it is impossible to observe where specific knowledge resides within the network.

[0079] Thus, the entire set of trained weight parameters of the basic neural network NN0 functions as an internal model (hereinafter referred to as the "world model") in which information about the body structure of the robot body 30, the contactable range of each sensor unit, the observable range of each external sensor unit, and the interaction between the robot body 30 and the surrounding environment is distributed and encoded. The world model is equivalent to a graph neural network version of the world knowledge that is considered to be inherent in the weight parameters in LLM, and is characterized in that it directly encodes relationships in three-dimensional space as weight parameters on a spherical graph structure Sg without going through a one-dimensional representation using language. Based on this world model, the basic neural network NN0 integrally processes detection signals from sensor devices 50(1) to 50(n) and multimodal information from external sensor units to generate first instruction information that instructs the operation of the entire robot body 30.

[0080] The control unit 1103 is primarily composed of a microcomputer with a processor and RAM, and comprehensively controls the higher-level control unit 110. The control unit 1103 includes an information processing unit 1103a, which is a functional configuration realized by the processor executing a program stored in the storage device 1102.

[0081] The information processing unit 1103a generates first instruction information that instructs the movement of the entire robot body 30 by inputting the third detection information output from the second intermediate processing unit 100, image data acquired by the camera device 40, distance data detected by the distance measuring sensor 41, and sound data acquired by the microphone device 42 into the basic neural network NN0. The information processing unit 1103a transmits the generated first instruction information to the third intermediate processing unit 120 via the communication device 1100.

[0082] (Configuration of the third intermediate processing unit) Next, the configuration of the third intermediate processing unit 120 will be described.

[0083] As shown in Figure 13, the third intermediate processing unit 120 includes a communication device 1200, a storage device 1201, and a control device 1202.

[0084] The communication device 1200 communicates with the higher-level control device 110 and the drive control devices 130-134 via the communication network Na.

[0085] The storage device 1201 is a non-volatile storage medium such as a hard disk drive, flash memory, or SSD. The storage device 1201 stores various programs for controlling the third intermediate processing unit 120. The storage device 1201 also stores the fourth neural network NN4. The fourth neural network NN4 is a trained neural network capable of generating multiple second instruction pieces corresponding to each of the multiple actuator devices 300 to 305 from the first instruction piece output from the higher-level control unit 110. The fourth neural network NN4 is constructed by inheriting the ability of the basic neural network NN0 to distribute operation instructions through lightweight processing such as distillation learning, which will be described later. The fourth neural network NN4 is also constructed by a neural network such as the one shown in Figure 8.

[0086] The control device 1202 is primarily composed of a microcomputer with a processor and RAM, and comprehensively controls the third intermediate processing unit 120. The control device 1202 includes an information processing unit 1202a, which is a functional configuration realized by the processor executing a program stored in the storage device 1201.

[0087] The information processing unit 1202a inputs the first instruction information output from the higher-level control unit 110 to the fourth neural network NN4, thereby generating multiple second instruction pieces corresponding to each of the multiple actuator devices 300 to 305. The information processing unit 1202a then transmits the generated multiple second instruction pieces to the drive control devices 130 to 134 via the communication device 1200.

[0088] (Configuration of the drive control system) Next, the configuration of the drive control devices 130 to 134 will be described.

[0089] As shown in Figure 2, the drive control devices 130 to 134 are provided to correspond to the actuator devices 300 to 305, respectively, and control the operation of each of the actuator devices 300 to 305. Since the basic configurations of the drive control devices 130 to 134 are the same, the configuration of the drive control device 134 corresponding to the actuator devices 304 and 305, that is, the legs 37a and 37b of the robot body 30, will be described as representative below.

[0090] As shown in Figure 14, the drive control device 134 includes a communication device 1300, an output I / F 1301, a storage device 1302, and a control device 1303.

[0091] The communication device 1300 communicates with the third intermediate processing unit 120 via the communication network Na.

[0092] Output I / F1301 outputs control signals, etc., to actuator devices 304 and 305.

[0093] The storage device 1302 is a non-volatile storage medium such as a hard disk drive, flash memory, or SSD. The storage device 1302 stores various programs for controlling the drive control device 134. The storage device 1302 also stores the fifth neural network NN5. The fifth neural network NN5 is a trained neural network capable of generating control signals from the second instruction information output from the third intermediate processing device 120 to cause the actuator devices 304 and 305 to perform operations corresponding to the second instruction information. The fifth neural network NN5 is constructed by inheriting the drive control capabilities of the basic neural network NN0 through lightweight processing such as distillation learning, which will be described later. The fifth neural network NN5 is also constructed using a neural network as shown in Figure 8, for example.

[0094] The control device 1303 is primarily composed of a microcomputer with a processor and RAM, and comprehensively controls the drive control device 134. The control device 1303 includes an information processing unit 1303a, which is a functional configuration realized by the processor executing a program stored in the storage device 1302.

[0095] The information processing unit 1303a inputs the second instruction information output from the third intermediate processing unit 120 to the fifth neural network NN5, thereby generating control signals corresponding to the actuator devices 304 and 305 provided on the legs 37a and 37b of the robot body 30. The information processing unit 1303a then outputs the control signals thus generated to the actuator devices 304 and 305 via the output I / F 1301, thereby operating the legs 37a and 37b.

[0096] (Neural network configuration) Next, we will describe the configurations of the first neural network NN1 installed in the signal processing units 70(1) to 70(n), the second neural network NN2 installed in the first intermediate processing units 90 to 95, the third neural network NN3 installed in the second intermediate processing unit 100, the basic neural network NN0 installed in the higher-level control unit 110, the fourth neural network NN4 installed in the third intermediate processing unit 120, and the fifth neural network NN5 installed in the multiple drive control units 130 to 134.

[0097] Figure 15 schematically shows the size of the model structures of the basic neural network NN0 and the first to fifth neural networks NN1 to NN5. In this embodiment, the basic neural network NN0, which is mounted on the higher-level control device 110 that comprehensively controls the robot body 30, has the largest model structure. The third neural network NN3, mounted on the second intermediate processing device 100 located below the higher-level control device 110, and the fourth neural network NN4, mounted on the third intermediate processing device 120, have more compressed model structures than the basic neural network NN0. Furthermore, the second neural network NN2, mounted on the first intermediate processing devices 90 to 95, is more compressed than the model structure of the third neural network NN3, and the first neural network NN1, mounted on the signal processing devices 70(1) to 70(n), is even more compressed than the model structure of the second neural network NN2. Furthermore, the fifth neural network NN5, which is installed in each of the multiple drive control devices 130 to 134, has a more compressed model structure than the fourth neural network NN4. This compression of the model structure involves reducing at least one of the following: the number of layers, parameters, computational complexity, and display dimensions.

[0098] In this way, in the robot device 20, by using a compressed neural network from the upper-level control device 110 towards the terminal devices, the operation of each terminal device, such as the signal processing devices 70(1) to 70(n) and the drive control devices 130 to 134, can be accelerated. As a result, the robot body 30 can be operated smoothly.

[0099] In this embodiment, as shown in Figure 16, a parent model LM is pre-constructed that is capable of comprehensively controlling the entire robot body 30 and generating highly accurate feature representations. The parent model LM in this embodiment corresponds to the basic neural network NN0. The first to fifth neural networks NN1 to NN5 are configured as neural networks with reduced computational complexity, model structure, or number of parameters, while retaining at least a portion of the feature representations or output information generated by the parent model LM. In other words, the first to fifth neural networks NN1 to NN5 can be considered child models of the parent model LM.

[0100] As shown in Figure 16, the parent model LM and the first to fifth neural networks NN1 to NN5 are each configured to represent the feature information generated based on the input data as latent feature vectors in a common latent representation space L. The common latent representation space L is a feature representation space that allows the feature information generated by each neural network NN1 to NN5 to be represented in a common coordinate system. This common latent representation space L is configured such that semantically similar feature representations are placed in close proximity to each other within the space.

[0101] As shown in Figure 17, the parent model LM is constructed, for example, through a learning process using training data. The training data includes, for example, input data and training data associated with the input data. The input data can be, for example, the output signals of the camera device 40, distance sensor 41, microphone device 42, speaker device 43, and sensor devices 50(1) to 50(n) shown in Figure 1, or detection information representing the state of the robot body 30 and the external environment acquired based on these output signals. The training data can be the control signals of each actuator device 300 to 305.

[0102] The first to fifth neural networks NN1 to NN5 are constructed after creating a parent model LM in advance. These networks retain the semantic features extracted by the parent model LM while having a lighter structure than the parent model LM. The first to fifth neural networks NN1 to NN5 are created by lightweighting processes such as distillation learning, pruning, or quantization. Figure 18 is a block diagram illustrating an example of the lightweighting process. In Figure 18, the procedure for lightweighting the first neural network NN1 using distillation learning is shown as an example.

[0103] As shown in Figure 18, in the lightweighting process, the distillation input data is first input to the trained parent model LM. The distillation input data includes the training data used to train the parent model LM, detection information detected in the real environment by sensor devices 50(1) to 50(n), etc., or virtual input data generated in the simulation space. When the parent model LM receives this input data, it performs inference processing including setting features for each node Nd of the spherical graph structure Sg, integrating multimodal information, updating features by message passing, and readout processing. In the distillation process, intermediate feature vectors generated for each node Nd during the inference process, graph-level feature vectors generated by the readout processing, or feature vectors output from the output layer are used as training information for distillation.

[0104] For example, in the lightweighting process of the first neural network NN1, partial data corresponding to the detection signals of the sensor devices 50 that the first neural network NN1 is responsible for, from among the detection signals of multiple sensor devices 50 included in the distillation input data, is input to the first neural network NN1. Meanwhile, the entire distillation input data is input to the parent model LM and inference processing is performed. At this time, in the intermediate layer of the parent model LM, the feature representation of the node corresponding to the sensor device is obtained as training information for distillation. That is, in the process in which the parent model LM performs calculations on a spherical graph structure Sg, the intermediate feature vector of the node corresponding to the target sensor device is used as training information for distillation from among the intermediate feature vectors generated for each node. Then, the parameters of the first neural network NN1 are updated so that the difference between the feature vector output from the output layer of the first neural network NN1 and the above training information for distillation is minimized. As a result, the first neural network NN1 can represent latent feature vectors in a latent representation space common to the parent model LM.

[0105] Similarly, in the lightweighting process for the second neural network NN2, an integrated feature representation of a group of nodes corresponding to a predetermined part of the robot body 30 is obtained as training information for distillation during the inference process of the parent model LM, and the second neural network NN2 is trained based on this training information. In the lightweighting process for the third neural network NN3, an integrated feature representation of a group of nodes corresponding to the entire body of the robot body 30, i.e., a graph-level feature vector generated by the readout process, is obtained as training information for distillation during the inference process of the parent model LM, and the third neural network NN3 is trained based on this training information. In the lightweighting process for the fourth neural network NN4, in the output layer after the readout process of the parent model LM, an intermediate feature vector of the process corresponding to the distribution from the first instruction information to individual instruction information corresponding to each of the multiple actuator devices 300 to 305 is obtained as training information for distillation, and the fourth neural network NN4 is trained based on this training information. In the lightweighting process of the fifth neural network NN5, feature representations related to the generation of control signals corresponding to each actuator device 300-305 are acquired as training information for distillation at the final stage of the output layer of the parent model LM, and the fifth neural network NN5 is trained based on this training information. In other words, the parent model LM is configured to be able to perform a series of processes end-to-end, from processing the detection signals of sensor devices 50(1)-50(n) to generating control signals for each actuator device 300-305, and the first to fifth neural networks NN1-NN5 are configured to be able to represent latent feature vectors in a latent representation space common to the parent model LM by distilling and learning the internal feature representations of the parent model LM corresponding to each stage of this series of processes as training information.

[0106] (Effects of this embodiment) With the above configuration, the following effects are achieved in this embodiment.

[0107] The higher-level control device 110 uses a basic neural network NN0 with a spherical graph structure Sg to process detection signals from multiple sensor devices 50(1) to 50(n) on a graph structure Sg that reflects their spatial relationships. This allows for more appropriate control of the actuator devices 300 to 305. As a result, the robot body 30 can be operated in a more versatile manner.

[0108] Furthermore, since the basic neural network NN0 and the first to fifth neural networks NN1 to NN5 are configured to generate feature representations in a common latent representation space L, information transfer between different neural networks is efficiently performed. This enables consistent feature-based processing even in a hierarchical processing structure, making it possible to operate the robot body 30 in a more versatile manner.

[0109] Furthermore, the first to fifth neural networks NN1 to NN5 are constructed using lightweight processing such as distillation learning, which uses the internal feature representations of the basic neural network NN0 as training information. Therefore, they can inherit the capabilities of the basic neural network NN0 while making the processing in each device lighter and faster.

[0110] Furthermore, since the entire set of trained weight parameters of the basic neural network NN0 functions as a world model that distributes information about the body structure of the robot body 30, the contactable range of each sensor part, and the observable range of each external sensor part, it becomes possible to control that directly reflects relationships in three-dimensional space without relying on one-dimensional representations using language.

[0111] <Other Embodiments> Furthermore, the above embodiment can also be implemented with the following modifications.

[0112] In the above embodiment, the robot device 20 is exemplified as being of the humanoid type, but the robot device 20 is not limited to the humanoid type. For example, it may be a quadrupedal type, a wheeled type, or a drone type robot device. In this case, the spherical graph structure Sg should be constructed based on an appropriate topological representation according to the shape of the robot body. For example, the graph structure Sg may be donut-shaped or the like.

[0113] In the above embodiment, the basic neural network is preferably a GNN, but the basic neural network is not limited to a GNN. For example, other architectures capable of representing relationships on a graph structure Sg, such as reservoir computing, may be used.

[0114] In the above embodiment, a hierarchical configuration comprising first intermediate processing units 90-95, a second intermediate processing unit 100, and a third intermediate processing unit 120 was illustrated, but the configuration of the robot device is not limited thereto. For example, the first detection information output from the signal processing units 70(1)-70(n) may be directly input to the higher-level control unit 110. Alternatively, the device may consist only of the first intermediate processing unit and not include the second intermediate processing unit.

[0115] In the above embodiment, a configuration comprising drive control devices 130 to 134 was illustrated, but a configuration in which actuator devices 300 to 305 are directly controlled based on first instruction information output from a higher-level control device 110 is also possible.

[0116] Even the above-mentioned examples, with appropriate design modifications by those skilled in the art, are included within the scope of this disclosure, as long as they possess the features of this disclosure. The elements, their arrangement, conditions, shapes, etc., of each of the above-mentioned examples are not limited to those exemplified and can be modified as appropriate. The elements of each of the above-mentioned examples can be combined in different ways as appropriate, as long as no technical inconsistencies arise. [Explanation of Symbols]

[0117] 20...Robot device, 30...Robot body, 40...Camera device, 41...Distance sensor, 42...Microphone device, 50(1)~50(n)...Sensor device, 70(1)~70(n)...Signal processing device, 90~95...First intermediate processing device, 100...Second intermediate processing device, 110...Higher-level control device, 120...Third intermediate processing device, 130~134...Drive control device, 300~305...Actuator device, Eg...Edge, L...Common latent representation space, LM...Parent model, Nd...Node, NN0...Basic neural network, NN1...First neural network, NN2...Second neural network, NN3...Third neural network, NN4...Fourth neural network, NN5...Fifth neural network, Sg...Graph structure.

Claims

1. Multiple sensor units that detect external conditions or the condition of the robot body and output detection signals corresponding to the detected external conditions or the condition of the robot body, An actuator unit that drives the movable part of the robot body, The robot comprises a control unit for controlling the robot body, The control unit, The system is configured to perform computational processing on a graph structure of a predetermined shape obtained by topologically transforming the external shape of the robot body using multiple nodes and multiple edges that indicate the connection relationships between each of the multiple nodes, and uses a basic neural network in which the state of the multiple nodes is sequentially updated as the feature quantities input to a predetermined node are reflected in other nodes connected to that predetermined node via the edges. The actuator unit is driven based on information obtained by inputting the detection signals from each of the multiple sensor units, or information generated from the detection signals from each of the multiple sensor units, into the multiple nodes of the basic neural network. Robotic device.

2. The aforementioned basic neural network is a graph neural network. The robotic apparatus according to claim 1.

3. The plurality of sensor units include at least one of a tactile sensor provided on the surface of the robot body to detect the state of contact with an object, and a temperature sensor to detect the temperature or heat transfer state of the object. The robotic apparatus according to claim 1.

4. The system further comprises multiple signal processing units that generate first detection information from the detection signals of each of the multiple sensor units using a first neural network, The control unit uses, as information generated from the detection signals of each of the multiple sensor units, a plurality of first detection pieces of information generated by the plurality of signal processing units, or information generated from the plurality of first detection pieces of information. The first neural network is configured to generate feature representations in a latent representation space shared with the basic neural network. The robotic apparatus according to claim 1.

5. Multiple signal processing units that generate first detection information from the detection signals of each of the multiple sensor units using a first neural network, The system further includes a first intermediate processing unit that converts the multiple first detection information outputs from each of the multiple signal processing units into a second detection information by a second neural network, The control unit uses the second detection information generated by the first intermediate processing unit, or the information generated from the second detection information, as information generated from the detection signals of each of the plurality of sensor units. The first neural network and the second neural network are configured to generate feature representations in a latent representation space common to the basic neural network. The robotic apparatus according to claim 1.

6. Multiple signal processing units that generate first detection information from the detection signals of each of the multiple sensor units using a first neural network, Multiple first intermediate processing units that convert the multiple first detection information outputs from each of the multiple signal processing units into a second detection information integrated therefrom using a second neural network, The system further comprises a second intermediate processing unit that converts the second detection information output from each of the plurality of first intermediate processing units into a third detection information formed by integrating them, using a third neural network. The control unit uses the third detection information generated by the second intermediate processing unit as information generated from the detection signals of each of the plurality of sensor units, The first neural network, the second neural network, and the third neural network are configured to generate feature representations in a latent representation space common to the basic neural network. The robotic apparatus according to claim 1.

7. The control unit generates first instruction information that instructs the robot body to move by inputting the detection signals from each of the multiple sensor units, or information generated from the detection signals from each of the multiple sensor units, to the multiple nodes of the basic neural network. The system further includes a drive control unit that generates a control signal by inputting the first instruction information, or information generated based on the first instruction information, into a fifth neural network, and controls the drive of the actuator unit based on the control signal. The fifth neural network is configured to generate feature representations in a latent representation space shared with the basic neural network. The robotic apparatus according to claim 1.

8. The control unit generates first instruction information that instructs the robot body to move by inputting the detection signals from each of the multiple sensor units, or information generated from the detection signals from each of the multiple sensor units, to the multiple nodes of the basic neural network. Multiple actuator units, A third intermediate processing unit that inputs the first instruction information to a fourth neural network to generate a plurality of second instruction pieces corresponding to each of the plurality of actuator pieces, The system further comprises a drive control unit that generates a control signal by inputting the second instruction information into a fifth neural network and controls the drive of the actuator based on the control signal, The fifth neural network and the fourth neural network are configured to generate feature representations in a latent representation space common to the basic neural network. The robotic apparatus according to claim 1.

9. The predetermined shape obtained by topologically transforming the external shape of the robot body is spherical. The robotic apparatus according to claim 1.

10. The predetermined shape obtained by topologically transforming the external shape of the robot body is spherical. The aforementioned spherical graph structure is constructed by arranging a plurality of nodes on a spherical surface obtained by mapping the external shape of the robot body to a spherical shape using topological transformation. The aforementioned plurality of sensor units are distributed and arranged on the surface of the robot body. Each of the aforementioned nodes is associated with one of the aforementioned sensor units based on the position obtained by mapping the arrangement positions of the aforementioned sensor units on the surface of the robot body onto the spherical surface using the topological transformation. The robotic apparatus according to claim 1.

11. The predetermined shape obtained by topologically transforming the external shape of the robot body is spherical. The robot body is further equipped with an external sensor unit for detecting the surrounding environment, The control unit integrates the external detection information acquired by the external sensor into the feature quantities of each of the multiple nodes in the spherical graph structure, and then performs computational processing using the basic neural network. The robotic apparatus according to claim 1.

12. The predetermined shape obtained by topologically transforming the external shape of the robot body is spherical. The control unit generates a simulation space that virtually reproduces the robot body and the surrounding environment based on the shape data of the robot body and the external detection information acquired by the external sensor unit. Based on the spatial correspondence between the surface of the robot body in the simulation space and the plurality of nodes in the spherical graph structure, the external detection information is integrated into the feature quantities of each of the plurality of nodes. The robotic apparatus according to claim 11.

13. The predetermined shape obtained by topologically transforming the external shape of the robot body is spherical. The basic neural network is configured to generate intermediate feature vectors for each of the multiple nodes during the process of performing computational processing on the spherical graph structure. The first neural network is constructed using distillation learning, where the intermediate feature vectors generated for the nodes associated with the corresponding sensor units during the process in which the basic neural network performs computational processing on the spherical graph structure are used as training information. The robotic apparatus according to claim 4.

14. The basic neural network is configured to perform a series of processes, from processing detection signals from the multiple sensor units to generating control signals for the actuator units. The first neural network, the second neural network, and the third neural network are each constructed by distillation learning, using the internal feature representations of the basic neural network corresponding to each stage of the series of processes as training information. The robotic apparatus according to claim 6.

15. The basic neural network is configured to perform a series of processes, from processing detection signals from the multiple sensor units to generating control signals for the actuator units. The fourth neural network and the fifth neural network are each constructed by distillation learning, using the internal feature representations of the basic neural network corresponding to each stage of the series of processes as training information. The robotic apparatus according to claim 8.

16. The aforementioned basic neural network has pre-trained weight parameters, Within the entirety of the learned weight parameters, information regarding the body structure of the robot body and information regarding the contactable range of the multiple sensor parts are distributed and encoded. The robotic apparatus according to claim 1.

17. The robot body is further equipped with an external sensor unit for detecting the surrounding environment, The information regarding the observable range of the external sensor is further distributed and encoded within the entire set of the learned weight parameters. The robotic apparatus according to claim 16.

Images