Human augmentation platform device and physical ability augmentation method

Abstract

This human augmentation platform device 100 comprises: a physical information acquisition unit 110 that acquires physical information indicating physical abilities of a sensing side and a controlled side; a motion data acquisition unit 120 that acquires motion data on the sensing side; a control data generation unit 150 that generates control data for the controlled side on the basis of the physical information acquired by the physical information acquisition unit 110 and the motion data acquired by the motion data acquisition unit 120; and an actuation unit 160 that actuates the body of the controlled side on the basis of the control data. The control data generation unit 150 generates control data within the range of the physical ability of the controlled side.

Description

[Technical field]

[0001] The present invention relates to a human augmentation platform device and a method for augmenting human physical abilities that support the augmentation of human physical abilities. [Background technology]

[0002] The 3rd Generation Partnership Project (3GPP) has developed specifications for the 5th generation mobile communication system (5G, also known as New Radio (NR) or Next Generation (NG)), and is also developing specifications for the next generation, known as Beyond 5G, 5G Evolution, or 6G.

[0003] One of the technologies to watch in the 6G era is human augmentation. Human augmentation can be broadly divided into "augmentation of physical ability," "augmentation of existence," "augmentation of perception," and "augmentation of cognitive ability."

[0004] Among these, the expansion of physical abilities is an approach that mainly involves sensing motion data such as electroencephalograms and electromyograms from the human brain and muscles, and then activating actual muscles and exoskeletons. This can realize prosthetics for lost physical abilities, the improvement of existing abilities, and the acquisition of new abilities. Power-assisted suits, artificial limbs, and third arms (robot arms) are being considered (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] NTT Docomo, "Docomo 6G White Paper 4.0 Edition", [online], November 2021, Internet <URL:https: / / www.nttdocomo.co.jp / binary / pdf / corporate / technology / whitepaper_6g / DOCOMO_6G_White_PaperJP_20211108.pdf> Summary of the Invention

[0006] The above-mentioned expansion of physical abilities has the following problems. Specifically, there is a possibility that differences in physical abilities, such as the range of motion, may occur between the sensing side and the expansion side (controlled side). In addition, there are also differences in the capabilities (performance) of the devices used for sensing and expansion (control).

[0007] For this reason, if the body movement data on the sensing side is directly reflected on the extended side, such as a human or robot, there is a problem that undue strain is placed on the extended side or the physical abilities of the extended side cannot be fully utilized.

[0008] Therefore, the following disclosure has been made in consideration of the above circumstances, and aims to provide a human augmentation platform device and a physical ability augmentation method that can improve the interoperability between the sensing side and the augmentation side (controlled side).

[0009] One aspect of the present disclosure is a human augmented platform device (human augmented platform device 100) that includes a physical information acquisition unit (physical information acquisition unit 110) that acquires physical information indicating physical abilities of the sensing side and the controlled side, a motion data acquisition unit (motion data acquisition unit 120) that acquires motion data of the sensing side, a control data generation unit (control data generation unit 150) that generates control data for the controlled side based on the physical information acquired by the physical information acquisition unit and the motion data acquired by the motion data acquisition unit, and an actuation unit (actuation unit 160) that actuates the body of the controlled side based on the control data, and the control data generation unit generates the control data that is within the range of the physical abilities of the controlled side.

[0010] One aspect of the present disclosure is a physical ability expansion method including the steps of acquiring physical information indicating the physical abilities of a sensing side and a controlled side, acquiring motion data of the sensing side, generating control data for the controlled side based on the physical information and the motion data, and operating the body of the controlled side based on the control data, wherein in the step of generating control data, the control data is generated to be within the range of the physical abilities of the controlled side. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram of the entire configuration of a human augmentation system 10 according to the first embodiment. [Diagram 2] FIG. 2 is a functional block configuration diagram of the human augmented platform device 100 according to the first embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of the configuration of the physical information DB according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the action DB according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating a schematic operation example of the human augmented platform device 100 according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing a schematic operation flow of the human augmented platform device 100 according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of an expansion operation of a physical ability between a sensing side (body A) and a controlled side (body B) according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a system configuration including the human augmented platform device 100 according to the second embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a designated sensing / actuation pattern according to the second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a sequence relating to account registration and service subscription according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing a sequence example (part 1) regarding body data registration according to the second embodiment. [Figure 12A] FIG. 12A is a diagram showing a sequence example (part 2) regarding body data registration according to the second embodiment. [Figure 12B] FIG. 12B is a diagram showing a sequence example (part 3) regarding body data registration according to the second embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a real-time data transmission / reception sequence by the service management function and the data linkage control function according to the second embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a sequence of skill data accumulation according to the second embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a sequence of service registration according to the second embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of the configuration of a data conversion element by a data linkage control function according to the second embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of option mapping according to the second embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a hardware configuration of the human augmentation platform device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment will be described with reference to the drawings. Note that the same or similar reference numerals are used to designate the same functions or configurations, and the description thereof will be omitted as appropriate.

[0013] [First embodiment] (1) Overall configuration of the human augmentation platform device Fig. 1 is a schematic diagram of the entire configuration of a human augmentation system 10 according to this embodiment. As shown in Fig. 1, the human augmentation system 10 includes a sensing side configured with a group of multiple sensors with different measurement targets, and an augmented side (which may also be called the controlled side or actuation side) in which a robot or a human being is the control target.

[0014] The human augmentation platform device 100 is connected to the sensing side and the augmentation side. The human augmentation platform device 100 may also be connected to a communication network 20.

[0015] In this embodiment, the human augmentation platform device 100 is connected to an electromyogram sensor 40 and an electroencephalogram sensor 50. The human augmentation platform device 100 is also connected to a robot 60 and an electromyogram actuator 70. Furthermore, the human augmentation platform device 100 may be connected to a tactile sensor 55.

[0016] The communication network 20 may include a wired network and a wireless network. The human augmentation platform device 100 may be able to access other systems, databases, application services, and the like via the communication network 20.

[0017] The myoelectric sensor 40 is configured so as to be attached to at least any of the four limbs of a human. Typically, the myoelectric sensor 40 can be attached to a human's upper limb (arm) or lower limb (leg). However, the subject to which the myoelectric sensor 40 is attached is not necessarily limited to a human. The myoelectric sensor 40 is a sensor that detects an electric signal generated during muscle contraction, and a general myoelectric (muscle potential) sensor may be used.

[0018] The electroencephalogram sensor 50 is attached to the human head. The electroencephalogram sensor 50 is a sensor that detects potential fluctuations generated from the human brain, and a general electroencephalogram sensor may be used.

[0019] The tactile sensor 55 is a sensor that can convert the tactile sensation felt by a human being when he or she comes into contact with an object into digital data. Typically, the tactile sensor 55 can convert the tactile sensation of a human hand or foot that comes into contact with the object into digital data.

[0020] The robot 60 is a humanoid mechanical device that operates based on a specified electrical signal. The robot 60 has at least one of upper limbs (arms) and lower limbs (legs) like a human. The robot 60 is not limited to arms (legs) and may include other parts such as fingers.

[0021] The myoelectric actuator 70 is configured so as to be attached to at least any of the four limbs of a human. Typically, the myoelectric actuator 70 can be attached to a human upper limb (arm) or lower limb (leg). However, similar to the myoelectric sensor 40, the subject to which the myoelectric actuator 70 is attached is not necessarily limited to a human. The myoelectric actuator 70 can generate an electric signal and can contract the muscles of a human finger, arm, etc.

[0022] The human augmentation platform device 100 is connected to the sensing side and the augmented side (controlled side) to realize human augmentation. Specifically, the human augmentation platform device 100 realizes the augmentation of physical abilities.

[0023] The expansion of physical abilities is an approach that mainly senses motion data such as electroencephalograms and electromyograms from the human brain and muscles, and then actuates actual muscles and exoskeletons. The human augmentation platform device 100 acquires motion data of the sensing body (body A) via the above-mentioned sensor group.

[0024] The human augmented platform device 100 converts the acquired motion data into control data using a conversion algorithm. The human augmented platform device 100 actuates the controlled body (assumed to be body B) based on the control data generated by converting the motion data. By executing such processing by the human augmented platform device 100, the motion of body A can be reflected in body B. Note that body B (or body A) is not limited to a human being and may include a robot.

[0025] The data (range) handled by the human augmentation platform device 100 is mainly myoelectric (electromyogram), but may also include electroencephalogram, bioacoustic wave, motion (image), etc. In the following, myoelectricity will be described as an example.

[0026] (2) Functional block configuration of the human augmentation platform device 2 is a functional block diagram of the human augmented platform device 100. As shown in FIG. 2, the human augmented platform device 100 includes a physical information acquisition unit 110, a motion data acquisition unit 120, a physical information DB 130, a motion DB 140, a control data generation unit 150, and an actuation unit 160.

[0027] It should be noted that Fig. 2 shows only main functional blocks related to the description of the embodiment, and the human augmentation platform device 100 has other functional blocks (e.g., a power supply unit, etc.). Fig. 2 shows the functional block configuration of the human augmentation platform device 100, and for the hardware configuration, please refer to Fig. 8.

[0028] The physical information acquiring unit 110 acquires physical information indicating the physical abilities of the sensing side and the controlled side. For example, the physical information acquiring unit 110 can acquire the movable range of the arms of the human (or robot) on the sensing side and the controlled side as the physical abilities. Note that the physical abilities are not limited to the arms, and may be the fingers, legs, etc. Furthermore, the movable range only needs to be controllable by the myoelectric actuator 70, and may be the range of bending and stretching based on the joints, or the range of twisting.

[0029] The physical information acquiring unit 110 may acquire the physical information via the communication network 20, or may acquire the physical information directly input to the human augmentation platform device 100. The acquired physical information is stored in the physical information DB 130.

[0030] The motion data acquisition unit 120 acquires motion data on the sensing side. Specifically, the motion data acquisition unit 120 acquires body movements on the sensing side via an electromyogram sensor 40 (an electroencephalogram sensor 50 may also be used; the same applies below).

[0031] For example, the motion data acquiring unit 120 can acquire motion data (such as the flexion and extension range based on the joint) of the arm of the person to which the myoelectric sensor 40 is attached. Specifically, the motion data acquiring unit 120 can generate motion data of the body part to which the myoelectric sensor 40 is attached based on the electrical signal output from the myoelectric sensor 40. The acquired motion data is stored in the motion DB 140.

[0032] When the sensing side is a robot, the motion data acquiring section 120 may use motion (images) instead of the myoelectric sensor 40 to generate motion data.

[0033] The physical information DB 130 stores the physical information acquired by the physical information acquisition unit 110. Specifically, the physical information DB 130 can store the physical information of each human (or robot) that is the target of human augmentation.

[0034] FIG. 3 shows an example of the configuration of a physical information DB. An ID is identification information for each person (or robot). As shown in FIG. 3, the physical information DB may be configured by body parts, range of motion, etc. Note that FIG. 3 shows an example of a physical information DB, and may also include physical abilities other than the range of motion, such as the amount of weight that can be handled.

[0035] The motion DB 140 stores the motion data acquired by the motion data acquisition unit 120. Specifically, the motion DB 140 can store motion data in which the motion of a human (or robot) is digitized in time series.

[0036] FIG. 4 shows an example of the configuration of the motion DB. As with the physical information DB, the ID is identification information for each person (or robot). As shown in FIG. 4, the motion DB may be configured by body parts, motion ranges indicating the ranges actually moved, and the like. Note that FIG. 4 shows an example of the motion DB, and may also include information other than the motion range, such as motion speed.

[0037] The control data generating unit 150 generates control data for the controlled side based on the physical information acquired by the physical information acquiring unit 110 and the motion data acquired by the motion data acquiring unit 120. Specifically, the control data generating unit 150 generates control data based on the physical information of the humans (or robots) on the sensing side and the controlled side and the motion data generated from the electrical signals acquired via the electromyography sensor 40.

[0038] More specifically, the control data generation unit 150 acquires physical information of the sensing side body A and the controlled side body B from the physical information DB 130. In addition, the control data generation unit 150 acquires motion data of the body A from the motion DB 140. The control data generation unit 150 can determine a conversion algorithm for generating control data based on the acquired physical information and motion data.

[0039] Here, the control data generating unit 150 can generate control data that falls within the range of the physical ability of the controlled side (body B). As a mechanism for converting motion data into control data, there are a number of methods, such as proportional interpolation of the body's movable range or the minimum and maximum values ​​of the ability of the myoelectric actuator 70, and control within the range of allowable values ​​(stopping at the maximum value).

[0040] For example, the control data generating unit 150 may generate control data in which the physical ability of the controlled side is proportional to the physical ability of the sensing side based on a comparison result between the range of the physical ability of the sensing side (body A) and the range of the physical ability of the controlled side (body B). In other words, even if there is a difference in the physical ability (e.g., range of motion) between body A and body B, control data for body B may be generated that is proportional to the motion data of body A while not exceeding the range of the physical ability of body B. Specific control examples will be described later.

[0041] Alternatively, the control data generating unit 150 may generate the control data based on the allowable value of the physical ability of the controlled party (body B). For example, the maximum value of the movable range of the arm of body B may be defined as the allowable value, and the control data may be generated so that the movable range is equal to or less than the allowable value.

[0042] Furthermore, when the part of the limb on the sensing side (body A) is different from the part of the limb on the controlled side (body B), the control data generating unit 150 can convert the motion data of the part on the sensing side into motion data of the part on the controlled side. The control data generating unit 150 can generate control data based on the body information of body A and body B and the converted motion data. In other words, in the human augmentation platform device 100, in order to actuate not only the same body part but also different body parts (for example, an arm on the sensing side and a leg on the controlled side), control data converted into the movement of the relevant part may be generated.

[0043] When the sensing side (body A) is a robot and the controlled side (body B) is a human, the control data generator 150 may generate control data in which the motion speed passes through an accelerating state, a constant state, and a decelerating state. Since the motion of a robot (motor) differs from that of a human, the motion speed is changed according to the motion of the human.

[0044] In addition, when the upper or lower limbs of the sensing side (body A) are either the left or right (e.g., the right arm) and the upper or lower limbs of the controlled side (body B) are the other (e.g., the left arm), the control data generation unit 150 may convert the range of motion of the sensing side to the range of motion of the controlled side and generate control data.

[0045] For example, if two myoelectric sensors 40 are attached to the left arm of body A and to a body A' different from body A, and two myoelectric actuators 70 are attached to the left arm and right arm of body B, control data for both arms of body B may be generated based on physical information of body A and body A', motion data of body A and body A', and physical information of body B.

[0046] Here, when the movable areas of the left arm and the right arm of the body B, that is, the motion ranges based on the motion data of each body part (the left arm and the right arm) overlap, the control data generating unit 150 may determine whether the left arm and the right arm will come into contact. Furthermore, when the control data generating unit 150 determines that the left arm and the right arm will come into contact, it may generate control data for avoiding the contact. The control data for avoidance may simply stop the motion before the contact, or may change the motion range or the motion timing so as to avoid the contact.

[0047] In addition, when information on both arms of the same body is stored in the physical information DB 130, the movable area of ​​the controlled side may use the stored information, or the movable area of ​​one arm (sensing side) may be considered to be the same as the movable area of ​​the other arm (controlled side) and may be converted into control data for the other arm. Note that the same process may be applied not only to the arms but also to both legs, etc.

[0048] It is preferable that the control data generating unit 150 executes real-time processing rather than retransmission control in order to deal with differences in resolution due to the capabilities of devices (robots, myoelectric actuators, etc.) or data loss due to communication errors.

[0049] In addition, when a new device is connected to the human augmentation platform device 100, the control data generation unit 150 may acquire base data for the device, determine a conversion algorithm for the device based on the acquired physical information, and generate control data for the device.

[0050] The actuation unit 160 operates the body of the controlled side (body B) based on the control data generated by the control data generation unit 150. Specifically, the actuation unit 160 controls the robot 60 or the myoelectric actuator 70 (in the case of a human) based on the control data.

[0051] Depending on the capabilities of the myoelectric actuator 70, any coefficient (such as a multiple) may be applied to the displacement, force, speed, and the like indicated by the motion data acquired by the motion data acquisition unit 120 for conversion.

[0052] Furthermore, the actuation section 160 may perform calibration for each combination of a sensor device (eg, an electromyogram sensor) and an actuation device (an electromyogram actuator).

[0053] (3) Operation of the Human Augmentation Platform Device Next, we will explain an operation example of the human augmented platform device 100. Fig. 5 shows a schematic operation example of the human augmented platform device 100. In the operation example shown in Fig. 5, the sensing side (body A) is the right arm of a human, and the controlled side (body B) is the right arm of a human or a robot.

[0054] Fig. 6 shows a schematic operation flow of the human augmentation platform device 100. Fig. 7 shows an example of an augmentation operation of physical ability between the sensing side (body A) and the controlled side (body B).

[0055] As shown in Fig. 6, the human augmented platform device 100 acquires physical information of the sensing side and the controlled side (S10). Specifically, the human augmented platform device 100 acquires physical information of body A and body B. Here, it is assumed that body A is assigned an ID of "001" and body B is assigned an ID of "002" (see Fig. 7).

[0056] More specifically, the human augmentation platform device 100 acquires the physical information of body A and body B stored in the physical information DB 130 (see FIG. 3).

[0057] Next, the human augmented platform device 100 compares the physical information of the body A (ID: 001) with the physical information of the body B (ID: 002) (S20). The right arm of the body A (ID: 001) can move in the range of 0° to 90° (see FIG. 3). The right arm of the body B (ID: 002) can move in the range of 10° to 80° (see FIG. 3). The human augmented platform device 100 compares such physical information on the sensing side with the physical information on the controlled side.

[0058] The human augmented platform device 100 acquires motion data associated with the movement of the body A (right arm) (S30). Specifically, the human augmented platform device 100 acquires motion data indicating that the right arm of the body A to which the myoelectric sensor 40 is attached has moved in the range of 60° to 90° (see FIG. 7). Note that 0° may be the horizontal direction, and 90° may be the vertical (perpendicular) direction. Also, the process of S30 may be executed prior to the process of S20.

[0059] The human augmented platform device 100 executes processing for operating the body B of the controlled side (S40). Specifically, the human augmented platform device 100 generates control data for operating the body B based on the comparison result between the body information of the sensing side and the body information of the controlled side and the motion data of the body A.

[0060] Specifically, the human augmentation platform device 100 generates control data for body B based on physical information of body A (movable in the range of 0° to 90°), physical information of body B (movable in the range of 10° to 80°), and motion data indicating that the right arm of body A moved in the range of 60° to 90° (i.e., 30°).

[0061] In this operation example, the control data for body B may be control data for myoelectric actuator 70 that moves the human right arm by 30°, or may be control data for a robot that moves the robot's right arm by 30°.

[0062] Alternatively, control data proportional to the physical ability of the controlled side may be generated based on the comparison result between the physical information of the sensing side and the physical information of the controlled side. For example, since the motion range / movable range of body A is (90-60) / (90-0) ≒ 0.33, control data may be generated to set the motion range of body B to (80-10) * 0.33 ≒ 23° (see FIG. 7).

[0063] (4) Actions and Effects According to the above-described embodiment, the following effects can be obtained. Specifically, the human augmented platform device 100 can generate control data for body B based on the body information of the sensing side (body A) and the controlled side (body B) and the motion data of body A. At this time, the human augmented platform device 100 can generate control data that is within the range of the physical ability of body B.

[0064] Therefore, even if there is a difference in physical ability, such as the range of arm movement, between the sensing side (body A) and the controlled side (body B), it is possible to significantly prevent problems such as excessive strain on the augmented side or inability to fully utilize the physical ability of the augmented side. In other words, the human augmentation platform device 100 can improve the mutual cooperation between the sensing side and the augmented side (controlled side).

[0065] For example, if the motion data of body A is directly reflected in body B, there is a possibility that a movement exceeding the tolerance of body B may be reflected due to differences in the range of motion or ability, which may cause a large burden on body B. There may also be differences in the capabilities (performance) of the devices used for sensing and augmentation (control). According to the human augmentation platform device 100, even when there are such differences in physical ability or device capabilities, appropriate control data can be generated, and smooth and appropriate augmentation of physical ability can be realized.

[0066] In this embodiment, the human augmentation platform device 100 can generate control data in which the physical ability of the controlled side is proportional to the physical ability of the sensing side based on the comparison result between the range of physical ability of the sensing side (body A) and the range of physical ability of the controlled side (body B). Therefore, even if there is a difference in physical ability between the sensing side and the controlled side, smooth and appropriate expansion of physical ability can be realized.

[0067] Furthermore, the human augmentation platform device 100 can generate control data based on the allowable value of the physical ability of the controlled side. Therefore, it is possible to realize the augmentation of physical ability without placing an excessive burden on the controlled side.

[0068] In this embodiment, when the part of the limb on the sensing side is different from the part of the limb on the controlled side, the motion data of the part on the sensing side can be converted into the motion data of the part on the controlled side, which allows for more flexible expansion of physical abilities.

[0069] In this embodiment, when the sensing side is a robot and the controlled side is a human, the human augmentation platform device 100 can generate control data in which the motion speed passes through an accelerating state, a constant state, and a decelerating state. This makes it possible to realize the augmentation of physical abilities in accordance with the actual movements of humans.

[0070] [Second embodiment] In this embodiment, the basic functions of the human augmented platform device 100 according to the first embodiment are further expanded, and an operation panel (user interface (UI)) is integrated with the human augmented platform device 100 (human augmented platform server). Here, the operation panel may be interpreted as a set of hardware and software that allows the user to specify sensing / actuation for data transmission and to start / stop data transmission.

[0071] In this embodiment, data transmission from sensing to actuation can be performed independently and simultaneously. The icons (devices) displayed on the operation panel can also be changed. In addition, the human augmentation platform device 100 according to this embodiment has particularly enhanced service management and data linkage control functions.

[0072] (1) System Configuration Fig. 8 shows an example of a system configuration including the human augmented platform device 100 according to this embodiment. As shown in Fig. 8, the human augmented platform device 100 is roughly divided into a service management function and a data linkage control function. In addition, the human augmented platform device 100 is connected to a service user interface (UI) and a body application corresponding to user A and user B.

[0073] Below, an overview of the functions provided by the human augmentation platform device 100 according to this embodiment and the components of the functions will be described.

[0074] (1.1) Data linkage control function This is the part that handles ubiquitous human data. It has an interface with sensing / actuation, and can receive body data, receive sensor data from sensing devices, and send it to actuation devices. It also has the functions of storing and playing back skill data, comparing bodies, and converting data.

[0075] (1.2) Body Comparison To eliminate the difference between the sensing side body and the actuation side (controlled side) body, the two body data are compared. Comparison methods include, for example, calculating the ratio of the actuation side / sensing side (B / A), or setting an upper limit that does not exceed the movable range of the actuation side body. A mechanism is provided for setting the comparison method according to the use case. Such body comparison can provide a fail-safe mechanism that takes the actuation side into consideration.

[0076] (1.3) Data Conversion The sensor data transmitted from the sensing side is subjected to magnification conversion, conversion of the target part (for example, from right hand to left hand), etc. The data conversion process is executed in the human augmentation platform device 100, and the converted result is sent to the actuation side. A mechanism is provided that allows the conversion method to be set according to the use case. This makes it possible to provide a mechanism that allows the service provider to specify the magnification and conversion method of the information to be transmitted while taking into consideration the actuation side.

[0077] (1.4) Skill data storage and playback By managing data for each user, it will be possible to handle the accumulation and playback of sensor data. Past sensor data will be managed in an operation database, and a function will be provided to select and play back past data along with a UI for selecting data from the database. This will provide a mechanism for specifying one of the accumulated skills and playing it back.

[0078] (1.5) Choice Mapping (EEG) In order to allow multiple choice selection (e.g., four or eight options) using brainwaves, a mechanism is provided that maps the options to the actions corresponding to the options. Information on the sensing side's actions (icon images, text, etc. indicating the action content) is presented, and the option (number) received from the sensing side is converted into the corresponding skill data, which is then sent to the actuation side. This makes it possible to provide a mechanism for mapping actions by selecting using images, for example.

[0079] (1.6) Service Management Function In addition to the data linkage control function, there are services that can link, compare, and convert data, and a management function for those services. This allows you to set data handling and conversion specifications for each service. Furthermore, services can handle data as if it were exclusive and independent of each other.

[0080] (1.7) Account Management The human augmentation platform device 100 can accumulate data for each user. For this purpose, a user identifier (account) is provided. The accounts belong to one of the following categories: (1) platform administrator, (2) service administrator, or (3) general user, and access rights are managed. In addition, an account management function is provided for creating, editing, and deleting accounts.

[0081] (1.8) Device Management A device management function is provided to manage the types of devices permitted to connect to the human augmented platform device 100. This function provides a mechanism for controlling whether or not each type of device can connect to the human augmented platform device 100. This makes it possible to connect only permitted devices to the human augmented platform device 100.

[0082] (1.9) Use case settings A mechanism is provided for setting the data transfer method (sensing / actuation designation), body comparison method, and data conversion method for each service. This enables independent data linkage control for each service.

[0083] (1.10) Service Management It is possible to set the users who can use the service, the devices that can be connected, the body application (hereinafter abbreviated as "app"), the body comparison method, and the data conversion method for each service individually. In addition, each service has a UI and is provided with functions such as specifying sensing / actuation, setting the body comparison method and the data conversion method, setting the accumulation and playback of skill data, and instructing the start / end of data transmission. This allows the contents of sensing / actuation to be set for each service to be provided.

[0084] (1.11) Body data and sensor data The human augmentation platform device 100 handles the following human-related data. Such data may be interpreted as types of human-related data that can be transmitted to actuators.

[0085] · Joint angles and lengths of each part of the body: Data on the joint angles and lengths of each part of the body Data on upper limb joint angles, trunk and lower limb joint angles, and upper and lower limb lengths could be expanded.

[0086] Grip strength: Data on grip strength when holding hands and feet A definition of foot grip strength could be added (for use in walking, etc.).

[0087] -Tactile: Digitalizing human touch and making it possible to handle it Tactile data definitions for the hands and feet may be provided.

[0088] Emotions: Human emotions are made digital and made manageable. Emotions (level of tension) are made digital and made manageable.

[0089] (1,12)SDK We provide an SDK (development kit) for device vendors that can reduce the difficulty of connecting to the human augmentation platform device 100. It is preferable that the SDK is obfuscated (encrypted) to increase the difficulty of analyzing the program contents in order to prevent diversion to other purposes or tampering.

[0090] (1.13) Device control app reference code To make it easier for device vendors to develop device control apps, a template for an app that is combined with the SDK may be provided as reference code at the same time. Device vendors can usually create device control apps by adding a section for sending and receiving data to and from the device based on the reference code.

[0091] (1.14) Test suite A test suite is provided as a test environment (for verifying test scenarios and test results) in the development of device control apps.

[0092] The scenario creation allows for testing by preparing sensor data in text format (JSON) in advance and sending the sensor data to the actuator. By testing various data sets in sequence, tests of normal, quasi-normal, maximum, minimum, and abnormal values ​​can be automated. Evidence of the test results is output as a log, and the suitability of the developed device control app for the human augmentation platform device 100 can be determined by aggregating the logs.

[0093] (2) Details of data linkage control function and service management function Next, the above-mentioned data linkage control function and service management function (including components) will be described in detail.

[0094] (2.1) Data linkage control function The data linkage control function may include a function for receiving data from a sensing device, transmitting data to an actuation device, storing data received from the sensing device, and playing back the stored data to the actuation device.

[0095] The data linkage control function also has a function for executing body comparison and data conversion for data to be sent to the actuation device. The data linkage control function may send and receive (transmit) data according to instructions from the service. The instructions from the service may include designation of sensing / actuation required for data transmission, settings of body comparison method and data conversion method, settings of skill data storage and playback, etc.

[0096] The following data transmission methods may be provided:

[0097] - Real-time transmission (transmitting data received from the sensing side to the actuation side) Accumulation of skill data (receives and stores data from the sensing side) - Skill data playback (reading stored data and sending it to actuation) The data linkage control function allows multiple services to transmit data simultaneously and independently. Service providers can specify the magnification and conversion method of the information to be transmitted, taking into account the actuation side.

[0098] (2.2) Body Comparison In order to fill in the body difference between the sensing side and the actuation side, the two body data may be compared. The method of comparing the data may be configurable according to the use case.

[0099] The comparison method can be selected from, for example, "ratio calculation (B / A)" or "upper limit setting (do not exceed B)." Here, A: body data on the sensing side, and B: body data on the actuation side. Note that other body data comparison methods may also be applied. Which comparison method to apply may be specified by the service.

[0100] That is, in this embodiment, the control data generation unit 150 of the human augmentation platform device 100 can generate control data based on the ratio between physical information on the sensing side and physical information on the actuation side (controlled side).

[0101] The conversion of the operation data (motion data) based on the body comparison data may be performed in the human augmentation platform device 100.

[0102] An application program interface (API) may be provided for registering body data necessary for body comparison. Methods for inputting body data on the app side include numerical input by the user or input of data obtained from a sensing device.

[0103] The input body data is managed for each user in a body DB.

[0104] The body DB can manage the body ID, user, body data, registration date and time, etc. Table 1 shows an example of the body DB configuration.

[0105] [Table 1]

[0106] (2.3) Data Conversion Before transmitting the sensor data to the actuation side, conversion processing may be performed according to the use case. Depending on the use case or the format of the data to be handled, the following conversion methods may be provided.

[0107] Scalar conversion: Performs a magnification operation on one element of data Matrix transformation: Transforms parts and performs magnification calculations for multiple element data such as joint angles Frequency characteristic transformation: Performs emphasis calculations on specific frequency bands using the fast Fourier transform (FFT), emphasis transformation, and inverse fast Fourier transform (IFFT) on waveform data. The conversion method to be applied may be specified by the service.

[0108] (2.4) Skill data storage and playback The data transmitted from the sensing side may be stored, and the skill data selected from the stored skill data may be transmitted to the actuation side. The data transmitted from the sensing side is managed in an action DB 140 (which may be called an operation DB) for each user.

[0109] The action DB 140 may manage a history ID, a body ID, a date and time, and sensor data. In addition to the action DB 140, a skill DB 180 is provided that manages the contents of the accumulated data (user, date and time, content of the action, device, etc.) as skill data. Table 2 shows an example of the action DB configuration.

[0110] [Table 2]

[0111] Skill DB 180 may manage skill IDs, users, dates and times, device IDs, operation details, body IDs, history IDs, etc. (The substance of body data and sensor data is stored in physical information DB 130 (which may be called a body DB) and operation DB 140, respectively, and skill DB 180 manages IDs for referencing data.) Table 3 shows an example of a skill DB configuration.

[0112] [Table 3]

[0113] A screen for registering, editing, and deleting skill data, and a screen for displaying a list of registered skill data and selecting and playing the data may be provided as a function on the service side. In skill data registration, a method for inputting and registering skill content for accumulated sensor data, and a method for acquiring sensor data after inputting skill content may be provided.

[0114] Thus, in this embodiment, the human augmentation platform device 100 includes a skill DB 180 (skill data storage unit) that stores motion data acquired by a motion data acquisition unit (which may be a body application, a sensor device, etc.) as skill data organized based on predetermined characteristics, and the control data generation unit 150 can generate control data for playback based on the skill data.

[0115] (2.5) Choice Mapping (Electroencephalography) In order to perform multiple choice (e.g., four or eight options) based on electroencephalograms, a mechanism is provided for mapping options to actions corresponding to the options. The actions corresponding to the options may correspond to the reproduction of skill data and a transition to another multiple choice.

[0116] The information on the mapping may include options (numbers) for one-way selection, operation contents corresponding to the options, and images (icons) showing the operation contents. Information on the mapping of options may be returned in response to a request from the sensing side.

[0117] If the sensor data sent from the sensing side is an option of brainwaves, an action corresponding to the option may be executed. If the action is replay of skill data, the corresponding sensor data may be sent to the actuation side. If the action is transition to another mapping, new mapping information may be sent back to the sensing side.

[0118] Thus, in this embodiment, the human augmentation platform device 100 includes an option mapping unit 170 that presents options of control data to the user, and the option mapping unit 170 can identify skill data associated with an option selected by the user. Also, the control data generating unit 150 can generate control data for playback based on the identified skill data.

[0119] (2.6) Service Management and Use Case Settings Service management allows users to register, edit, and delete services. Information about services is managed in a service DB. Table 4 shows an example of the service DB configuration.

[0120] [Table 4]

[0121] The information about the service may include a service ID, a service name, information about users (service subscribers) who can use the service, a connectable device body application, a body comparison method, and a data conversion method. Each service may have an independent UI.

[0122] The UI of the service may provide functions such as specifying sensing / actuation, setting the body comparison method, data conversion method, and skill data storage and playback, and starting / ending data transmission. Each service can also manage users currently connected to the service. When the start of data transmission is selected on the UI, the service issues an instruction to the data linkage control function according to the set use case (sensing / actuation, body comparison method, data conversion method, etc.).

[0123] (2.7) Account Management It is possible to register, edit, and delete an account in order to identify the user when accessing the human augmentation platform device 100. As described above, the types of accounts may be (1) a platform administrator, (2) a service administrator, and (3) a general user.

[0124] Information about an account is managed in an account DB. Information about an account may include a user ID, a user name, and an account type. Table 5 shows an example of the account DB configuration.

[0125] [Table 5]

[0126] Existing platform administrators can register, edit, and delete accounts for platform administrators and service administrators on the administrator registration screen. General user accounts can be registered, edited, and deleted by users themselves on the general user screen. Accounts can also be edited and deleted on the administrator registration screen. When accessing the human augmentation platform device 100, it is necessary to log in by specifying a registered account.

[0127] (2.8) Device Management Device information can be registered, edited, and deleted to identify devices that are permitted to access the human augmentation platform 100 (platform). A combination of device (hardware) and body app (software) may be registered. This is because the same device may have multiple body apps with different functions.

[0128] The device-related information is managed in a device DB. Table 6 shows an example of the device DB configuration.

[0129] [Table 6]

[0130] The data regarding the device may include a device ID, a device name, a body application name, a type (sensing or actuation), and an API key. Device management may be permitted only to the platform administrator. An API key may be issued when the device information is registered.

[0131] The body app sets an API key when accessing the platform, and the platform recognizes that the device is authorized to access using the API key. The platform administrator notifies the body app developer of the issued API key, and the body app developer sets the notified API key in the body app.

[0132] (2.9) Body Data / Sensor Data The body data may be interpreted as data related to the user's body that needs to be compared with other bodies. The sensor data may be interpreted as data for conveying the state of the user's body. The following data related to humans may be defined as the body data and the sensor data:

[0133] Height Age · Joint angles of upper and lower limbs, length of upper and lower limb parts Grip strength of hands and feet Touch on hands and feet ·Emotions (level of tension) Such data may be interpreted as the type of data about a human that can be conveyed to an actuator.

[0134] (2.10) SDK Functions For device developers, an SDK (development kit) that can reduce the difficulty of connecting to the human augmentation platform device 100 may be provided. The SDK includes an API for accessing the human augmentation platform device 100 from the body application (connection, status notification, data transmission, data acquisition, etc.). The SDK may include a callback mechanism for the body application to receive notifications from the human augmentation platform device 100.

[0135] (3) Sensing / actuation designation pattern Fig. 9 shows an example of a designation pattern of sensing / actuation. As shown in Fig. 9, a case may be assumed in which the initiator of actuation is the sensing side, the actuation side, or another operator.

[0136] For example, the sensing side may specify the delivery destination. The top part of Fig. 9 shows a use case of a piano lesson, in which the teacher selects student B who is in the lesson.

[0137] In addition, the actuation side may specify the data to be used by itself. The middle part of Fig. 9 shows a use case of live distribution of a skill or distribution of accumulated skill data, in which user B selects data A.

[0138] Also, the designation may be made by an operator other than the sensing side or the actuation side.The lower part of Fig. 9 shows a use case of the operation panel for demonstration, in which the presenter starts data transfer by operating the panel.

[0139] (4) Sequence example 10 to 15 show example sequences related to the data linkage control function and the service management function of the human augmentation platform device 100 according to this embodiment.

[0140] Fig. 10 shows an example of a sequence relating to account registration and service subscription. As shown in Fig. 10, the service management function uses an account registration API to register account information in an account DB, and registers services used by the account in a service DB.

[0141] Fig. 11 shows a sequence example (part 1) relating to body data registration. Fig. 12A shows a sequence example (part 2) relating to body data registration. Fig. 12B shows a sequence example (part 3) relating to body data registration.

[0142] As shown in FIG. 11, the service management function and the data linkage control function acquire body data (physical information) of general users via the WebUI and the control application, and register the acquired body data in the body DB.

[0143] 12A and 12B, the service management function and the data linkage control function allow a user to edit the body data registered in the body data DB. This allows the user's body data to be associated with a user ID (account DB), etc.

[0144] Fig. 13 shows an example of a real-time data transmission and reception sequence by the service management function and the data linkage control function. As shown in Fig. 13, the service management function and the data linkage control function can execute real-time transfer of sensor data between the sensing side and the actuation side based on the service selected by the general user.

[0145] Fig. 14 shows an example of a sequence for storing skill data. As shown in Fig. 14, the service management function and the data linkage control function store the sensor data acquired via the control application in the operation DB. In addition, the service management function can register the sensor data in the skill DB as skill data.

[0146] Fig. 15 shows an example of a sequence for service registration. As shown in Fig. 15, the service management function registers account information in an account DB and registers service information in a service DB. The service management function can also register updated service information (such as a comparison method or a conversion method) in the service DB.

[0147] (5) Details of data conversion Fig. 16 shows a configuration example of a data conversion element using a data linkage control function. The human augmentation platform device 100 (control data generation unit 150) executes conversion processing of the sensor data and transmits it to the actuation side. As shown in Fig. 16, any of the following methods may be applied to convert the sensor data depending on the use case or the format of the data to be handled.

[0148] Scalar conversion: Performs a magnification operation on one element of data Matrix transformation: Transform parts (e.g. from right hand to left hand) or perform magnification calculations on multiple element data such as joint angles Frequency characteristic conversion: For waveform data such as tactile data (audio data), FFT, emphasis conversion, and IFFT are used to emphasize specific frequency bands. When the number of types of data to be handled increases, the conversion method may be selectable from among existing conversion methods.

[0149] (6) Choice mapping (EEG) An example of option mapping is shown in Fig. 17. The human augmentation platform device 100 performs mapping between options and actions corresponding to the options in order to perform one-to-one selection based on electroencephalograms.

[0150] As shown in Fig. 17, a large category may be presented first, and depending on the category selected, more detailed content may be displayed. Also, options may be presented hierarchically.

[0151] A mechanism may be provided for registering the contents (table) of the option mapping via a service setting screen.

[0152] (7) Actions and Effects According to the human augmentation platform device 100 of this embodiment, the data linkage control function and service management function can execute body comparison and data conversion (scalar conversion, matrix conversion, and frequency characteristic conversion) processing, thereby realizing even more advanced interoperability between the sensing side and the actuation side (controlled side).

[0153] In addition, the data linkage control function and the service management function enable past sensor data (skill data) to be managed in the operation DB, and past data can be selected and played back along with a UI for selecting data from the operation DB, enabling the provision of more diverse and advanced human augmentation-related services.

[0154] Furthermore, the data linkage control function and service management function provide a mechanism for mapping options to actions corresponding to each option in order to allow multiple-choice selection (for example, four or eight options) based on brainwaves. This makes it possible to convert the option (number) received from the sensing side into the corresponding skill data and send it to the actuation side, enabling the provision of more diverse and advanced human augmentation-related services.

[0155] [Other embodiments] The present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and it will be obvious to those skilled in the art that various modifications and improvements are possible.

[0156] For example, in the above-described embodiment, the myoelectric sensor 40 and the electroencephalogram sensor 50 are connected to the human augmentation platform device 100, but the type of sensor is not limited to these and may be a sensor or device that detects (acquires) a specific motion such as sound waves (acoustic), images (video), or sports.

[0157] In addition, while the robot 60 and the myoelectric actuator 70 are connected to the human augmentation platform device 100, the target of actuation is not limited to these and may be a device that is worn by a human and simulates a specific movement such as a sport.

[0158] Moreover, the block diagrams (FIGS. 2 and 8) used in the description of the above-mentioned embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically combined, or may be realized by using two or more devices that are physically or logically separated and directly or indirectly connected (for example, by using wires, wirelessly, etc.). The functional blocks may be realized by combining the one device or the multiple devices with software.

[0159] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs the function of transmission is called a transmitting unit or a transmitter. As described above, there is no particular limitation on the method of realization of any of these.

[0160] Furthermore, the above-mentioned human augmented platform device 100 may function as a computer that performs processing of the wireless communication method of the present disclosure. Fig. 18 is a diagram showing an example of a hardware configuration of the human augmented platform device 100. As shown in Fig. 18, the human augmented platform device 100 may be configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, and a bus 1007.

[0161] In the following description, the term "apparatus" may be interpreted as a circuit, a device, a unit, etc. The hardware configuration of the apparatus may be configured to include one or more of the apparatuses shown in the drawings, or may be configured to exclude some of the apparatuses.

[0162] Each functional block of the device (see FIGS. 2 and 8) is realized by any hardware element of the computer device, or a combination of the hardware elements.

[0163] In addition, each function in the device is realized by loading a specific software (program) onto hardware such as the processor 1001, memory 1002, etc., so that the processor 1001 performs calculations, controls communication by the communication device 1004, and controls at least one of reading and writing of data in the memory 1002 and storage 1003.

[0164] The processor 1001 controls the entire computer by running, for example, an operating system, and may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control unit, an arithmetic unit, a register, and the like.

[0165] Furthermore, the processor 1001 reads out programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. As the programs, programs that cause a computer to execute at least a part of the operations described in the above-mentioned embodiments are used. Furthermore, the above-mentioned various processes may be executed by one processor 1001, or may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted from a network via a telecommunications line.

[0166] The memory 1002 is a computer-readable recording medium, and may be configured by at least one of, for example, a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Random Access Memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store a program (program code), a software module, etc., capable of executing a method according to an embodiment of the present disclosure.

[0167] Storage 1003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a Compact Disc ROM (CD-ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, and the like. Storage 1003 may be referred to as an auxiliary storage device. The above-mentioned recording medium may be, for example, a database, a server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0168] The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0169] The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, and the like to realize, for example, at least one of Frequency Division Duplex (FDD) and Time Division Duplex (TDD).

[0170] The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated into one device (e.g., a touch panel).

[0171] In addition, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0172] Furthermore, the device may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc., and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0173] Each aspect / embodiment described in the present disclosure may be applied to at least one of a system using Long Term Evolution (LTE), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), Future Radio Access (FRA), New Radio (NR), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), other suitable systems, and next-generation systems extended based on these. In addition, a combination of multiple systems (e.g., a combination of at least one of LTE and LTE-A and 5G, etc.) may be applied.

[0174] The order of the steps, sequences, flow charts, etc. of each aspect / embodiment described in this disclosure may be changed unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0175] Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched according to execution. In addition, notification of specific information (e.g., notification that "X is the case") is not limited to being done explicitly, but may be done implicitly (e.g., not notifying the specific information).

[0176] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0177] Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired and / or wireless technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave, etc.), then these wired and / or wireless technologies are included within the definition of transmission media.

[0178] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0179] In addition, the terms described in this disclosure and the terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Also, a signal may be a message. Also, a component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

[0180] As used in this disclosure, the terms "system" and "network" are used interchangeably.

[0181] In addition, the information, parameters, etc. described in the present disclosure may be represented using absolute values, may be represented using relative values ​​from a predetermined value, or may be represented using other corresponding information. For example, a radio resource may be indicated by an index.

[0182] In this disclosure, terms such as "base station (BS)", "radio base station", "fixed station", "NodeB", "eNodeB (eNB)", "gNodeB (gNB)", "access point", "transmission point", "reception point", "transmission / reception point", "cell", "sector", "cell group", "carrier", "component carrier", etc. may be used interchangeably. A base station may also be referred to by terms such as a macro cell, a small cell, a femto cell, a pico cell, etc.

[0183] A base station can accommodate one or more (e.g., three) cells (also called sectors). When a base station accommodates multiple cells, the entire coverage area of ​​the base station can be divided into multiple smaller areas, and each smaller area can also be provided with communication services by a base station subsystem (e.g., a small indoor base station (Remote Radio Head: RRH)).

[0184] The term "cell" or "sector" refers to part or all of the coverage area of ​​a base station and / or a base station subsystem that provides communication services within that coverage.

[0185] In this disclosure, the terms "Mobile Station (MS)", "user terminal", "User Equipment (UE)", "terminal", etc. may be used interchangeably.

[0186] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terminology.

[0187] At least one of the base station and the mobile station may be called a transmitting device, a receiving device, a communication device, etc. At least one of the base station and the mobile station may be a device mounted on a moving body, the moving body itself, etc. The moving body may be a vehicle (e.g., a car, an airplane, etc.), an unmanned moving body (e.g., a drone, an autonomous vehicle, etc.), or a robot (manned or unmanned). At least one of the base station and the mobile station may include a device that does not necessarily move during communication operation. For example, at least one of the base station and the mobile station may be an Internet of Things (IoT) device such as a sensor.

[0188] Furthermore, a base station in the present disclosure may be read as a mobile station (user terminal, the same applies below). For example, each aspect / embodiment of the present disclosure may be applied to a configuration in which communication between a base station and a mobile station is replaced with communication between multiple mobile stations (which may be called, for example, Device-to-Device (D2D) or Vehicle-to-Everything (V2X)). In this case, the mobile station may be configured to have the functions of a base station. Furthermore, terms such as "uplink" and "downlink" may be read as terms corresponding to terminal-to-terminal communication (for example, "side"). For example, an uplink channel, a downlink channel, etc. may be read as a side channel (or side link).

[0189] Similarly, a mobile station in the present disclosure may be interpreted as a base station, in which case the base station may have the functions of a mobile station.

[0190] The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0191] The reference signal may also be abbreviated as Reference Signal (RS), and may be called a pilot depending on the applicable standard.

[0192] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0193] The "means" in the configuration of each of the above devices may be replaced with "part," "circuit," "device," etc.

[0194] Any reference to an element using a designation such as "first," "second," etc., used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some way.

[0195] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term "comprising." Further, as used in this disclosure, the term "or" is not intended to be an exclusive or.

[0196] In this disclosure, where articles have been added due to translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0197] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, search, inquiry (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), and the like. In addition, "judgment" and "decision" can include considering resolving, selecting, choosing, establishing, comparing, etc., to be a "judgment" or "decision." In other words, "judgment" and "decision" can include considering some action to be a "judgment" or "decision." In addition, "judgment" can be interpreted as "assuming," "expecting," "considering," etc.

[0198] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0199] (Additional Note) The above disclosure may be expressed as follows:

[0200] A first feature is that the human augmentation platform device includes a physical information acquisition unit that acquires physical information indicative of the physical abilities of the sensing side and the controlled side, a motion data acquisition unit that acquires motion data of the sensing side, a control data generation unit that generates control data for the controlled side based on the physical information acquired by the physical information acquisition unit and the motion data acquired by the motion data acquisition unit, and an actuation unit that actuates the body of the controlled side based on the control data, wherein the control data generation unit generates the control data within the range of the physical abilities of the controlled side.

[0201] A second feature is that in the first feature, the control data generation unit generates the control data in which the physical ability of the controlled side is proportional to the physical ability of the sensing side based on a comparison result between the range of physical ability of the sensing side and the range of physical ability of the controlled side.

[0202] A third feature based on the first or second feature is that the control data generation unit generates the control data based on an allowable value of physical ability of the controlled party.

[0203] A fourth feature based on any one of the first to third features is that the control data generation unit generates the control data based on a ratio between the physical information of the sensing side and the physical information of the controlled side.

[0204] A fifth feature is that, in the first to fourth features, when the part of the limb on the sensing side is different from the part of the limb on the controlled side, the control data generation unit converts motion data of the part of the limb on the sensing side into motion data of the part of the limb on the controlled side, and generates the control data based on the physical information and the converted motion data.

[0205] A sixth feature is that, in the first to fifth features, a skill data storage unit is provided that stores the motion data acquired by the motion data acquisition unit as skill data organized based on predetermined features, and the control data generation unit generates the control data for playback based on the skill data.

[0206] A seventh feature, in any of the first to sixth features, includes an option mapping unit that presents options of the control data to a user, the option mapping unit identifies the skill data associated with the option selected by the user, and the control data generation unit generates the control data for playback based on the identified skill data.

[0207] The eighth feature is, in the first to seventh features, when there are multiple parts of the controlled limb and the motion ranges based on the motion data of each of the parts overlap, the control data generation unit determines whether any of the parts will come into contact with the other parts, and if it determines that contact will occur, generates the control data to avoid the contact.

[0208] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure. [Explanation of symbols]

[0209] 10 Human Augmentation Systems 20. Communication Networks 40 EMG Sensor 50 Brainwave Sensor 55 Tactile Sensor 60 Robot 70 Myoelectric Actuator 100 Human Augmentation Platform Device 110 Physical Information Acquisition Department 120 Operation data acquisition unit 130 Physical information DB 140 Operation DB 150 Control data generation unit 160 Actuation Section 170 Choice Mapping Section 180 Skill DB 1001 Processor 1002 Memory 1003 Storage 1004 Communication equipment 1005 Input Device 1006 Output device 1007 Bus

Claims

1. a physical information acquisition unit that acquires physical information indicating physical capabilities of a sensing side and a controlled side; An operation data acquisition unit that acquires operation data on the sensing side; a control data generating unit that generates control data for the controlled side based on the physical information acquired by the physical information acquiring unit and the motion data acquired by the motion data acquiring unit; an actuation unit that operates the body of the controlled person based on the control data; Equipped with The control data generation unit generates the control data within the range of physical ability of the controlled party.

2. 2. The human augmentation platform device according to claim 1, wherein the control data generation unit generates the control data in which the physical ability of the controlled side is proportional to the physical ability of the sensing side based on a comparison result between a range of physical ability of the sensing side and a range of physical ability of the controlled side.

3. The human augmentation platform device according to claim 1 , wherein the control data generation unit generates the control data based on an allowable value of physical ability of the controlled side.

4. The human augmentation platform device according to claim 1 , wherein the control data generation unit generates the control data based on a ratio between the physical information of the sensing side and the physical information of the controlled side.

5. The control data generation unit When the part of the limb on the sensing side is different from the part of the limb on the controlled side, converting the motion data of the part on the sensing side into motion data of the part on the controlled side; The human augmentation platform device according to claim 1 , wherein the control data is generated based on the physical information and the converted motion data.

6. a skill data storage unit that stores the motion data acquired by the motion data acquisition unit as skill data organized based on a predetermined feature, The human augmentation platform device according to claim 1 , wherein the control data generation unit generates the control data for playback based on the skill data.

7. an option mapping unit that presents options of the control data to a user; The option mapping unit identifies the skill data associated with the option selected by the user, The human augmentation platform device according to claim 6 , wherein the control data generation unit generates the control data for playback based on the identified skill data.

8. The control data generation unit When there are a plurality of parts of the limb on the controlled side, when the motion ranges based on the motion data of each of the parts overlap, it is determined whether or not any of the parts comes into contact with another of the parts; The human augmentation platform device according to claim 1 , wherein when it is determined that contact will occur, the control data is generated to avoid the contact.

9. 2. The human augmented platform device according to claim 1, wherein when the sensing side is a robot and the controlled side is a human, the control data generation unit generates the control data in which the motion speed passes through an accelerating state, a constant state, and a decelerating state.

10. acquiring physical information indicative of physical capabilities of a sensing side and a controlled side; acquiring operational data of the sensing side; generating control data for the controlled device based on the physical information and the motion data; operating the body of the controlled person based on the control data; Including, A physical ability augmentation method, in which the step of generating control data generates the control data within the range of the physical ability of the controlled party.

Images