Human augmentation platform device and perception augmentation method

Abstract

A human augmentation platform device 100 includes: a sensitivity information acquisition unit 110 for presenting, to a user, a question about perception including at least one of taste or smell, and acquiring a plurality of pieces of sensitivity information obtained by combining sensitivity information including a sensitivity score of perception of the user with basic data of the perception; a comparison processing unit 150 for comparing first sensitivity information selected from among the plurality of pieces of sensitivity information with second sensitivity information selected from among the plurality of pieces of sensitivity information; a data conversion unit 160 for acquiring converted control data by combining, with the basic data, a difference between a first sensitivity score included in the first sensitivity information and a second sensitivity score included in the second sensitivity information; and an actuation unit 170 for operating a specific device on the basis of the control data.

Description

Human augmentation platform device and perception augmentation method

[0001] The present disclosure relates to a human augmentation platform device and a method for augmenting human perception that support the augmentation of human perception.

[0002] The 3rd Generation Partnership Project (3GPP) is developing specifications for the 5th generation mobile communication system (5G, also known as New Radio (NR) or Next Generation (NG)), and is also developing specifications for the next generation, known as Beyond 5G, 5G Evolution, or 6G.

[0003] One of the technologies to watch in the 6G era is human augmentation, which can be broadly divided into "augmented physical abilities," "augmented existence," "augmented perception," and "augmented cognitive abilities."

[0004] Among these, the expansion of perception involves sharing and expanding the five senses. Vision and hearing have already reached a certain technological level, and touch is currently being developed using various interface technologies. Taste and smell are still at the frontier stage, and research into receptors is ongoing (Non-Patent Documents 1 and 2).

[0005] NTT Docomo, "Docomo 6G White Paper 5.0 Edition," [online], November 2022, Internet <URL:https: / / www.docomo.ne.jp / binary / pdf / corporate / technology / whitepaper_6g / DOCOMO_6G_White_PaperJP_20221116.pdf> Human Augmentation Platform for Realizing New Communication, [online], October 2023, Internet <URL:https: / / www.docomo.ne.jp / corporate / technology / rd / technical_journal / bn / vol31_3 / index.html>

[0006] It is known that taste and smell vary greatly from person to person, and that sensitivity varies from person to person. It is also known that taste perception changes with age.

[0007] For this reason, even if we try to simply augment our sense of taste and smell using a human augmentation platform device such as the Human Augmentation Platform (registered trademark), there is often a large gap between what is perceived by individuals and the actual sense, making accuracy an issue.

[0008] The following disclosure has been made in light of the above circumstances, and aims to provide a human augmentation platform device and a method for augmenting perception that can accommodate individual differences in taste and smell.

[0009] One aspect of the present disclosure is a human augmented platform device (human augmented platform device 100) that includes an acquisition unit (sensitivity information acquisition unit 110) that presents a user with questions regarding perceptions including at least either taste or smell, and acquires multiple pieces of sensitivity information by multiplying the user's perception sensitivity score with basic data about the perception; a comparison unit (comparison processing unit 150) that compares first sensitivity information selected from the multiple pieces of sensitivity information acquired by the acquisition unit with second sensitivity information selected from the multiple pieces of sensitivity information acquired by the acquisition unit; a conversion unit (data conversion unit 160) that converts the control data into control data by multiplying the basic data by the difference between the first sensitivity score included in the first sensitivity information and the second sensitivity score included in the second sensitivity information; and an actuation unit (actuation unit 170) that activates a predetermined device based on the control data.

[0010] FIG. 1 is a schematic diagram of the overall configuration of the human augmentation system 10. FIG. 2 is a functional block diagram of the human augmentation platform 100. FIG. 3A is a diagram showing an example of a detailed block configuration of the human augmentation platform 100 (human augmentation platform). FIG. 3B is a diagram showing an example of a detailed block configuration of the human augmentation platform 100 (human augmentation platform). FIG. 4 is a schematic explanatory diagram of the taste augmentation operation using the human augmentation platform 100 (human augmentation platform). FIG. 5 is a diagram showing the flow of the taste augmentation process. FIG. 6 is a diagram showing the flow of the olfactory augmentation process. FIG. 7 is a diagram showing an example of taste sensitivity data (score) and corrected taste data for a specific user. FIG. 8 is a diagram showing an example of olfactory sensitivity data (score) and corrected taste data for a specific user. FIG. 9 is an explanatory diagram of taste transmission correction due to differences in granularity. FIG. 10 is an explanatory diagram of correction by age for taste augmentation subjects (users). FIG. 11 is a graph showing changes in taste with aging. FIG. 12 is an explanatory diagram of building an AI model for personalized taste transmission. Fig. 13 is an explanatory diagram of the construction of an AI model for personalized taste transmission. Fig. 14 is an explanatory diagram of the construction of an AI model for converting language to taste. Fig. 15 is an explanatory diagram of the construction of an AI model for converting taste to language. Fig. 16 is a diagram showing an example of the hardware configuration of the human augmentation platform device 100.

[0011] Hereinafter, embodiments will be described with reference to the drawings. Note that the same or similar reference numerals are used to designate the same functions or configurations, and descriptions thereof will be omitted as appropriate.

[0012] (1) Overall Schematic Configuration of the Human Augmentation Platform Device Figure 1 is a schematic diagram of the overall configuration of a human augmentation system 10 according to this embodiment. As shown in Figure 1, the human augmentation system 10 includes a sensing side composed of a group of multiple sensors with different measurement targets, and an augmentation side (which may also be called the controlled side or actuation side) where a human or a robot is the control target.

[0013] The human augmentation platform device 100 is connected to the sensing side and the augmentation side. The human augmentation platform device 100 may also be connected to a communication network 20.

[0014] In this embodiment, the human augmentation platform device 100 is connected to a taste sensor 40, an olfactory sensor 50, and a language input unit 55. The human augmentation platform device 100 is also connected to a language output unit 60 and a taste / smell actuator 70.

[0015] The communication network 20 may include a wired network and a wireless network. The human augmentation platform device 100 may be able to access other systems, databases, application services, and the like via the communication network 20.

[0016] The taste sensor 40 may also be called a taste sensor or a taste recognition device. The taste sensor 40 can digitize components of taste, such as bitterness, sweetness, sourness, and saltiness. Specifically, the taste sensor 40 has the same mechanism as the human tongue and digitizes the bitterness, sweetness, sourness, and saltiness of various foods, medicines, and the like. As the taste sensor 40, for example, a taste sensor that mimics the biological taste reception mechanism can be used.

[0017] The olfactory sensor 50 may also be called an odor sensor or an odor sensor. The olfactory sensor 50 can digitize odor (smell) components (e.g., fresh, volatile odor, stinky odor). The olfactory sensor 50 may be, for example, a quartz oscillator type or a semiconductor type. In particular, a quartz oscillator type is preferably used in this embodiment.

[0018] The language input unit 55 inputs linguistic expressions of the user of the human augmentation platform device 100. Specifically, the language input unit 55 can input linguistic expressions related to the user's sense of taste or smell. The input method may be voice input or input using a keyboard or the like. For example, the user can use the language input unit 55 to input something like, "It smells like roses and has a taste that seems to rise from the ground. It's similar to brand X, which costs around 30,000 yen. It has a sharp taste / a well-balanced taste."

[0019] The language output unit 60 outputs language information related to the sense of taste or smell. Specifically, the language output unit 60 can output audio data or display text information related to the sense of taste or smell input from the human augmentation platform device 100.

[0020] The taste / smell actuator 70 is a device that generates a specific taste or smell based on taste or smell control data input from the human augmentation platform 100. As described above, taste control data (which may also be called taste data) may be composed of values ​​such as bitterness, sweetness, sourness, and saltiness, and smell control data (which may also be called smell data) may be composed of values ​​of odor components (e.g., freshness, volatile odor, and bad odor).

[0021] The human augmentation platform device 100 is connected to the sensing side and the augmented side (controlled side) to realize human augmentation. Specifically, the human augmentation platform device 100 realizes the augmentation of perception. In this embodiment, perception may include at least one of taste and smell. In addition to taste and smell, perception may also include sight, hearing, touch, and the like. The augmentation of perception may mean the sharing or augmentation of the five senses.

[0022] (2) Functional Block Configuration of the Human Augmentation Platform Fig. 2 is a functional block configuration diagram of the human augmentation platform 100. As shown in Fig. 2, the human augmentation platform 100 includes a sensitivity information acquisition unit 110, a sensor data acquisition unit 120, a language information acquisition unit 125, a body DB 130, an operation DB 140, a comparison processing unit 150, a data conversion unit 160, and an actuation unit 170.

[0023] The sensitivity information acquisition unit 110 acquires sensitivity information related to the user's perceptual sensitivity. Specifically, the sensitivity information acquisition unit 110 presents a question related to perception including at least one of taste and smell to the human user, and acquires a plurality of pieces of sensitivity information including the user's perceptual sensitivity score. In this embodiment, the sensitivity information acquisition unit 110 may constitute an acquisition unit.

[0024] More specifically, the sensitivity information acquisition unit 110 presents the user with questions regarding perceptions including at least one of taste and smell, and can acquire multiple pieces of sensitivity information by multiplying the user's perception sensitivity score by basic perception data.

[0025] The questions presented by the sensitivity information acquisition unit 110 may be in the form of a questionnaire. For example, the sensitivity information acquisition unit 110 may present the questionnaire to user A and user B and obtain answers from each of user A and user B. The number of questions included in the questionnaire is not particularly limited, but in order to efficiently recognize each individual's taste or smell sensitivity score, it is preferable to have about 20 to 30 questions (details will be described later).

[0026] The sensitivity score (which may also be called sensitivity data) may be data in which at least one of bitterness, sweetness, sourness, saltiness, and umami is quantified (e.g., percentage) in the case of taste, or data in which odor components (e.g., freshness, volatile odor, odorous) are quantified (e.g., percentage) in the case of olfaction.

[0027] For example, the basic data of perception may be composed of the values ​​of bitterness, sweetness, sourness, saltiness, and umami contained in a specific dish in the case of taste. The sensitivity information acquisition unit 110 may acquire the sensitivity information by multiplying the value of the user's sensitivity score by the value (e.g., percentage) of each component of the basic data.

[0028] The sensitivity information acquiring section 110 may acquire the user's gender, age, preferences, etc. in addition to the responses to such a questionnaire regarding taste or smell.

[0029] The sensor data acquiring unit 120 acquires data output from various sensors. In this embodiment, the sensor data acquiring unit 120 can acquire data acquired from the taste sensor 40 and the olfactory sensor 50.

[0030] The sensor data acquiring unit 120 acquires numerically converted data relating to taste output from the taste sensor 40. The sensor data acquiring unit 120 also acquires numerically converted data relating to smell output from the olfactory sensor 50.

[0031] The language information acquisition unit 125 acquires language information based on the user's language expression input via the language input unit 55. Specifically, the language information acquisition unit 125 can acquire the language expression input via the language input unit 55 as character (or voice) language information.

[0032] The language information acquisition unit 125 stores the acquired language information in the operation DB 140.

[0033] The body DB 130 stores information about the user's body. In this embodiment, the body DB 130 stores sensitivity information acquired by the sensitivity information acquisition unit 110. Specifically, the body DB 130 may be configured with a sensitivity score (sensitivity data) for each user and personal information such as gender, age, and preferences. Note that the users may include celebrities, and the body DB 130 can store the sensitivity information of the celebrities.

[0034] The operation DB 140 stores the data acquired by the sensor data acquiring unit 120. In this embodiment, the operation DB 140 can store sensitivity data for each specific food based on the data acquired by the sensor data acquiring unit 120.

[0035] The comparison processing unit 150 compares the sensitivity information for each user stored in the body DB 130. For example, the comparison processing unit 150 can compare the sensitivity information of user A with the sensitivity information of user B.

[0036] Specifically, the comparison processing unit 150 can compare the sensitivity information of user A (first sensitivity information) selected from the plurality of sensitivity information acquired by the sensitivity information acquisition unit 110 with the sensitivity information of user B (second sensitivity information) selected from the plurality of sensitivity information acquired by the sensitivity information acquisition unit 110. In this embodiment, the comparison processing unit 150 may constitute a comparison unit.

[0037] Such a comparison of user sensitivity information by the comparison processing unit 150 may be called a body comparison. The comparison processing unit 150 compares the value of the sensitivity score (sensitivity data) included in the sensitivity information of user A with the value of the sensitivity score (sensitivity data) included in the sensitivity information of user B, and can output the difference between each component of the sensitivity score to the data conversion unit 160.

[0038] The data conversion unit 160 generates control data to be output to the actuation unit 170 based on the comparison result of the sensitivity information output from the comparison processing unit 150. For example, the data conversion unit 160 can correct the first sensitivity score included in the first sensitivity information based on the comparison result between the sensitivity information of user A (first sensitivity information) and the sensitivity information of user B (second sensitivity information). The data conversion unit 160 corrects the first sensitivity score to convert it into control data that matches the second sensitivity score included in the second sensitivity information. In this embodiment, the data conversion unit 160 may constitute a conversion unit.

[0039] Specifically, the data conversion unit 160 can convert the difference between the first sensitivity score contained in the sensitivity information of user A (first sensitivity information) and the second sensitivity score contained in the sensitivity information of user B (second sensitivity information) into control data multiplied by basic perceptual data.

[0040] For example, the data conversion unit 160 uses the difference between the first sensitivity score and the second sensitivity score, that is, in the case of taste, the difference between data in which at least one of bitterness, sweetness, sourness, saltiness, and umami is quantified (e.g., percentage). Furthermore, the data conversion unit 160 uses basic data of a specific dish, specifically, the numerical values ​​of bitterness, sweetness, sourness, saltiness, and umami contained in the specific dish, as described above.

[0041] The data conversion unit 160 can convert the difference in sensitivity scores into control data by multiplying the difference by the value of the basic data of the dish. Specifically, the data conversion unit 160 can multiply each difference in bitterness, sweetness, sourness, saltiness, and umami by the value (percentage) of bitterness, sweetness, sourness, saltiness, and umami in the basic data of the dish.

[0042] In this way, the data conversion unit 160 can generate control data using the sensitivity score of user A, the sensitivity score of user B, and basic data such as food. The taste perceived by user A can be generated by multiplying the basic data by the sensitivity score of user A. Similarly, the taste perceived by user B can be generated by multiplying the basic data by the sensitivity score of user B.

[0043] Therefore, by multiplying the basic data by the difference between the first sensitivity score and the second sensitivity score, user B can sense the taste of user A when eating the dish.

[0044] Although the explanation is given here for the sense of taste, similar processing may be applied to the sense of smell. Furthermore, the difference may be simply a value obtained by subtracting one of the first sensitivity score or the second sensitivity score from the other, or the difference between the first sensitivity score and a certain reference value, or the ratio of the difference between the reference value and the second sensitivity score, etc. may be used.

[0045] The data conversion unit 160 may convert the ratio of components of the sensitivity score based on the result of such a comparison of sensitivity information between users (body comparison). The data conversion unit 160 may also correct the first sensitivity score to match the second sensitivity score by changing the granularity of the components included in the taste object. Examples of changing the granularity will be described later.

[0046] Furthermore, the data conversion unit 160 may correct the first sensitivity score according to the user's age group to match it with the second sensitivity score. For example, it is known that taste (the way tastes are perceived) changes depending on age. When the age (age) of user A and the age (age) of user B are different, the data conversion unit 160 may correct the sensitivity score to correct such differences in taste perception depending on age. Examples of corrections by age group will be described later.

[0047] The data conversion unit 160 may input the sensitivity information to an artificial intelligence (AI) system. For example, the data conversion unit 160 can input the sensitivity information (sensitivity score) provided by the comparison processing unit 150 to a generative AI system to obtain linguistic information in which the sensitivity score is verbalized based on the sensitivity information. The type of generative AI system is not particularly limited as long as it can output some kind of linguistic information by inputting a taste or smell sensitivity score, i.e., a numerical value for each component. The data conversion unit 160 may provide control data including the linguistic information acquired from the AI ​​system to the actuation unit 170.

[0048] Furthermore, the data conversion unit 160 can input linguistic information verbalized by the user to the AI ​​system, and acquire a sensitivity score that is quantified based on the linguistic information. The data conversion unit 160 may provide the actuation unit 170 with control data that uses the sensitivity score acquired from the AI ​​system.

[0049] Furthermore, the data conversion unit 160 may apply feedback control between the first sensitivity score and the second sensitivity score. For example, the data conversion unit 160 may perform feedback of the corrected second sensitivity score using PID (Proportional, Integral, Differential) control to adjust the correction level of the first sensitivity score.

[0050] The actuation unit 170 operates a predetermined device based on the control data provided by the data conversion unit 160. Specifically, the actuation unit 170 operates the taste / smell actuator 70 based on the control data. More specifically, the actuation unit 170 can output control data in which components of taste or smell are quantified to the taste / smell actuator 70.

[0051] In addition, the actuation unit 170 can output control data including the language information provided by the data conversion unit 160 to the language output unit 60. Furthermore, the actuation unit 170 can output control data using the sensitivity score provided by the data conversion unit 160 to the taste / smell actuator 70.

[0052] 3A and 3B show an example of a detailed block configuration of the human augmentation platform device 100 (human augmentation platform). As shown in Fig. 3A and 3B, in order to prepare taste and smell sensitivity information, the body application on the input (user A) side may include a language input / output screen and a screen for a questionnaire.

[0053] The human augmentation platform can synthesize taste and smell data based on individual sensitivity, and transmit the data to others after applying various transmission corrections (granularity, age, etc.). Users can select any taste or smell data from the data stored (accumulated) in the human augmentation platform. For example, users can select whether they want to experience the taste or smell of a specific user from a list of users, such as celebrities.

[0054] The data linkage control in the human augmentation platform may calculate sensitivity scores. The data conversion process in the human augmentation platform may include compositing individual taste and smell data, ratio conversion by comparing body data, transmission correction (by granularity, age, etc.), magnification conversion, and limit conversion based on an upper limit value.

[0055] After comparing the body data, sensing data (control data) consisting of taste and smell data after various corrections is provided to the body application on the output side (user B).

[0056] (3) Operation of the Human Augmentation Platform Next, we will explain the operation of the human augmentation platform 100. Specifically, we will explain an example of the operation of expanding the senses of taste and smell using the human augmentation platform 100 (human augmentation platform).

[0057] (3.1) Outline of Operation Fig. 4 is a schematic explanatory diagram of the taste augmentation operation by the human augmentation platform 100 (human augmentation platform). The human augmentation platform 100 can provide the following functions regarding taste augmentation.

[0058] ・Transmission correction model based on differences in taste and granularity ・Taste transmission correction model by age group ・Personalization for specific users (taste transmission from linguistic expression, evaluation from taste transmission by linguistic expression) As shown in Figure 4, on the user A (body A) side, body data, sensor data (using taste sensors), and food taste data are prepared. The human augmentation platform may perform a comparison process of the sensitivity information of the selected user information and construct a taste transmission correction model. On the user B (body B) side, body data, body comparison data, operation data (which may also be called control data), and food taste data are prepared.

[0059] Fig. 5 shows the flow of taste augmentation processing. As shown in Fig. 5, the human augmentation platform can perform ratio conversion (which may include frequency characteristic conversion) based on individual data comparison, limiting conversion (normalization processing) based on upper limits, actuator activation, and output of verbalized information (linguistic output).

[0060] Fig. 6 shows the flow of olfactory augmentation processing. As shown in Fig. 6, the human augmentation platform can perform ratio conversion (which may include frequency characteristic conversion) based on individual data comparison, limiting conversion (normalization processing) based on upper limits, actuator activation, and output of verbalized information (linguistic output), as in the case of taste shown in Fig. 5. In the case of olfaction, compared to taste, correction of age and granularity does not need to be supported.

[0061] In addition, in the case of taste and smell, data conversion such as region conversion and magnification conversion, such as expansion of physical abilities, may not be applied.

[0062] FIG. 7 shows an example of taste sensitivity data (scores) and corrected taste data for a specific user. As shown in FIG. 7 , in this embodiment, taste may include bitter, sweet, sour, salty, and umami components. In the graph on the right side of FIG. 7 , the taste data corrected by the above-described correction is indicated by a dotted line. For example, the taste data is corrected based on the difference in the taste sensitivity data between user A and user B, so that user B will perceive a taste similar to that perceived by user A.

[0063] FIG. 8 shows an example of olfactory sensitivity data (score) and corrected taste data for a specific user. In this embodiment, the olfactory sense may include the components of berry, sweets, volatile odor, odor, and freshness. In the graph on the right side of FIG. 8, the olfactory data corrected by the above-described correction is indicated by a dotted line. This olfactory data is, for example, data corrected based on the difference in sensitivity data between user A and user B so that user B will perceive an odor similar to the odor perceived by user A.

[0064] As described above, in the case of taste, the value obtained by multiplying basic data such as food by the difference between user A's sensitivity score (first sensitivity score) and user B's sensitivity score (second sensitivity score) may be used as the corrected data.

[0065] (3.2) Operational Examples Next, we will explain operational examples of the human augmentation platform device 100 related to the augmentation of taste or smell. Specifically, we will explain operational examples related to taste transmission correction based on differences in granularity, age-specific taste transmission correction, AI model construction for personalized taste (smell) transmission, language-to-taste (smell) conversion AI model construction, and taste-to-language conversion AI model construction.

[0066] (3.2.1) Taste transmission correction due to particle size difference Figure 9 is an explanatory diagram of taste transmission correction due to particle size difference. This correction is a transmission correction of the seasoning ratio at the time of output based on the ratio of the seasoning particle size at the time of input (body A) to the seasoning particle size at the time of output (body B).

[0067] As shown in Figure 9, the particle size of the sugar:glutamine:salt particles of the target object (e.g., Western-style rice crackers) measured by the taste sensor was 45:1:3, but based on the difference in sensitivity scores between body A (user A) and body B (user B), the particle size was roughly corrected and output to body B as 45:3:6.

[0068] In this way, a correction model or learning device may be constructed depending on the particle granularity. For example, in the case of a linear model, the corrected output may be expressed as k * taste sensor * granularity ratio + b (where granularity ratio = output granularity / taste sensor granularity), where k and b are coefficients.

[0069] (3.2.2) Age-specific taste transmission correction Fig. 10 is an explanatory diagram of age-specific correction for a taste augmentation target (user). Taste perception changes depending on age. The human augmentation platform automatically corrects taste perception by age.

[0070] Figure 11 is a graph showing changes in taste with age. As shown in Figure 11, there are large differences in how salty and bitter tastes are perceived, particularly depending on age. For example, in the case of beer, the bitterness perceived by a person in their 30s needs to be made 1.5 times more bitter when conveying it to a person in their 50s. Similarly, in the case of edamame, the saltiness perceived by a person in their 50s needs to be made 2.5 times more salty when conveying it to a person in their 70s.

[0071] This correction allows other people's taste buds (how they perceive taste) to be shared across generations, as shown in Figure 10. For example, adults can understand how children perceive bitterness. Also, children can perceive the taste of foods and drinks that children cannot consume, such as beer.

[0072] (3.2.3) Building an AI model for personalized taste transmission Figures 12 and 13 are explanatory diagrams for building an AI model for personalized taste transmission. First, as a preparation, the correlation between the "individual condition survey and taste distribution survey (questionnaire)" and the "individual detailed taste survey" may be analyzed.

[0073] Next, based on the analysis results, the "individual condition survey and taste distribution survey (questionnaire)" is reconstructed by narrowing down the questions to those with high correlation. The human augmentation platform may then correct the personalized taste transmission. This allows the number of questions making up the questionnaire to be reduced from around 100 to around 20-30.

[0074] As shown in Fig. 13, the human augmentation platform may combine the results of a detailed survey of individual tastes with the taste data acquired by the sensor to create personal taste data. The human augmentation platform may use this personal taste data to perform the taste transmission correction described above and output operation data to the side that will augment the taste (body B side).

[0075] Conducting a detailed survey of each individual's taste buds individually would take time, but by using a "personal condition survey (questionnaire)" that can be completed in a short time with a limited number of questions, and by using taste transfer correction, it is possible to save time and easily share tastes between different users. This makes it easy to have taste experiences such as "feeling the taste of others," "being able to eat poisonous mushrooms (with the same taste)," and "enjoying the taste of Hunan Province, a region famous for spicy food."

[0076] (3.2.4) Building an AI model for converting language into taste Fig. 14 is an explanatory diagram for building an AI model for converting language into taste. As shown in Fig. 14, the human augmentation platform may execute processing related to, for example, wine (food) expression and taste factor analysis.

[0077] For example, as shown in Figure 14, a sommelier for a particular wine describes its taste in words, saying, "It has a rose-like aroma and a taste that seems to rise from the ground. It's similar to brand X, which costs around 30,000 yen. It has a sharp, well-balanced taste." Based on this description, the human augmentation platform can obtain taste data associated with the language from the AI ​​system and convert the language into taste data. The human augmentation platform can also obtain olfactory data associated with the language from the AI ​​system and convert the language into olfactory data. This makes it possible to add aromas and flavors to the actuator in real time.

[0078] (3.2.5) Building an AI model for converting taste into language Fig. 15 is an explanatory diagram for building an AI model for converting taste into language. As shown in Fig. 15, the human augmentation platform may execute processing related to, for example, evaluating wine (food) using linguistic expressions based on its taste.

[0079] As shown in Figure 15, the human augmentation platform can acquire linguistic expressions associated with taste or smell data acquired using a taste sensor or an olfactory sensor from an AI system, and convert the taste or smell data into linguistic information (e.g., "smells like roses").

[0080] The human augmentation platform does not necessarily have to use an AI system. For example, if an olfactory sensor can classify the scent of roses, the human augmentation platform can generate linguistic information directly without using an AI system.

[0081] (4) Actions and Effects According to the above-described embodiment, the following actions and effects can be obtained. Specifically, the human augmented platform device 100 compares sensitivity information (first sensitivity information) of a specific user selected from the acquired multiple pieces of sensitivity information with sensitivity information (second sensitivity information) of another user selected from the multiple pieces of sensitivity information. Based on the comparison result between the first sensitivity information and the second sensitivity information, the human augmented platform device 100 corrects the sensitivity score (first sensitivity score) included in the first sensitivity information and converts it into control data by multiplying the difference from the sensitivity score (second sensitivity score) included in the second sensitivity information by basic taste data such as food. The human augmented platform device 100 can operate a predetermined device (taste / smell actuator 70) based on the control data.

[0082] Therefore, even in the case of taste and smell, which vary greatly from person to person, it is possible to expand perception using the human augmentation platform. In other words, the human augmentation platform device 100 can accommodate individual differences in taste and smell.

[0083] In this embodiment, the human augmentation platform device 100 can correct the first sensitivity score to match the second sensitivity score by changing the particle size of the components contained in the taste object. Therefore, even if taste sensitivity differs between individuals, the device can be operated so that people perceive similar tastes.

[0084] In this embodiment, the human augmentation platform device 100 can correct the first sensitivity score according to the user's age and match it to the second sensitivity score. Therefore, even if taste sensitivity differs depending on age, the device can be operated so that users can perceive similar tastes.

[0085] In this embodiment, the human augmentation platform device 100 can input sensitivity information into an artificial intelligence (AI) system to output control data including information on verbalized sensitivity scores (as described above, it is also possible to generate taste / olfactory data from linguistic information). This enables mutual conversion between taste / olfactory data and linguistic information that expresses taste / olfactory sensations using that data. This makes it easy to verbalize tastes and smells, or to generate taste / olfactory data from linguistic information.

[0086] In this embodiment, the human augmentation platform device 100 can apply feedback control (PID control) between the first sensitivity score and the second sensitivity score, which can improve the accuracy of the control data that matches the second sensitivity score.

[0087] (5) Other Embodiments The contents of the present proposal have been explained above using examples, but it will be obvious to those skilled in the art that the present proposal is not limited to these descriptions and that various modifications and improvements are possible.

[0088] For example, in the above-described embodiment, the human augmentation platform device 100 supports the expansion of taste and smell, but it does not necessarily have to support both, and may support only one of them. Furthermore, in the above-described embodiment, the human augmentation platform device 100 realizes conversion between language and taste / smell data using an AI system, but an AI system is not necessarily required. Furthermore, the type of AI system is typically a generation AI, but is not particularly limited, and the type of generation AI is also not particularly limited.

[0089] The block diagram ( FIG. 2 ) used to explain the above-described embodiment shows functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (e.g., wired, wireless, etc.) and these multiple devices. The functional block may be realized by combining software with the single device or multiple devices.

[0090] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how each is implemented.

[0091] Furthermore, the above-described human augmented platform 100 may function as a computer that performs processing of the wireless communication method of the present disclosure. Fig. 16 is a diagram showing an example of the hardware configuration of the human augmented platform 100. As shown in Fig. 16, the human augmented platform 100 may be configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

[0092] In the following description, the term "apparatus" can be interpreted as a circuit, a device, a unit, etc. The hardware configuration of the apparatus may be configured to include one or more of the apparatuses shown in the drawings, or may be configured to exclude some of the apparatuses.

[0093] Each functional block of the device (see FIG. 2) is realized by any hardware element of the computer device or a combination of the hardware elements.

[0094] In addition, each function of the device is realized by loading specified software (programs) onto hardware such as processor 1001 and memory 1002, causing processor 1001 to perform calculations, control communication via communication device 1004, and control at least one of reading and writing data in memory 1002 and storage 1003.

[0095] The processor 1001 controls the entire computer by running, for example, an operating system, and may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control unit, an arithmetic unit, and registers.

[0096] The processor 1001 also reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. Furthermore, the various processes described above may be executed by a single processor 1001, or may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted from a network via a telecommunications line.

[0097] The memory 1002 is a computer-readable recording medium and may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 may store a program (program code), a software module, etc., capable of executing a method according to an embodiment of the present disclosure.

[0098] Storage 1003 is a computer-readable recording medium, and may be, for example, at least one of an optical disk such as a Compact Disc ROM (CD-ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray disc), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned recording medium may be, for example, a database, a server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0099] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0100] The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc. to realize, for example, at least one of Frequency Division Duplex (FDD) and Time Division Duplex (TDD).

[0101] The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that outputs to the outside. Note that the input device 1005 and the output device 1006 may be integrated into one device (e.g., a touch panel).

[0102] Furthermore, each device such as the processor 1001 and the memory 1002 is connected to a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0103] Furthermore, the device may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0104] Each aspect / embodiment described in the present disclosure may be applied to at least one of a system using Long Term Evolution (LTE), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, a 4th generation mobile communication system (4G), a 5th generation mobile communication system (5G), Future Radio Access (FRA), New Radio (NR), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), or other suitable system, and a next-generation system enhanced based on these. Furthermore, a combination of multiple systems (e.g., a combination of at least one of LTE and LTE-A and 5G) may also be applied.

[0105] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0106] The aspects / embodiments described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0107] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0108] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0109] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0110] Note that terms described in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Furthermore, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

[0111] As used in this disclosure, the terms "system" and "network" are used interchangeably.

[0112] Furthermore, the information, parameters, etc. described in the present disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information. For example, a radio resource may be indicated by an index.

[0113] In this disclosure, terms such as "base station (BS)," "radio base station," "fixed station," "NodeB," "eNodeB (eNB)," "gNodeB (gNB)," "access point," "transmission point," "reception point," "transmission / reception point," "cell," "sector," "cell group," "carrier," and "component carrier" may be used interchangeably. Base stations may also be referred to by terms such as macrocell, small cell, femtocell, and picocell.

[0114] A base station can accommodate one or more (e.g., three) cells (also called sectors). When a base station accommodates multiple cells, the overall coverage area of ​​the base station can be divided into multiple smaller areas, and each smaller area can be provided with communication services by a base station subsystem (e.g., a small indoor base station (Remote Radio Head: RRH)).

[0115] The terms "cell" or "sector" refer to part or all of the coverage area of ​​a base station and / or base station subsystem that provides communication services within that coverage area.

[0116] In this disclosure, the terms "Mobile Station (MS)," "user terminal," "User Equipment (UE)," "terminal," etc. may be used interchangeably.

[0117] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terminology.

[0118] At least one of the base station and the mobile station may be referred to as a transmitting device, a receiving device, a communication device, etc. At least one of the base station and the mobile station may be a device mounted on a mobile object, the mobile object itself, etc. The mobile object may be a vehicle (e.g., a car, an airplane, etc.), an unmanned mobile object (e.g., a drone, an autonomous vehicle, etc.), or a robot (manned or unmanned). At least one of the base station and the mobile station may also include devices that do not necessarily move during communication operations. For example, at least one of the base station and the mobile station may be an Internet of Things (IoT) device such as a sensor.

[0119] Furthermore, a base station in the present disclosure may be read as a mobile station (user terminal, the same applies hereinafter). For example, the aspects / embodiments of the present disclosure may be applied to a configuration in which communication between a base station and a mobile station is replaced with communication between multiple mobile stations (which may be called, for example, Device-to-Device (D2D) or Vehicle-to-Everything (V2X)). In this case, the mobile station may be configured to have the functions of a base station. Furthermore, terms such as "uplink" and "downlink" may be read as terms corresponding to terminal-to-terminal communication (for example, "side"). For example, terms such as an uplink channel and a downlink channel may be read as a side channel (or sidelink).

[0120] Similarly, a mobile station in the present disclosure may be interpreted as a base station, in which case the base station may have the functions of a mobile station.

[0121] The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using one or more wires, cables, and / or printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0122] The reference signal may also be abbreviated as Reference Signal (RS) and may be called a pilot depending on the applicable standard.

[0123] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0124] The "means" in the configuration of each of the above devices may be replaced with "part," "circuit," "device," etc.

[0125] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some way.

[0126] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.

[0127] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0128] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0129] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0130] (Additional Note) The above disclosure may be expressed as follows: A first feature is a human augmentation platform device including: an acquisition unit that presents a user with a question regarding perception including at least one of taste and smell, and acquires a plurality of pieces of sensitivity information by multiplying the user's perception sensitivity score with basic data of the perception, a comparison unit that compares first sensitivity information selected from the plurality of pieces of sensitivity information acquired by the acquisition unit with second sensitivity information selected from the plurality of pieces of sensitivity information acquired by the acquisition unit, a conversion unit that converts the difference between the first sensitivity score included in the first sensitivity information and the second sensitivity score included in the second sensitivity information into control data by multiplying the basic data, and an actuation unit that operates a predetermined device based on the control data.

[0131] A second feature is that in the first feature, the conversion unit corrects the first sensitivity score to match the second sensitivity score by changing the granularity of components contained in the taste object.

[0132] A third feature is the first or second feature, wherein the conversion unit corrects the first sensitivity score according to an age of the user to match the first sensitivity score with the second sensitivity score.

[0133] A fourth feature is that, in the first to third features, the conversion unit inputs the sensitivity information into an artificial intelligence system to obtain linguistic information in which the sensitivity score is verbalized based on the sensitivity information, and the actuation unit outputs control data including the linguistic information.

[0134] A fifth feature is that, in the first to fourth features, the conversion unit inputs linguistic information verbalized by the user into an artificial intelligence system, thereby obtaining the sensitivity score quantified based on the linguistic information, and the actuation unit outputs the control data using the sensitivity score.

[0135] According to a sixth feature, in any one of the first to fifth features, the conversion unit applies feedback control between the first sensitivity score and the second sensitivity score.

[0136] 10 Human Augmentation System 20 Communication Network 40 Taste Sensor 50 Olfactory Sensor 55 Language Input Unit 60 Language Output Unit 70 Taste / Smell Actuator 100 Human Augmentation Platform Device 110 Sensitivity Information Acquisition Unit 120 Sensor Data Acquisition Unit 125 Language Information Acquisition Unit 130 Body DB 140 Operation DB 150 Comparison Processing Unit 160 Data Conversion Unit 170 Actuation Unit 1001 Processor 1002 Memory 1003 Storage 1004 Communication Device 1005 Input Device 1006 Output Device 1007 Bus

Claims

1. A human augmentation platform device comprising: an acquisition unit that presents a user with a question regarding perception including at least one of taste and smell, and acquires a plurality of pieces of sensitivity information by multiplying the perception sensitivity score of the user by basic data of the perception; a comparison unit that compares first sensitivity information selected from the plurality of pieces of sensitivity information acquired by the acquisition unit with second sensitivity information selected from the plurality of pieces of sensitivity information acquired by the acquisition unit; a conversion unit that converts the difference between a first sensitivity score included in the first sensitivity information and a second sensitivity score included in the second sensitivity information into control data by multiplying the basic data; and an actuation unit that operates a predetermined device based on the control data.

2. The human augmentation platform device according to claim 1, wherein the conversion unit corrects the first sensitivity score to match the second sensitivity score by changing the grain size of components contained in the taste object.

3. The human augmentation platform device according to claim 1, wherein the conversion unit corrects the first sensitivity score according to the age of the user to match it with the second sensitivity score.

4. The human augmentation platform device of claim 1, wherein the conversion unit inputs the sensitivity information into an artificial intelligence system to obtain linguistic information in which the sensitivity score is verbalized based on the sensitivity information, and the actuation unit outputs the control data including the linguistic information.

5. The human augmentation platform device according to claim 1, wherein the conversion unit obtains the sensitivity score quantified based on the linguistic information by inputting the linguistic information verbalized by the user into an artificial intelligence system, and the actuation unit outputs the control data using the sensitivity score.

6. The human augmentation platform device according to claim 1, wherein the conversion unit applies feedback control between the first sensitivity score and the second sensitivity score.

7. A method for expanding perception, comprising the steps of: presenting a user with a question regarding perception including at least one of taste and smell, and acquiring a plurality of pieces of sensitivity information by multiplying the user's perception sensitivity score by basic data of the perception; comparing first sensitivity information selected from the acquired plurality of pieces of sensitivity information with second sensitivity information selected from the acquired plurality of pieces of sensitivity information; converting the obtained sensitivity information into control data by multiplying the difference between the first sensitivity score included in the first sensitivity information and the second sensitivity score included in the second sensitivity information by the basic data; and operating a predetermined device based on the control data.

Images