Biological information detection system, biological information detection method, and program

Abstract

A biological information detection system according to one embodiment of the present technology is provided with one or more biological sensors, one or more feedback devices, and an operation control unit. The one or more biological sensors detect biological information of a user. The one or more feedback devices present sensory feedback to the user. The operation control unit controls the sensory feedback presentation operation of the one or more feedback devices on the basis of detection position information relating to the detection positions of biological information of the one or more biological sensors.

Description

Biological information detection system, biological information detection method, and program

[0001] The present technology relates to a biological information detection system, a biological information detection method, and a program that can be applied to detecting various types of biological information such as heart rate and myoelectric potential.

[0002] Patent Document 1 discloses a biological information processing device that enables highly accurate heart rate measurement.

[0003] International Publication No. 2017 / 199597

[0004] Thus, there is a demand for technology that can improve the detection accuracy of biosensors that detect biometric information of users.

[0005] In view of the above circumstances, an object of the present technology is to provide a biological information detection device, a biological information detection method, and a program that can improve the detection accuracy of a biological sensor.

[0006] To achieve the above object, a biological information detection system according to one embodiment of the present technology includes one or more biological sensors, one or more feedback devices, and an operation control unit. The one or more biological sensors detect biological information of a user. The one or more feedback devices present sensory feedback to the user. The operation control unit controls the presentation operation of the sensory feedback of the one or more feedback devices based on detection position information regarding detection positions of the biological information of the one or more biological sensors.

[0007] In this biometric information detection system, the sensory feedback presentation operation of the feedback device is controlled based on the detected position information of the biometric sensor, thereby improving the detection accuracy of the biometric sensor.

[0008] The operation control unit may control the presentation operation so as to suppress an influence of the presentation of the sensory feedback on detection of the biological information by the one or more biological sensors.

[0009] The operation control unit may select an important biosensor from the one or more biosensors that has a relatively high importance for detecting the biosensor based on detection-related information regarding the detection of the biosensor by the one or more biosensors, including the detection position information, and control the presentation operation so that the impact of the presentation of the sensory feedback on the detection of the biosensor by the important biosensor is suppressed.

[0010] The operation control unit may determine, based on the detection-related information, whether or not there is an operating biosensor among the one or more biosensors that is currently detecting the biosensor information, and may control the presentation operation if there is an operating biosensor.

[0011] The operation control unit may determine, based on the detection-related information, which of the one or more biosensors is in operation to detect the bioinformation, and select the important biosensor from the in operation biosensors.

[0012] The detection-related information may include at least one of detection information detected by the one or more biosensors, generation information generated based on the detection information, and application information related to an application that uses at least one of the detection information and the generation information.

[0013] The biological information detection system may further include a mounting unit configured to hold the one or more biological sensors and the one or more feedback devices and to be wearable by the user. In this case, the detection-related information may include at least one of posture information of the user wearing the mounting unit and information regarding a wearing state of the mounting unit relative to the user.

[0014] The operation control unit may perform at least one of selecting an executive feedback device that performs presentation of the sensory feedback and adjusting parameters related to the presentation operation, as the control of the presentation operation.

[0015] The operation control unit may adjust the intensity of the sensory feedback as the adjustment of the parameter related to the presented operation.

[0016] The detection position information may include information regarding a positional relationship between the detection positions of the one or more biosensors and the sensory feedback presentation positions of the one or more feedback devices.

[0017] The operation control unit may determine a positional relationship between the detection position of the important biosensor and the presentation position of the one or more feedback devices based on the detection position information, and control the presentation operation based on the determined positional relationship.

[0018] The operation control unit may determine the distance between the detection position of the important biosensor and the presentation position of the one or more feedback devices based on the detection position information, and adjust the intensity of the sensory feedback based on the determined distance.

[0019] The one or more feedback devices may apply vibration to the presentation position, and in this case, the operation control unit may adjust the intensity of the vibration based on the distance between the detection position of the important biological sensor and the presentation position of the one or more feedback devices.

[0020] The one or more biosensors may be a plurality of biosensors, and in this case the one or more feedback devices may be a plurality of feedback devices.

[0021] The one or more biosensors may include at least one of a myoelectric potential sensor, a muscle sound sensor, a temperature sensor, a heart rate sensor, a sweat sensor, a velocity sensor, an acceleration sensor, an angular velocity sensor, an IMU (Inertial Measurement Unit) sensor, an inertial sensor, an optical sensor, an image sensor, a strain sensor, and a capacitance sensor.

[0022] The one or more biosensors may be a myoelectric potential sensor.

[0023] The one or more feedback devices may include at least one of a tactile sensation providing device, an electrical stimulation device, an electromagnetic wave generating device, and a thermal sensation providing device.

[0024] A biological information detection system according to another embodiment of the present technology includes the operation control unit.

[0025] A biological information detection method according to one embodiment of the present technology includes detecting biological information of a user with one or more biological sensors, providing sensory feedback to the user with one or more feedback devices, and controlling, by a computer system, the operation of providing the sensory feedback of the one or more feedback devices based on detection position information regarding detection positions of the biological information by the one or more biological sensors.

[0026] A program according to one embodiment of the present technology causes a computer system including one or more biosensors and one or more feedback devices to execute the bioinformation detection method.

[0027] 1 is a schematic diagram showing a configuration example of a bioinformation detection system according to the present technology. FIG. 2 is a block diagram showing a configuration example of each wearable device shown in FIG. 1. FIG. 3 is a schematic diagram showing a specific configuration example of one or more biosensors and one or more feedback devices. FIG. 4 is a flowchart showing an operation example of a wearable device constituting the bioinformation detection system. FIG. 5 is a schematic diagram for explaining an example of control of a presentation operation of sensory feedback. FIG. 6 is a perspective view showing an example of an HMD. FIG. 7 is a schematic diagram showing an external appearance example of a watch-type device. FIG. 8 is a block diagram showing a configuration example of a watch-type device. FIG. 9 is a flowchart showing an operation example of the bioinformation detection system according to the present embodiment. FIG. 10 is a schematic diagram for explaining XR application example 1. FIG. 11 is a schematic diagram for explaining XR application example 1. FIG. 12 is a schematic diagram for explaining XR application example 2. FIG. 13 is a schematic diagram for explaining XR application example 3. FIG. 14 is a schematic diagram showing another configuration example of one or more biosensors (myogenic potential sensors) and one or more feedback devices (vibrators). FIG. 15 is a block diagram showing an example of the hardware configuration of a computer that can be used to build the bioinformation detection system according to the present technology.

[0028] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0029] 1 is a schematic diagram showing a configuration example of a biological information detection system according to the present technology. The biological information detection system 1 is a system capable of both detecting biological information of a user 2 and providing sensory feedback to the user 2.

[0030] For example, as shown in FIG. 1, a biological information detection system 1 can be configured using a plurality of wearable devices 3.

[0031] In the example shown in Figure 1, multiple wearable devices 3 are illustrated, including an HMD (Head Mounted Display) 4 worn on the head, a headband-type (head-mounted) device 5 also worn on the head, a neckband-type device 6 worn around the neck, a bracelet-type device 7 worn on the upper arm, a wristband-type device 8 worn on the wrist, a torso device 9 worn on the chest, a belt-type device 10 worn on the waist, and an anklet-type device 11 worn on the ankle.

[0032] The form of the wearable device 3 constituting the biological information detection system 1 is not limited, and any form of wearable device 3 can be used. Of course, the biological information detection system 1 according to the present technology can also be constituted by a single wearable device 3.

[0033] Furthermore, the present technology is not limited to a wearable device 3 that can be worn by a user 2, but can also be applied to other devices, such as any UI (User Interface) device that can input various operations from a user 2, such as a remote controller.

[0034] 2 is a block diagram showing an example of the configuration of each wearable device 3 shown in FIG. 1. The wearable device 3 according to this embodiment includes a biological information detector 13, a feedback presenter 14, and a controller 15.

[0035] The biometric information detection unit 13 has one or more biometric sensors 16 that detect biometric information of the user 2. In the example shown in Fig. 2, a plurality of biometric sensors 16-1 to 16-n are arranged.

[0036] In the present disclosure, biometric information includes any information related to a living organism. For example, any information indicating the biometric characteristics of user 2 or any information from which the biometric characteristics of user 2 can be extracted is included in biometric information. Also, any information related to the bioactivity of user 2 or any information generated by the activity of user 2 is included in biometric information. In addition, any information related to a living organism is included in biometric information.

[0037] For example, biological information includes body temperature, blood pressure, pulse rate, heart rate, sweat rate, blood glucose level, water content, body fat percentage, weight, height, myoelectric potential, muscle sounds, muscle mass balance, bone density, respiratory rate, lung capacity, blood test results, MRS test results, urine test results, blood alcohol concentration, etc. Biological information also includes information on whether a person is asleep or awake, and the type of sleep state (REM sleep, non-REM sleep, deep sleep, light sleep, etc.).

[0038] The biometric information also includes exercise information related to the exercise performed by the user 2. For example, information can be acquired as biometric information while walking, running, traveling by train, driving, etc. Information related to posture, such as whether the user is sitting, standing, leaning forward, looking to the side, or looking up, can also be acquired as biometric information.

[0039] The biometric information also includes information on the movement, shape, posture, etc. of parts of the body of the user 2, such as the head, face, neck, shoulders, upper arms, forearms, hands (palms), chest, abdomen, waist, thighs, shins, ankles, and feet (feet). The biometric information also includes skeletal information of the user 2.

[0040] Biometric information also includes images of the user 2 captured by a camera, audio information of the user 2 acquired by a microphone, etc. In the present disclosure, images include both still images and moving images (video).

[0041] In the present disclosure, the biosensor 16 includes any device capable of detecting signals, data, etc. including the above-described bioinformation. In addition, devices capable of detecting signals and data that can generate the above-described bioinformation are also included in the biosensor according to the present technology.

[0042] Examples of the biosensor 16 include an electromyography (EMG) sensor, a muscle sound sensor, a temperature sensor, a heart rate sensor, a sweat sensor, a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), an inertial measurement unit (IMU) sensor, an inertial sensor, an optical sensor, an image sensor (camera), a distortion sensor, and a capacitance sensor. Of course, any other device capable of detecting bioinformation may be used. Note that an electromyography sensor is also called an electromyography sensor.

[0043] The feedback providing unit 14 has one or more feedback devices 17 that provide sensory feedback to the user 2. In the example shown in Fig. 2, a plurality of feedback devices 17-1 to 17-m are arranged.

[0044] In the present disclosure, sensory feedback includes feedback that stimulates various senses of user 2, such as tactile (skin sensation) feedback, vibration feedback, force feedback, thermal (temperature sensation) feedback, visual feedback, auditory feedback, olfactory feedback, and taste feedback.

[0045] The feedback device 17 includes any device capable of providing the above-described sensory feedback. Typically, a feedback device 17 that provides tactile (skin-sensing) feedback is used, but the scope of application of the present technology is not limited thereto.

[0046] The feedback device 17 may be, for example, a tactile sensation presentation device configured with a vibrator (vibration actuator) such as a motor, a piezoelectric element (piezoelectric actuator), or the like. Other examples include an electrical stimulation device, an electromagnetic wave generator, and a temperature presentation device. Any other device capable of presenting sensory feedback may also be used. The presentation of sensory feedback can also be referred to as the output of sensory feedback.

[0047] The controller 15 controls the operation of each block of the wearable device 3. The controller 15 controls the operation of one or more biosensors 16 of the biometric information detection unit 13. The controller 15 also controls the operation of one or more feedback devices 17 of the feedback presentation unit 14.

[0048] The controller 15 includes hardware necessary for a computer, such as a processor such as a CPU, GPU, or DSP, memory such as a ROM or RAM, and a storage device such as an HDD. The processor loads a program according to the present technology stored in the memory or storage device into the RAM and executes the program, thereby executing the biological information detection method and the sensory feedback presentation method according to the present technology.

[0049] The controller 15 may be, for example, a programmable logic device (PLD) such as a field programmable gate array (FPGA), or another device such as an application specific integrated circuit (ASIC).

[0050] In the present embodiment, the processor of the controller 15 executes a program (e.g., an application program) according to the present technology, thereby configuring the operation control unit 18 as a functional block. Of course, dedicated hardware such as an IC (integrated circuit) may be used to realize this functional block.

[0051] The program is installed in the wearable device 3 via, for example, various recording media. Alternatively, the program may be installed via the Internet or the like. There are no limitations on the type of recording media on which the program is recorded, and any computer-readable recording media may be used. For example, any computer-readable non-transitory storage medium may be used.

[0052] 1 , when a plurality of wearable devices 3 are used, the biological information detection method according to the present technology may be executed by cooperation between the controllers 15 of the respective wearable devices 3. Alternatively, the operation of each wearable device 3 may be controlled by the controller 15 of a single wearable device 3 (e.g., HMD 4) among the plurality of wearable devices 3, and the biological information detection method according to the present technology may be executed.

[0053] Alternatively, when a computer such as a server device configured on a network is connected to multiple wearable devices 3 so that they can communicate with each other, the computer on the network may be equipped with the functions of the controller 15 shown in Fig. 2. The computer on the network may then control the operation of each wearable device 3, and the biological information detection method according to the present technology may be executed.

[0054] 3 is a schematic diagram showing a specific example configuration of one or more biosensors 16 and one or more feedback devices 17. In FIG. 3, a wristband-type device 8 is shown as an example of the wearable device 3.

[0055] The wristband-type device 8 has a wearing band 20, six surface myoelectric potential sensors (hereinafter simply referred to as myoelectric potential sensors) 21 (21a to 21f), and six vibrators 22 (22a to 22f). The six myoelectric potential sensors 21a to 21f function as the plurality of biosensors 16 shown in Fig. 2. The six vibrators 22a to 22f function as the plurality of feedback devices 17 shown in Fig. 2.

[0056] 3, the six myoelectric potential sensors 21a to 21f and the six vibrators 22a to 22f are held by a wearing band 20. The wearing band 20 is configured to be wearable on the wrist of the user 2, and functions as an embodiment of the wearing unit according to the present technology. The specific configuration of the wearing band 20 is not limited and may be designed arbitrarily.

[0057] The six myoelectric potential sensors 21a to 21f are arranged at approximately equal intervals around the wrist of the user 2. The six myoelectric potential sensors 21a to 21f are arranged so that the electrodes come into contact with the skin of the user 2's wrist.

[0058] 3, the positions where the electrodes of the myoelectric potential sensors 21a to 21f are in contact are detection positions DPa to DPf of the myoelectric potential sensors 21a to 21f, respectively. Each of the myoelectric potential sensors 21 detects the myoelectric potential of a muscle present in the vicinity of the detection position DP as a measurement target.

[0059] 3, the six vibrators 22a to 22b are arranged at approximately the same positions as the six myoelectric potential sensors 21a to 21f. Therefore, in this embodiment, when each of the six vibrators 22a to 22b operates, vibrations are applied to the detection positions DPa to DPf of the six myoelectric potential sensors 21a to 21f.

[0060] That is, in this embodiment, the vibration feedback presentation positions PP (PPa to PPf) of the six vibrators 22a to 22b are substantially the same as the detection positions DPa to DPf of the six myoelectric potential sensors 21a to 21f.

[0061] When applying the present technology, it is not necessary to strictly define the detection positions DPa to DPf of the six myoelectric potential sensors 21a to 21f and the vibration feedback presentation positions PPa to PPf of the six vibrators 22a to 22b. As long as it is possible to define the approximate detection positions DP and the sensory feedback presentation positions PP for the user 2, a sufficient effect is achieved.

[0062] For example, in the case of a sensor such as the myoelectric potential sensor 21, in which the contact position of the myoelectric potential sensor 21 with the user 2 is the detection position DP, the present technology can also be applied with the position of the sensor as the detection position of the sensor.

[0063] Furthermore, when vibration is applied to the position where the vibrator 22 contacts the user 2, the present technology can also be applied with the position of the vibrator 22 as the presentation position PP of the sensory feedback. Of course, the effect of the present technology can be improved by defining the detection position DP and the presentation position PP with high accuracy.

[0064] The wearable device 3 having one or more biosensors 16 and one or more feedback devices 17 as exemplified in FIGS. 3 and 4 may also be called a wearable input / output device.

[0065] 4 is a flowchart showing an example of the operation of the wearable device 3 constituting the biological information detection system 1. The operation control unit 18 of the controller 15 shown in FIG. 2 monitors whether the conditions for providing sensory feedback are satisfied (step 101).

[0066] The sensory feedback presentation condition is a condition that serves as a criterion for determining whether or not to present sensory feedback, and any condition can be set. For example, the presentation condition can be set in relation to the detection of biological information by the multiple biological sensors 16 shown in FIG.

[0067] For example, it is possible to set the presentation condition for sensory feedback as to whether or not the detection information detected by the biosensor 16 satisfies a predetermined condition. It is also possible to set the presentation condition for sensory feedback as to whether or not the generated information generated based on the detection information detected by the biosensor 16 satisfies a predetermined condition.

[0068] Of course, there may be cases where the generated information generated based on the detection information of the biosensor 16 is included in the bioinformation of the user 2. For example, assume that information on the muscle movement state is generated based on the detection results of the myoelectric potential sensor 21 used as the biosensor 16. In this case, the detection results of the myoelectric potential sensor 21 are included in the detection information, and the information on the muscle movement state is included in the generated information. Furthermore, both the detection results of the myoelectric potential sensor 21 and the information on the muscle movement state are included in the bioinformation.

[0069] Furthermore, a sensor that measures myoelectric potential and generates information on the state of muscle movement is used as the biosensor 16. In this case, both the measured myoelectric potential and the information on the state of muscle movement generated based on the myoelectric potential are included in the detection information. Of course, both the measured myoelectric potential and the information on the state of muscle movement generated based on the myoelectric potential are included in the bioinformation.

[0070] It is also possible to set a condition unrelated to the detection of biometric information by the multiple biosensors 16 shown in Fig. 2 as a presentation condition for sensory feedback. For example, assume that a user 2 shown in Fig. 1 is experiencing an XR (Extended Reality) application using an HMD 4. A condition related to the XR application (a condition unrelated to the detection of biometric information by the multiple biosensors 16) can be set as a presentation condition for sensory feedback. Alternatively, it is also possible to set whether or not an email or the like has been received as a presentation condition for sensory feedback.

[0071] In addition, XR includes any technology that combines the real world (real space) with the virtual world (virtual space), such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality).

[0072] If the conditions for presenting sensory feedback are met (Yes in step 101), the operation control unit 18 controls the presentation operation of sensory feedback of one or more feedback devices 17 based on detection position information regarding the detection position of bioinformation of one or more biosensors 16 (step 102).

[0073] In the example shown in FIG. 2 , the operation control unit 18 controls the sensory feedback presentation operations of the plurality of feedback devices 17 based on detection position information relating to the detection positions of the biological information of the plurality of biological sensors 16 .

[0074] In the example shown in Figure 3, the operation control unit 18 controls the vibration feedback presentation operation of the six vibrators 22a to 22f based on detection position information regarding the detection positions DPa to DPf of the biological information of the six myoelectric potential sensors 21a to 21f.

[0075] The detection position information regarding the detection position of the bioinformation of one or more biosensors 16 includes any position information regarding the detection position. For example, information regarding the detection position of the biosensor 16 defined for the user 2 is included in the detection position information. The detection position information also includes the position information of the biosensor 16. In the example shown in FIG. 3 , the detection position information regarding the detection position DP of the bioinformation of the myoelectric potential sensor 21 includes information regarding the detection position DP and the position information of the myoelectric potential sensor 21.

[0076] The detection position information regarding the detection position includes information regarding the positional relationship between the detection position of one or more biosensors 16 and the presentation position of the sensory feedback of one or more feedback devices 17. In the example shown in Fig. 3, information regarding the positional relationship between the detection position DP of the myoelectric potential sensor 21 and the presentation position PP of the vibrator 22 is included in the detection position information.

[0077] In addition, the information regarding the positional relationship between the detection position and the presentation position includes information regarding the positional relationship between the detection position and the presentation position, information regarding the positional relationship between the position of the biosensor 16 and the presentation position, information regarding the positional relationship between the detection position and the feedback device 17, and information regarding the positional relationship between the position of the biosensor 16 and the feedback device 17.

[0078] In the example shown in Figure 3, the information regarding the positional relationship between the detection position DP and the presentation position PP includes information regarding the positional relationship between the detection position DP and the presentation position PP, information regarding the positional relationship between the position of the electromyography sensor 21 and the presentation position PP, information regarding the positional relationship between the detection position DP and the vibrator 22, and information regarding the positional relationship between the position of the electromyography sensor 21 and the vibrator 22.

[0079] For example, information on the positional relationship between the biosensor 16 and the feedback device 17 can be used as information on the positional relationship between the two devices within the wearable device 3. In other words, the information on the positional relationship between the biosensor 16 and the feedback device 17 can be determined even when the wearable device 3 is not worn by the user 2.

[0080] Specific examples of information on the positional relationship include a specific distance value, the magnitude of the distance (information on relative size), and the like.

[0081] In step 102, the operation control unit 18 controls the operation of providing sensory feedback from the one or more feedback devices 17 so as to reduce the influence of the presentation of sensory feedback on the detection of biological information by the one or more biological sensors 16.

[0082] 5A and 5B are schematic diagrams illustrating an example of control of a presentation action of sensory feedback, Fig. 5A is a schematic diagram illustrating a comparative example in which control of a presentation action according to the present technology is not performed, and Fig. 5B is a schematic diagram illustrating an example of control of a presentation action according to the present technology.

[0083] 3, suppose that six myoelectric potential sensors 21a to 21f are operating to measure muscle M near detection position DPd of myoelectric potential sensor 21d. Then, suppose that the condition for providing sensory feedback in step 101 is satisfied while the six myoelectric potential sensors 21a to 21f are detecting biological information.

[0084] In the example shown in Fig. 5A, six vibrators 22a to 22f apply vibrations at the vibration feedback presentation position PP with the same vibration strength (amplitude) to present sensory feedback (vibration feedback) to the user 2. Note that Fig. 5 schematically illustrates the vibrators 22 vibrating, reflecting the magnitude of the vibrations.

[0085] 5A , vibrations applied to the detection position DPd where the electrodes of the myoelectric potential sensor 21d are in contact with the user 2 may introduce noise into the myoelectric potential signal measured by the myoelectric potential sensor 21d, reducing the measurement accuracy. That is, the detection accuracy of the biological information detected by the myoelectric potential sensor 21d may be reduced, and the detection accuracy of biological information such as the operating state of the muscle M may also be reduced.

[0086] In the control example of the presentation action according to the present technology shown in FIG. 5B, the intensity of the vibrations applied by the six vibrators 22a to 22f is adjusted based on the distance between the detection position DPd of the electromyography sensor 21d, which is positioned closest to the muscle M, and the presentation positions PPa to PPf of the six vibration feedbacks.

[0087] 5B , the distance from the detection position DPd of the myoelectric potential sensor 21 d to the presentation position PPa of the vibrator 22 a is the greatest. The presentation operation of the vibrator 22 a is controlled so that vibration is applied to the presentation position PPa with the greatest vibration intensity.

[0088] The distance to the presentation position PPb of the vibrator 22b and the distance to the presentation position PPf of the vibrator 22f from the detection position DPd of the myoelectric potential sensor 21d are the second largest. These distances are also equal to each other. The presentation operation is controlled so that the vibrators 22b and 22f apply vibrations at the presentation positions PPb and PPf with the second largest vibration intensity (i.e., the vibration intensity is reduced).

[0089] The distance from the detection position DPd of the myoelectric potential sensor 21d to the presentation position PPc of the vibrator 22c and the distance from the detection position DPd of the myoelectric potential sensor 21d to the presentation position PPe of the vibrator 22e are the third largest. These distances are also equal to each other. The presentation operation of the vibrators 22c and 22e is controlled so that vibrations are applied to the presentation positions PPb and PPf with the third largest vibration intensity (i.e., the vibration intensity is further reduced).

[0090] The distance from the detection position DPd of the myoelectric potential sensor 21d to the presentation position PPc of the vibrator 22b is the smallest (almost zero). The presentation operation is controlled so that vibration is not applied to the vibrator 22d. In other words, vibration feedback is not performed for the vibrator 22d.

[0091] In the example shown in Figure 5B, the distance between the detection position DPd of the electromyography sensor 21d and the presentation positions PPa to PPf of the six vibration feedbacks is determined, and the presentation operation of the vibration feedback by the six vibrators 22a to 22f is controlled so that the vibration intensity decreases as the distance decreases.

[0092] This makes it possible to suppress noise from being introduced into the myoelectric potential signal measured by the myoelectric potential sensor 21 d due to the presentation of vibration feedback, thereby improving the detection accuracy of the myoelectric potential sensor 21 d and enabling detection of biological information such as the operating state of the muscle M with high accuracy.

[0093] When controlling the sensory feedback presentation operation shown in step 102, for example, the operation control unit 18 selects a biosensor 16 that is relatively important for detecting bioinformation (hereinafter referred to as an important biosensor) from one or more biosensors 16. Then, the presentation operation is controlled so as to suppress the influence of the presentation of sensory feedback on the detection of bioinformation by the important biosensor.

[0094] The selection of the important biological sensors is performed based on detection-related information related to the detection of biological information by one or more biological sensors 16. The detection-related information includes any information related to the detection of biological information. For example, the detection-related information includes detection position information related to the detection position of the biological information by the one or more biological sensors 16.

[0095] The detection-related information also includes detection information detected by the biosensor 16 and generated information generated based on the detection information detected by the biosensor 16. The detection-related information also includes application information related to an application, such as an XR application, that uses at least one of the detection information and the generated information.

[0096] The detection-related information also includes posture information of the user 2 wearing the attachment unit such as the attachment belt shown in FIG. 5, and information about the attachment state of the attachment unit relative to the user 2.

[0097] The detection-related information also includes information indicating that the biosensor 16 is detecting bioinformation, information indicating that the biosensor 16 is not detecting bioinformation, and information that enables determination of whether the biosensor 16 is detecting bioinformation, etc. Based on such detection-related information, it is possible to select important biosensors.

[0098] For example, the biosensor 16 that contributes most to detecting biometric information or the biosensor 16 that has the largest output is selected as the important biosensor. Alternatively, in a situation where it is desired to measure the extensor / flexor muscles of a specific finger such as the index finger or middle finger, the biosensor 16 whose detection position is set near the extensor / flexor muscles of that finger is selected as the important biosensor.

[0099] Alternatively, when the user 2 is experiencing an XR application or the like, the biometric information to be detected may be specified based on the content of the application, etc. In such a case, the biometric sensor 16 for detecting the biometric information is selected as the important biometric sensor. Alternatively, the method for selecting the important biometric sensor may be set arbitrarily.

[0100] An important biological sensor may be selected based on whether the biological sensor 16 is in the process of detecting biological information. For example, the operation control unit 18 may determine, based on the detection-related information, which of the one or more biological sensors 16 is in the process of detecting biological information, and select an important biological sensor from the active biological sensors.

[0101] In addition, the operation control unit 18 may determine, based on the detection-related information, whether or not there is an operating biosensor among one or more biosensors 16 that is currently detecting biosensor information, and if there is an operating biosensor, the operation of presenting sensory feedback may be controlled.

[0102] In other words, it is possible to set the sensory feedback presentation control to be executed only when there is an operating biosensor during the bioinformation detection operation, and not to execute the sensory feedback presentation control when there is no operating biosensor.

[0103] 5B, the myoelectric potential sensor 21d is selected as the important biological sensor from the six myoelectric potential sensors 21a to 21f. For example, the important biological sensor can be selected based on the myoelectric potential signals, which are the detection information of each of the myoelectric potential sensors 21a to 21f.

[0104] Alternatively, it is also possible to select an important biosensor based on generated information generated based on myoelectric potential signals detected by the myoelectric potential sensors 21 a to 21 f. For example, assume that hand shape, gestures performed with the hand, magnitude of force, fatigue level, etc. are generated as generated information based on the myoelectric potential signals. Based on such generated information, it is also possible to select, for example, the myoelectric potential sensor 21 that can detect bioinformation necessary for gesture determination as an important biosensor.

[0105] Alternatively, since in this scene it is necessary to determine the hand movements of user 2 based on application information relating to an application that uses electromyography signals and gesture information generated based on the electromyography signals, it is also possible to select an electromyography sensor 21 that can detect bioinformation for making such a determination as the important biometric sensor.

[0106] Examples of controlling the sensory feedback presentation operation include selecting a feedback device 17 (hereinafter referred to as an execution feedback device) that performs the presentation of sensory feedback, and adjusting parameters related to the presentation operation.

[0107] An example of adjusting the parameters related to the presented action is adjusting the intensity of the sensory feedback. Note that the parameters related to the presented action can also be said to be adjusting the output parameters of the sensory feedback.

[0108] 5B, vibrators 22a to 22c, 22e, and 22f are selected as performance feedback devices so as to suppress the influence of the vibration feedback on the detection of biological information by the important biological sensor (myogenic potential sensor 21d). Also, the strength of the vibration is adjusted to adjust the strength of the sensory feedback.

[0109] In the example shown in Figure 5B, the positional relationship (magnitude of distance) between the detection position DPd of the important biosensor (myogenic potential sensor 21d) and the presentation positions PPa to PPd of the six vibrators 22a to 22d is determined based on the detection position information, and the vibration intensity is adjusted based on the determined positional relationship (magnitude of distance).

[0110] Specifically, the vibration feedback presentation operation of the six vibrators 22a to 22f is controlled so that the vibration intensity decreases as the distance between the detection position DPd of the electromyography sensor 21d selected as the important biosensor and the presentation position PP of the six vibration feedbacks decreases.

[0111] The presenting operation is not limited to this control, and various presentation methods that can suppress the influence of the presentation of sensory feedback on the detection of biological information by the important biological sensor may be adopted.

[0112] For example, it is possible to control so that only the vibrator 22a, which is the farthest distance, is driven with the maximum vibration intensity.Furthermore, the vibrator 22d, which is the shortest distance, is not limited to being not driven (zero vibration) but may be controlled to generate weak vibrations that bring noise within an acceptable range.

[0113] When sensory feedback is presented, it is possible to adopt any presentation control that suppresses changes in the detection situation (environment) of biological information by the important biological sensor.

[0114] 4, the operation control unit 18 determines whether the presentation of sensory feedback has ended. For example, when the conditions for providing sensory feedback are no longer satisfied or when the user 2 inputs an instruction to end the presentation of sensory feedback, the presentation of sensory feedback is ended (Yes in step 103), and the process returns to step 101.

[0115] If the presentation of the sensory feedback has not ended (No in step 103), the process returns to step 102, and the control of the presentation operation of the sensory feedback continues. In this case, the control content of the presentation operation of the sensory feedback may be dynamically changed.

[0116] For example, the selection of the performance feedback device and the adjustment of parameters related to the presented motion may be dynamically changed. For example, in the example shown in Fig. 5B, the measurement target muscle M may be changed in the order of the muscle near the detection position DPd, the muscle near the detection position DPf, and the muscle near the detection position DPb.

[0117] In this case, the important biosensor is also changed in the order of myoelectric potential sensor 21d → myoelectric potential sensor 21f → myoelectric potential sensor 21b. Accordingly, the control of the performance feedback device and the parameters (amplitude) related to the presented motion are also changed. In this way, it is possible to dynamically change the control content of the presented motion. As a result, it is possible to improve the detection accuracy of the biosensor 16.

[0118] [Detailed Embodiment of Biological Information Detection System] A detailed embodiment of the biological information detection system 1 according to the present technology will be described. In this embodiment, a user 2 wears an HMD 4 on his / her head and a watch-type device 24, which is a watch-type wearable device 3, on his / her wrist. The user 2 can then experience an XR application.

[0119] 6 is a perspective view showing an example of the HMD 4. The HMD 4 has a frame 25 in the shape of glasses, and a left eye lens 26a and a right eye lens 26b attached to the frame 25.

[0120] The HMD 4 also has a left-eye display 27a and a right-eye display 27b, which are transmissive displays. The left-eye display 27a and the right-eye display 27b are disposed so as to cover a portion of the left-eye lens 26a and the right-eye lens 26b, respectively.

[0121] The left eye display 27 a and the right eye display 27 b display a left eye image and a right eye image, respectively. The user 2 wearing the HMD 4 can view the left eye image and the right eye image at the same time as viewing the real scenery. This allows the user 2 to experience an AR application.

[0122] The HMD 4 also has an outward-facing camera 28 that is arranged facing outward in the center of the frame 25. The outward-facing camera 28 is capable of capturing an image of the real space in front of the user 2.

[0123] The HMD 4 is also equipped with a speaker, a communication unit, a connector, operation buttons, various sensors (including a biosensor 16), a feedback device 17, etc. The operations of these devices are controlled by a controller having hardware necessary for a computer, such as a processor, memory, and storage device.

[0124] Furthermore, the HMD 4 is not limited to the see-through type as shown in Fig. 6, and an immersive type HMD may be used. In this case, the user 2 can experience a VR application.

[0125] Fig. 7 is a schematic diagram showing an example of the appearance of the watch-type device 24. Fig. 8 is a block diagram showing an example of the configuration of the watch-type device 24.

[0126] As shown in Fig. 7, the watch-type device 24 has a wearing band 30, a main body 31, and a display 32. As shown in Fig. 8, the watch-type device 24 has a biological information detection unit 13, a feedback presentation unit 14, and a controller 15, similar to the wearable device 3 shown in Fig. 2.

[0127] The biological information detection unit 13 is provided with a plurality of biological sensors 16. The feedback presentation unit 14 is provided with a plurality of feedback devices 17.

[0128] As shown in FIG. 7 , a wearing band 30 is connected to the top and bottom of the main body 25, and by wearing the wearing band 30 on the wrist of the user 2, the watch-type device 24 can be worn on the wrist of the user 2.

[0129] The plurality of biosensors 16 and the plurality of feedback devices 17 are arranged on the back surface of the main body 31 on the wrist side and on the inner peripheral surface of the wrist side of the wearing band 30. For example, similar to the wristband-type device 8 shown in FIG. 3, it is also possible to arrange six myoelectricity sensors 21a to 21f and six vibrators 22a to 22f at approximately equal intervals around the wrist of the user 2.

[0130] In this case, for example, the myoelectric potential sensor 21a and the vibrator 22a are provided on the back surface of the main body 31. The other myoelectric potential sensors 21b to 21f and the other vibrators 22b to 22f are arranged on the inner peripheral surface of the wearing band 30. In this way, in the example shown in Fig. 7, the wearing band 30 and the main body 31 realize one embodiment of the wearing unit according to the present technology. Of course, the present technology is not limited to such a configuration.

[0131] 8 , the processor of the controller 15 executes a program (e.g., an application program) according to the present technology, thereby configuring functional blocks such as a recognition unit 33, an application unit 34, a measurement position calculation unit 35, and a presentation operation control unit 36. Dedicated hardware such as an IC (integrated circuit) may be used to realize each functional block.

[0132] In the example shown in FIG. 8, the recognition unit 33, the application unit 34, the measurement position calculation unit 35, and the presentation operation control unit 36 ​​cooperate with each other to realize the function of the operation control unit 18 shown in FIG.

[0133] Note that all or part of the functional blocks of the controller 15 shown in Fig. 8 may be configured in the controller mounted on the HMD 4. Alternatively, all or part of the functional blocks of the controller 15 shown in Fig. 8 may be configured by a computer configured on a network.

[0134] In other words, the biometric information detection method according to the present technology may be executed by the watch-type device 24 alone, or the biometric information detection method according to the present technology may be executed by the watch-type device 24, the HMD 4, or a computer on a network working together.

[0135] The recognition unit 33 generates biometric information (generated information) of the user 2 based on biometric information (detected information) detected by the multiple biometric sensors 16. For example, the recognition unit 33 estimates information related to physical activity, such as the shape of the hand, gestures made with the hand, the magnitude of force generated, and fatigue level. In addition, various other types of biometric information related to the user 2 may be generated and estimated.

[0136] There are no particular limitations on the specific technology (algorithm) for generating various types of bioinformation based on the detection information detected by the multiple biosensors 16, and any technology (algorithm) may be used. For example, any machine learning algorithm using a deep neural network (DNN), a recurrent neural network (RNN), a convolutional neural network (CNN), or the like may be used. For example, by using AI (artificial intelligence) that performs deep learning, it is possible to generate highly accurate bioinformation.

[0137] The application of a machine learning algorithm may be performed for any process within the present disclosure, i.e., any process described within the present disclosure may be subjected to a process using machine learning.

[0138] The application unit 34 controls the XR application based on the biological information such as physical activity generated by the recognition unit 33. For example, various processes necessary for the user 2 to experience the XR application are executed, such as display control of virtual objects displayed on the HMD 4 and output control of sounds (including virtual sounds) output from the speakers.

[0139] In this embodiment, the application unit 34 determines whether or not to present sensory feedback.

[0140] The measurement position calculation unit 35 calculates detection position information of the biosensors 16 that have a large influence on the generation of bioinformation by the recognition unit 33, based on detection information (biological signals such as myoelectric potential signals) detected by the multiple biosensors 16, bioinformation (recognition results of physical activity, etc.) generated by the recognition unit 33, and application information (information on the application state, operation, etc.) related to the XR application output from the application unit 34.

[0141] The biosensor 16 having a large influence on the generation of biometric information by the recognition unit 33 corresponds to an embodiment of an important biosensor having a relatively high importance for the detection of biometric information according to the present technology. Hereinafter, the biosensor 16 having a large influence on the generation of biometric information by the recognition unit 33 may be referred to as an important biosensor.

[0142] The presentation operation control unit 36 ​​controls the presentation operation so as to suppress the influence of the presentation of sensory feedback on the detection of biological information by the important biological sensor. For example, the presentation operation control unit 36 ​​selects an execution feedback device that executes the presentation of sensory feedback, adjusts parameters related to the presentation operation, etc. The control of the presentation operation according to the present technology can also be called optimization of sensory feedback.

[0143] 9 is a flowchart showing an example of the operation of the biological information detection system 1 according to this embodiment. The user 2 wears the HMD 4 and the watch-type device 24. Then, the HMD 4 and the watch-type device 24 are started up (step 201). For example, the user 2 presses the power buttons of both devices to start up both devices. Of course, the start-up method is not limited to this.

[0144] An XR application using the biological information detection system 1 according to the present technology is launched (step 202). For example, the XR application is launched by the user 2 performing a predetermined operation on the display 32 of the watch-type device 24, or by issuing an instruction by voice input. Of course, the launching method is not limited to these.

[0145] In response to the activation of both devices in step 201 and the activation of the XR application in step 202, the multiple biosensors 16, the multiple feedback devices 17, the recognition unit 33, the application unit 34, the measurement position calculation unit 35, and the presentation operation control unit 36 ​​configured in the controller 15 shown in FIG. 8 are also activated.

[0146] Sensing is performed by the multiple biosensors 16 to detect biometric information of the user 2 (step 203). The recognition unit 33 generates biometric information of the user 2 based on the detection information detected by the biosensors 16. For example, the recognition unit 33 estimates information related to the physical activity of the user 2 (step 204).

[0147] The application unit 34 controls the XR application based on the recognition result of the recognition unit 33 (step 205). The recognition unit 33 and the application unit 34 determine whether the conditions for presenting sensory feedback are satisfied (step 206).

[0148] For example, whether or not the detection information detected by the biosensor 16 satisfies a predetermined condition can be set as the presentation condition for sensory feedback. Also, whether or not the generated information generated based on the detection information detected by the biosensor 16 satisfies a predetermined condition can be set as the presentation condition for sensory feedback. Alternatively, a condition related to an XR application can be set as the presentation condition for sensory feedback. Alternatively, whether or not an email or the like has been received can be set as the presentation condition for sensory feedback.

[0149] If the conditions for providing sensory feedback are not met (No in step 206), the process returns to step 203. Whether the conditions for providing sensory feedback are met is monitored while the biometric information is detected by the biometric sensor 16, the biometric information is generated by the recognition unit 33, and the XR application is controlled by the application unit 34.

[0150] If the conditions for providing sensory feedback are met (Yes in step 206), the application unit 34 transmits a request to the presentation operation control unit 36 ​​to perform sensory feedback (step 207).

[0151] The measurement position calculation unit 35 selects important biosensors based on the detection information (biological signals such as myoelectric potential signals) detected by the multiple biosensors 16, the bioinformation (recognition results of physical activity, etc.) generated by the recognition unit 33, and the application information regarding the XR application output from the application unit 34 (information on the application status, operation, etc.), and calculates the detection position information of the required biosensors (step 208).

[0152] The presentation operation control unit 36 ​​controls the presentation operation so as to reduce the effect of the presentation of sensory feedback on the detection of biological information by the important biological sensor (step 209). For example, the presentation operation control unit 36 ​​selects an execution feedback device that performs the presentation of sensory feedback, adjusts parameters related to the presentation operation, and so on.

[0153] The plurality of feedback devices 17 present sensory feedback in accordance with the control of the presentation operation by the presentation operation control unit 36 ​​(step 210). For example, the execution feedback device selected by the presentation operation control unit 36 ​​presents sensory feedback with the parameters of the presentation operation adjusted by the presentation operation control unit 36.

[0154] The recognition unit 33 executes signal processing to reduce the influence of the presentation of sensory feedback on the detection of bioinformation by the important biosensors and on the generation of bioinformation by the recognition unit 33. Specifically, noise reduction processing is executed in accordance with the selection of an execution feedback device by the presentation operation control unit 36 ​​and the adjustment of parameters related to the presentation operation (step 211).

[0155] The XR application is terminated by an input from the user 2 or the like (step 212). Of course, a step of determining whether the XR application has been terminated may be provided.

[0156] The user 2 stops the operation of the HMD 4 and the watch-like device 24 and removes both devices. For example, the user 2 presses the power buttons of both devices to turn off the power of both devices. Then, both devices are removed from the user 2 (step 214).

[0157] [Specific Example of XR Application] A specific example of an XR application using the biological information detection system 1 according to the present technology will be described.

[0158] (XR Application Example 1) The user 2 can experience the following XR Application Example 1 by wearing the HMD 4 shown in Fig. 6 and the watch-type device 24 shown in Fig. 7 and Fig. 8. As the multiple biosensors 16 and multiple feedback devices 17, multiple myoelectric potential sensors 21a to 21f and multiple vibrators 22a to 22f are arranged around the wrist as shown in Fig. 4.

[0159] In addition, in this XR application example 1, hand tracking is possible using images captured by the outward-facing camera 28 of the HMD 4. Note that the specific configuration and algorithm for realizing the hand tracking function are not limited, and any configuration or algorithm may be adopted.

[0160] 10 and 11 are schematic diagrams for explaining XR application example 1. In Fig. 10, the watch-type device 24 is not shown.

[0161] In the XR application example 1, the user 2 can manipulate a virtual object 38 in the XR space by performing a specific gesture and changing the strength of the force used when performing the gesture. In the XR application example 1, vibration feedback is presented in accordance with the timing of changes in the strength of the force applied by the user 2.

[0162] 10 , a “pinching” action of bringing the thumb 39 and index finger 40 into contact with each other is set as a specific gesture. At this time, the content of the operation on the virtual object 38 is changed based on the magnitude of the force (pinching force) with which the thumb 39 and index finger 40 are brought into contact with each other.

[0163] 11 , when the strength of the force is between 0% and 20% of the maximum pinch force set in advance by user 2, no operation is assigned to virtual object 38. In other words, when thumb 39 and index finger 40 are lightly touching each other, nothing happens even if the tips of both fingers (contact portions) touch virtual object 38.

[0164] When the strength of the force is between 20% and 40%, a "select" operation on the virtual object 38 is assigned. That is, the user touches the virtual object 38 with the tips of both fingers while applying some force to the contact between the thumb 39 and index finger 40. This inputs a selection operation by the user 2 on the virtual object 38. Note that if the tips of both fingers are moved to another position away from the virtual object 38 with the same strength of force, the selection on the virtual object 38 is canceled.

[0165] When the force strength is between 40% and 60%, a "grab and move" operation on the virtual object 38 is assigned. That is, the user applies even more force to the thumb 39 and index finger 40, bringing them together, and then touches the virtual object 38 with the tips of both fingers. Then, the user moves the tips of both fingers from the virtual object 38 to another position. This causes the virtual object 38 to move in accordance with the movement of the tips of both fingers. In this way, by increasing the magnitude of the force, it is possible to move the virtual object 38 in the XR space.

[0166] If the strength of the force is greater than 60%, a "delete" operation for the virtual object 38 is assigned. That is, the user applies a large amount of force to the thumb 39 and index finger 40, pressing them together firmly, and then touches the virtual object 38 with the tips of both fingers. This causes a deletion process for the virtual object 38 to be executed. For example, a virtual representation is produced in which the virtual object 38 disappears from the XR space.

[0167] In such an XR application, by using the present biometric information detection system 1, it is possible to provide feedback by vibration that the content of the assigned operation has changed in accordance with changes in the pinching force of the thumb 39 and index finger 40.

[0168] Specifically, after the hand posture condition is met (the hand assumes a pinch posture), vibration feedback is provided each time the magnitude of the force increases by a certain percentage (20%) to indicate to User 2 that the operation has changed. This allows User 2 to properly grasp the current operation mode and allows them to experience an XR application with high usability.

[0169] For example, suppose that all of the vibrators 22 apply vibrations to the operating position DP with the same strength, as illustrated in Fig. 5A, without using the present biological information detection system 1. In this case, the waveform of the myoelectric potential is significantly distorted when vibrations are applied, which is likely to deteriorate the accuracy of detecting the strength of the force applied by the user 2. As a result, it becomes difficult to switch to an appropriate operation mode, and usability decreases.

[0170] In this embodiment, the biological information detection system 1 can select two myoelectric potential sensors 21 as important biological sensors for the recognition of the "pinching" action and the estimation of the pinching force by the recognition unit 33: one myoelectric potential sensor 21 whose detection position DP is set near the extensor / flexor muscles of the thumb 39, and another myoelectric potential sensor 21 whose detection position DP is set near the extensor / flexor muscles of the index finger 40. Note that there may be cases where the myoelectric potentials of the extensor / flexor muscles of the thumb 39 and the index finger 40 are measured using one myoelectric potential sensor 21.

[0171] The presentation operation can be controlled so as to suppress the influence of vibration on the detection of bioinformation by the important biosensors. That is, it is possible to make the vibration zero or sufficiently small in the vicinity of the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40. This improves the detection accuracy of the myoelectric potential sensor 21. As a result, it is possible to estimate changes in pinch force with high accuracy, and switch operation modes with high accuracy.

[0172] In this way, by using the present biological information detection system 1, it is possible to perform accurate myoelectric potential measurement while providing feedback through vibration, and it is possible to prevent a loss of functionality and operability of the system. Conversely, it is possible to provide feedback to the user 2 regarding recognition of the magnitude of force while continuously and accurately measuring myoelectric potential, thereby improving the accuracy and operability of object manipulation.

[0173] An example of the operation of the biological information detection system 1 according to the present technology will be described in XR application example 1 illustrated in FIGS. 10 and 11 .

[0174] 9 , the XR application is started and enters a standby state for a knob operation. At this time, the user 2 may be informed via the HMD 4, the display 32 of the watch-type device 24, or the like, of the operation mode corresponding to the knob operation in the XR space and that the operation mode changes depending on the strength of the force.

[0175] Furthermore, when the XR application is launched, a virtual object 38 is displayed in the XR space.

[0176] 9, the biosensor 16 measures the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40. In step 204, the hand shape and the strength of the pinch force are recognized (predicted).

[0177] In step 204, it is assumed that a pinch operation by user 2 is detected based on classification by a machine learning model, for example. In this case, in step 205, the XR application enters an application state in which a pinch operation is being performed by user 2.

[0178] During the pinch operation state, the force of the pinch operation (pinch force) performed by the user 2 is detected, and if the strength of the force meets a specific condition, it is determined that the presentation condition is met in step 206. Then, in step 207, the execution of vibration feedback and the content of the feedback are determined.

[0179] In this example, when the force changes across thresholds of 20%, 40%, and 60% of the maximum pinch force set in advance by the user 2, a weak vibration of 50 Hz is applied to the entire circumference of the wrist for 0.5 seconds. Note that the content of the feedback determined in step 207 is determined without taking into consideration the position of the myoelectricity sensor 21. For example, the content of the vibration feedback may be set as shown in FIG. 5A.

[0180] In step 208, the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 is selected as the important biosensor. For example, the important biosensor can be selected based on the gesture recognition result, the state of the application, etc.

[0181] Alternatively, it is also possible to select important biosensors based on indices such as the amplitude and integrated myoelectric potential of the myoelectric potential signals of each of the multiple myoelectric potential sensors 21, or the waveform of the myoelectric potential signal. For example, by selecting the myoelectric potential sensor 21 whose amplitude absolute value of the myoelectric potential signal is the largest when the application is in a pinch operation state, or the myoelectric potential sensor 21 whose integrated myoelectric potential increases significantly the moment the recognition unit 33 detects a pinch operation, it is possible to select the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 as important biosensors.

[0182] In step 209, the vibration feedback presentation operation by the multiple vibrators 22 is controlled so as to suppress the effect of vibration on the detection of bioinformation by the important biosensor. For example, the strength of the vibration is weighted based on the position of the important biosensor on the watch-type device 24 and the position of each vibrator 22 so as to reduce noise in the myoelectric potential signal due to vibration for the important biosensor.

[0183] For example, the positional relationship between the multiple myoelectric potential sensors 21 and the multiple vibrators 22 may be stored in advance. Then, based on the information on the positional relationship, it is possible to perform presentation control such that the strength of vibration is reduced by a predetermined percentage (for example, 20%) as it approaches the myoelectric potential sensor 21 that is the important biosensor, as exemplified in Fig. 5B.

[0184] Alternatively, presentation control is possible such that the strength of vibration of the vibrators 22 near the myoelectric potential sensor 21, which is the important biological sensor, is reduced by a predetermined percentage (for example, 20%). Furthermore, presentation control is also possible such that the vibrators 22 closest to the myoelectric potential sensor 21, which is the important biological sensor, (or the vibrators 22 located within a predetermined distance) do not generate vibration (are not selected as the actual feedback device).

[0185] In step 210 , the vibrator 22 provides vibration feedback based on the feedback content determined in step 209 .

[0186] Noise reduction by signal processing is performed in step 211. For example, a band cut filter of 50 Hz, which is the frequency of the vibration feedback, may be applied to the myoelectric potential signal measured from the vital biosensor while the vibration feedback is being performed.

[0187] (Considerations Regarding Each Step) Further considerations regarding each step shown in Fig. 8 will be described. Regarding the execution of vibration feedback and the determination of the content of the feedback in step 207, the content of the feedback to be determined may be defined in advance or may be dynamically determined depending on the situation based on a specific algorithm.

[0188] Regarding the selection of important biosensors in step 208, in addition to the selection based on the state of the indicator of the myoelectric potential signal, selection may also be made based on the magnitude of other indicators such as RMS (Root Square Mean), integrated myoelectric potential, or frequency domain information.

[0189] It is also possible to register and reference the EMG sensor 21 to be used in advance for the detected gesture or application state, or to use a recognition algorithm such as machine learning to directly select important biosensors from the estimated results of the EMG signal or gesture.

[0190] It is also possible to determine the detection position to be detected for the user 2 based on the recognition result by the recognition unit 33 and application information related to the XR application. For example, in XR application example 1 shown in Figures 10 and 11, the detection position to be detected is the detection position where the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 can be measured. It is possible to select the biosensor 16 whose detection position DP is closest to the determined detection position to be detected as the important biosensor.

[0191] Furthermore, when selecting an important biosensor when the presentation conditions for sensory feedback are satisfied, a combination of a gesture and an important biosensor (which may be a detection position DP) is registered in a database in advance by calibration or the like. Alternatively, a machine learning model capable of outputting a combination of a gesture and an important biosensor (which may be a detection position DP) is constructed. Then, when the presentation conditions are satisfied, the important biosensor (which may be a detection position DP) may be determined by referring to the database or using the output from the machine learning model.

[0192] Regarding the control of the presentation operation in step 209, in order to optimize the method of presenting sensory feedback, information indicating the positional relationship between each biosensor 16 and the feedback device 17 may be stored in advance. Then, in response to the selection of an important biosensor, control of the presentation operation may be executed based on the position information. Note that the information indicating the positional relationship between each biosensor 16 and the feedback device 17 is included in the detected position information of one or more biosensors 16 according to the present technology.

[0193] It is also possible to optimize the sensory feedback (control the presented action) by selecting an important biosensor based on information about the muscles used and the relative posture of the wearable device 3 with respect to the body. The important biosensor may also be selected taking into consideration information about the wearing state of the wearable device 3, such as misalignment of the device, body movement, and the contact, distance, and pressure between the skin and the biosensor 16.

[0194] For example, a myoelectric potential sensor 21 that is registered as being capable of measuring the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 when the watch-type device 24 is worn on the wrist is selected as the important biosensor. In this case, if the sensor is misaligned, a myoelectric potential sensor 21 other than the registered myoelectric potential sensor 21 (for example, the myoelectric potential sensor 21 next to it) may be closer to the detection position where the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 should be measured. In such a case, it becomes possible to select the myoelectric potential sensor 21 that is optimal for measuring the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 as the important biosensor, thereby improving detection accuracy.

[0195] One method for optimizing vibration feedback is to selectively vibrate the vibrators 22 that are far from the important biosensor based on relative position information (detection position information).Another effective method is to weight the amplitude of all vibrators 22 so that the amplitude of the vibrators 22 that are far from the important biosensor is larger.

[0196] Another possible method is to modulate the frequency or waveform around the detection position DP of the important biosensor to one that is less susceptible to noise. Another possible method is to generate vibrations of opposite phase to the overall vibration so that the vibrations around the detection position DP of the important biosensor are smaller. Another possible method is to stop or reduce the vibrations around the detection position DP of the important biosensor for a certain time, period, or pattern.

[0197] Another effective method is to add noise to a frequency domain that is cut by a digital filter or the like that is different from the frequency domain that is being used.

[0198] Furthermore, as optimization of sensory feedback (control of the presented action), the content of the sensory feedback determined in step 207 and the content of the optimization of the sensory feedback may be linked and registered in advance. For example, adjustment values ​​of parameters related to execution feedback and the presented action may be linked and registered with the content of the sensory feedback. When controlling the presented action of the sensory feedback, the optimization of the sensory feedback is performed with reference to the registered information. Such a setting is also possible.

[0199] Furthermore, sensory feedback is presented once with the content of the sensory feedback determined in step 207. Then, optimization of the sensory feedback (control of the presentation operation) may be performed based on the detection result of the biosignal at the timing of presenting the sensory feedback and the recognition result by the recognition unit 33. For example, optimization may be omitted when determining the content of the feedback, and sensory feedback may be performed, and optimization processing such as reducing vibrations at positions of the biosensor 16 where the waveform changed significantly during the sensory feedback may be performed. Such processing is also possible.

[0200] During the presentation of optimized sensory feedback, the content of optimization may be dynamically adjusted based on the detection result of the biosensor 16, the recognition result of the recognition unit 33, or application information related to the XR application. For example, parameters related to the execution feedback device or the presented action may be dynamically adjusted during the presentation of sensory feedback.

[0201] Furthermore, when multiple types of biosensors 16 are installed, the detection results of these may be comprehensively interpreted to optimize the feedback. For example, when a myoelectric potential sensor, an IMU, a temperature sensor, etc. are installed as the biosensors 16, the sensory feedback may be optimized based on the detection information of each of these sensors.

[0202] Furthermore, when multiple types of feedback devices 17 are installed, optimization of the sensory feedback may include optimization for using the multiple feedback devices 17 appropriately, such as switching between them or using them simultaneously while weighting them.

[0203] 5, the biosensor 16 and the feedback device 17 may be integrally configured. However, the biosensor 16 and the feedback device 17 may be distributed and disposed in separate devices, for example.

[0204] Regarding noise reduction by signal processing in step 211, it is possible to employ, as signal processing, processing such as adjusting sensor gain, applying a noise removal filter such as a band cut filter, extracting measurement results from sections where vibration has stopped, averaging with preferential weighting, switching machine learning models, optimizing parameters, etc. For example, by executing signal processing according to optimization of sensory feedback, it is possible to achieve effective noise reduction.

[0205] In addition, in the present biological information detection system 1, the detection accuracy is improved by optimizing the sensory feedback. Therefore, it is possible to omit the noise reduction by signal processing in step 211. This is advantageous for simplifying the process, shortening the processing time, reducing costs, etc.

[0206] (XR Application Example 2) Fig. 12 is a schematic diagram for explaining XR Application Example 2. In Fig. 12, the watch-type device 24 is not shown.

[0207] In the XR application example 2, the user 2 can change the shape of his or her hand to perform different gestures, thereby changing the process executed in the XR space.

[0208] For example, when the hand is relaxed as shown in Fig. 12, nothing is displayed on the screen 42 in the XR space. When a so-called number one gesture 1 in which only the index finger 40 is extended is detected, predetermined content A is displayed on the screen 42. When a so-called peace gesture 2 in which the index finger 40 and middle finger 43 are extended is detected, content B, which is different from content A, is displayed on the screen 42.

[0209] User 2 can switch the content displayed on screen 42 by switching between "relaxed state," "gesture 1," and "gesture 2." Note that "relaxed state" is also included in the gestures.

[0210] In such an XR application, when the biometric information detection system 1 detects any one of the gestures "relaxed state," "gesture 1," and "gesture 2," sensory feedback is provided in the form of short-term vibrations. This allows the user 2 to know whether or not their gesture has been recognized, enabling them to experience a highly usable XR application.

[0211] On the other hand, if the presentation of vibration feedback causes noise to be mixed into the myoelectric potential measurement by the myoelectric potential sensor 21, this may affect the accuracy of gesture detection and classification, possibly resulting in erroneous recognition.

[0212] In this embodiment, the biometric information detection system 1 can appropriately select important biometric sensors for the detection of "relaxed state," "gesture 1," and "gesture 2" by the recognition unit 33, and can control the presentation operation so that the impact of vibration on the detection of biometric information by the important biometric sensors is suppressed.

[0213] For example, when detecting a transition from "gesture 1" to "relaxed state," the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the index finger 40 is selected as the important biological sensor. In Fig. 12, a detection position DP1 for measuring the myoelectric potential of the extensor / flexor muscles of the index finger 40 is schematically illustrated.

[0214] The myoelectric potential sensor 21 that can detect the detection position DP1 is selected as the important biosensor. Vibration feedback is optimized to suppress the influence of the selected important biosensor on the myoelectric potential measurement. This makes it possible to detect the transition from "Gesture 1" to the "Relaxed State" with high accuracy.

[0215] When detecting a transition from "gesture 2" to a "relaxed state," the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the index finger 40 and middle finger 43 is selected as the important biological sensor. In Fig. 12, a detection position DP1 for measuring the myoelectric potential of the extensor / flexor muscles of the index finger 40 and a detection position DP2 for measuring the myoelectric potential of the extensor / flexor muscles of the middle finger 43 are schematically illustrated.

[0216] The myoelectric potential sensor 21 that can detect the detection positions DP1 and DP2 is selected as the important biosensor. Vibration feedback is optimized to suppress the influence of the selected important biosensor on the myoelectric potential measurement. This makes it possible to detect the transition from "Gesture 2" to the "Relaxed State" with high accuracy.

[0217] In order to detect the transition from "Gesture 1" to "Gesture 2," it is necessary to detect not only the myoelectric potential of the extensor / flexor muscles of the index finger 40, but also the myoelectric potential of the extensor / flexor muscles of the middle finger 43. That is, the myoelectric potential sensor 21 that can detect the detection positions DP1 and DP2 is selected as the important biosensor. Vibration feedback is optimized so as to suppress the influence of the selected important biosensor on the measurement of the myoelectric potential. This makes it possible to detect the transition from "Gesture 1" to "Gesture 2" with high accuracy.

[0218] When "Gesture 1" is detected in this way, it is necessary to detect not only the transition to the "relaxed state" but also the transition to "Gesture 2," so the myoelectric potential sensor 21 that can detect detection positions DP1 and DP2 is selected as the important biological sensor.

[0219] That is, when detecting multiple types of gestures and gesture transitions, important biosensors are selected taking into consideration the detection of each gesture. In other words, the myoelectricity sensors 21 required for detecting other gestures are also included in the optimization of vibration feedback. This enables highly accurate detection of bioinformation, enabling users to experience highly usable XR applications.

[0220] (XR Application Example 3) Fig. 13 is a schematic diagram for explaining XR Application Example 3. In Fig. 13, the watch-type device 24 is omitted from the illustration.

[0221] In the XR application example 3, similar to the XR application example 1, the user 2 can perform a “pinch” gesture to grab and move the virtual object 38 in the XR space.

[0222] In addition, in XR application example 3, when the virtual object 38 reaches a specific position in the XR space and assumes a specific posture, vibration feedback is provided to notify the user 2 that the movement of the virtual object 38 has been completed. This allows the user 2 to know whether the operation of grabbing and moving the virtual object 38 has been successfully completed, enabling the user 2 to experience an XR application with high usability.

[0223] On the other hand, if the presentation of vibration feedback causes noise to be mixed into the myoelectric potential measurement by the myoelectric potential sensor 21, this may affect the accuracy of detecting and classifying pinch gestures, potentially resulting in erroneous operations.

[0224] In this embodiment, the biometric information detection system 1 makes it possible to select the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40 as the important biometric sensor, similar to XR application example 1.

[0225] It is possible to optimize vibration feedback (control of the presented motion), such as by reducing vibration to zero or making the vibration sufficiently small, in the vicinity of the myoelectric potential sensor 21 that measures the myoelectric potential of the extensor / flexor muscles of the thumb 39 and index finger 40. This makes it possible to improve the detection accuracy of the myoelectric potential sensor 21.

[0226] In this application example 3, when the user 2 moves the virtual object 38 using a grab gesture, the posture of the hand may change depending on whether the movement is over a wide range or a long distance.

[0227] For example, even with the same grasping gesture, the posture may be such that the back of the hand faces vertically upward and the palm faces downward as shown in Fig. 13A . Also, as shown in Fig. 12B , the posture may be such that the extension direction of the thumb 39 is in the up-down direction (vertical direction). Furthermore, although not shown, the posture may be such that the palm faces vertically upward and the back of the hand faces downward, which is upside down from the posture shown in Fig. 13A .

[0228] 13, even with the same grasping gesture, the muscles used and their activity levels may change if the posture is different. For example, it is considered that the activity levels of the extensor / flexor muscles of the thumb 39 and index finger 40 change due to the change in the direction of gravity depending on whether the back of the hand is facing vertically upward or downward during the pinch gesture.

[0229] In this biometric information detection system 1, it is possible to control the presentation of sensory feedback based on posture information of the user 2 wearing the watch-type device 24 and information regarding the state in which the watch-type device 24 is worn by the user 2.

[0230] That is, in this Application Example 3, it is possible to optimize sensory feedback by combining hand posture information. By using the detection results of inertial sensors, optical sensors, etc., and machine learning models, it is possible to obtain posture information of the hand and the watch-type device 24 (biometric sensor 16 and feedback device 17).

[0231] Based on the acquired posture information, it is possible to select important biosensors with high accuracy. It is also possible to select a real feedback device and adjust parameters related to the presented motion with high accuracy. For example, even if the posture of the hand or the wearable device 3 changes significantly, it is possible to perform high-accuracy optimization of sensory feedback. As a result, it is possible to improve the accuracy of detection of bioinformation by the biosensor 16, enabling users to experience highly usable XR applications.

[0232] As described above, in the bioinformation detection system 1 according to this embodiment, the operation of providing sensory feedback by the feedback device 17 is controlled based on the detection position information of the biosensor 16. This makes it possible to suppress the influence of the presentation of sensory feedback on the detection of bioinformation, thereby improving the detection accuracy of the biosensor 16.

[0233] In a wearable device 3 equipped with multiple biosensors 16 such as myoelectric potential sensors, when sensory feedback such as vibration is performed that affects the contact surface between the skin and the electrodes, noise may be introduced into the measured signal, reducing the measurement accuracy of the signal.

[0234] By applying this technology, it is possible to select the feedback device 17 according to the sensing position so that the sensory feedback does not significantly affect the position, and to optimize the feedback parameters. As a result, it is possible to achieve both accurate biosignal measurement and sensory feedback, and to reduce losses in the operability and experience of the wearable device 3.

[0235] For example, there are limitations to simply removing noise through signal processing while measuring biosignals, as it is not possible to eliminate physical factors. Also, preventing noise contamination by simply changing the timing of biosignal measurement and sensory feedback may affect measurement accuracy and the operability and experience of the system due to the constraints imposed on the timing of biosignal measurement and feedback.

[0236] Furthermore, the wearable device 3 equipped with multiple biosensors 16 and multiple feedback devices 17, as illustrated in Figure 2, etc., is a system that has only recently emerged, and there has not been sufficient discussion about how to utilize it.

[0237] In view of these problems and current circumstances, the present technology is a novel and highly effective technology that can simultaneously detect biological information using the biological sensor 16 and provide sensory feedback using the feedback device 17.

[0238] By applying this technology, it is possible to detect various types of biometric information and provide various types of sensory feedback, and it is also possible to physically reduce noise contamination of biometric signals.

[0239] In XR application examples 1 to 3, the present technology is applied to a watch-type device 24 that can provide vibration feedback in response to hand movement and the magnitude of force. Of course, the present technology is not limited to wearable devices 3 used for such purposes, and can be applied to various devices and systems, such as fitness devices worn on the body that perform motion analysis, and accessibility devices that substitute input through the activity of specific muscles.

[0240] Other Embodiments The present technology is not limited to the above-described embodiments, and various other embodiments can be realized.

[0241] FIG. 14 is a schematic diagram showing another example of the configuration of one or more biosensors 16 (myogenic potential sensors 21) and one or more feedback devices 17 (vibrators 22).

[0242] 14A, one vibrator 22a is mounted for six myoelectric potential sensors 21a to 21f. In this case, the vibration intensity of the vibrator 22a is adjusted based on, for example, the distance between the presentation position PPa of the vibrator 22a and the detection position DP of the myoelectric potential sensor 21 selected as the important biological sensor. For example, the vibration intensity is reduced as the detection position DP approaches. Furthermore, if the distance from the detection position DP is greater than a predetermined threshold, vibration feedback is not optimized and vibration is applied at normal vibration intensity. Such settings are also possible.

[0243] 14A, one myoelectric potential sensor 21a and six vibrators 22a to 22f are mounted. In this case, for example, optimization of vibration feedback is performed only during the detection operation of the myoelectric potential sensor 21a. Furthermore, during the detection operation of the myoelectric potential sensor 21a, the vibration intensity of the vibrators 22a to 22f is adjusted based on the distance between the detection position DP1 of the myoelectric potential sensor 21a and the presentation position PP of the vibrator 22, etc.

[0244] In this way, the present technology can be applied even when there is a single biosensor 16 (myogenic potential sensor 21) or a single feedback device 17 (vibrator 22).

[0245] It is also possible to configure a biological information detection system according to the present technology using only the controller 15 shown in Fig. 2. In this case, the biological information detection system has the following configuration.

[0246] A biometric information detection system including an operation control unit that controls the sensory feedback presentation operation of one or more feedback devices that present sensory feedback to a user, and the operation control unit controls the presentation operation based on detection position information relating to the detection position of biometric information of one or more biometric sensors that detect biometric information of the user.

[0247] The biological information detection method according to the present technology described above can be described as follows.

[0248] A biometric information detection method comprising: detecting biometric information of a user (2) using one or more biometric sensors (16); presenting sensory feedback to the user (2) using one or more feedback devices (17); and controlling, using a computer system (controller (15)), the operation of presenting the sensory feedback of the one or more feedback devices (17) based on detection position information relating to the detection position of the biometric information of the one or more biometric sensors (16).

[0249] Furthermore, the program for executing the biometric information detection method of the present technology can be described as a program that causes a computer system (biometric information detection system 1) including one or more biometric sensors 16 and one or more feedback devices 17 to execute the above-mentioned biometric information detection method.

[0250] 15 is a block diagram showing an example of a hardware configuration of a computer 60 that can be used to construct a biological information detection system according to the present technology. For example, the computer 60 shown in FIG. 15 may be communicably connected to one or more biological sensors 16 and one or more feedback devices 17.

[0251] The computer 60 includes a CPU 61, a ROM 62, a RAM 63, an input / output interface 65, and a bus 64 interconnecting these components. The input / output interface 65 is connected to a display unit 66, an input unit 67, a storage unit 68, a communication unit 69, a drive unit 70, and other components. The display unit 66 is a display device using, for example, an LCD or EL display. The input unit 67 is a keyboard, a pointing device, a touch panel, or other operating device. If the input unit 67 includes a touch panel, the touch panel may be integrated with the display unit 66. The storage unit 68 is a non-volatile storage device such as a HDD, flash memory, or other solid-state memory. The drive unit 70 is a device capable of driving a removable storage medium 71 such as an optical storage medium or magnetic recording tape. The communication unit 69 is a modem, router, or other communication device connectable to a LAN, WAN, or the like for communicating with other devices. The communication unit 69 may communicate via either a wired or wireless connection. The communication unit 69 is often used separately from the computer 60. Information processing by the computer 60 having the above-described hardware configuration is realized by cooperation between software stored in the storage unit 68 or the ROM 62, etc. and the hardware resources of the computer 60. Specifically, the information processing method according to the present technology is realized by loading a program constituting the software stored in the ROM 62, etc., into the RAM 63 and executing it. The program is installed in the computer 60 via, for example, the recording medium 71. Alternatively, the program may be installed in the computer 60 via a global network, etc. Alternatively, any computer-readable, non-transitory storage medium may be used.

[0252] The biometric information detection method, sensory feedback presentation method, and program according to the present technology may be executed by cooperation between multiple computers connected to each other via a network or the like, thereby constructing a biometric information detection system according to the present technology. That is, the biometric information detection method, sensory feedback presentation method, and program according to the present technology may be executed not only in a computer system composed of a single computer, but also in a computer system in which multiple computers operate in conjunction with each other. In this disclosure, a "system" refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are contained in the same housing. Therefore, both multiple devices housed in separate housings and connected via a network and a single device housed in a single housing with multiple modules are considered systems.

[0253] The execution of the biometric information detection method, sensory feedback presentation method, and program according to the present technology by a computer system includes both cases where, for example, control of a biometric sensor, generation of biometric information, control of the feedback device 17, control of the sensory feedback presentation operation, selection of important biometric sensors, adjustment of parameters related to the presentation operation, and control of an application are performed by a single computer, and cases where each process is performed by a different computer. Furthermore, the execution of each process by a specific computer also includes having another computer execute part or all of the process and obtaining the results. In other words, the biometric information detection method, sensory feedback presentation method, and program according to the present technology can also be applied to a cloud computing configuration in which a single function is shared and processed collaboratively by multiple devices via a network.

[0254] The configurations of the biometric information detection system, wearable device, biometric sensor, feedback device, and controller, and the processing flows of the biometric sensor control, biometric information generation, feedback device 17 control, sensory feedback presentation operation control, important biometric sensor selection, parameter adjustment for presentation operation, and application control described with reference to the drawings are merely one embodiment and can be modified as desired without departing from the spirit of the present technology. In other words, any other configurations, algorithms, etc. for implementing the present technology may be adopted.

[0255] In this disclosure, terms such as "about," "approximately," "almost," and "roughly" may be used as appropriate to facilitate understanding of the description. However, there is no clear difference between using and not using terms such as "about," "approximately," "almost," and "approximately." In other words, in this disclosure, concepts that define shape, size, positional relationship, state, etc., such as "center," "middle," "uniform," and "equal," are concepts that include "substantially center," "substantially central," "substantially uniform," and "substantially equal." For example, states that fall within a predetermined range (e.g., a range of ±10%) based on "completely centered," "completely central," "completely uniform," and "completely equal" are also included. Therefore, even if terms such as "approximately," "almost," and "approximately" are not used, concepts expressed by adding "approximately," "almost," and "approximately" may be included. Conversely, states expressed by adding terms such as "approximately," "almost," and "approximately" do not necessarily exclude perfect states.

[0256] In the present disclosure, expressions using "than", such as "greater than A" and "smaller than A", are expressions that comprehensively include both concepts that include the case where it is equivalent to A and concepts that do not include the case where it is equivalent to A. For example, "greater than A" is not limited to cases that do not include equivalent to A, but also includes "A or greater". Furthermore, "smaller than A" is not limited to "less than A" but also includes "A or less". When implementing the present technology, specific settings and the like can be appropriately adopted from the concepts included in "greater than A" and "smaller than A" so that the effects described above can be achieved.

[0257] It is also possible to combine at least two of the features of the present technology described above. That is, the various features described in each embodiment may be arbitrarily combined without distinguishing between the embodiments. Furthermore, the various effects described above are merely examples and are not intended to be limiting, and other effects may also be achieved.

[0258] The present technology may also be configured as follows. (1) A biometric information detection system comprising: one or more biometric sensors that detect biometric information of a user; one or more feedback devices that present sensory feedback to the user; and an operation control unit that controls an operation of presenting the sensory feedback of the one or more feedback devices based on detection position information related to detection positions of the biometric information of the one or more biometric sensors. (2) The biometric information detection system according to (1), wherein the operation control unit controls the presentation operation so as to suppress an influence of the presentation of the sensory feedback on detection of the biometric information by the one or more biometric sensors. (3) The biometric information detection system according to (1) or (2), wherein the operation control unit selects an important biometric sensor from the one or more biometric sensors that has a relatively high importance for detection of the biometric information based on detection-related information related to detection of the biometric information by the one or more biometric sensors, including the detection position information, and controls the presentation operation so as to suppress an influence of the presentation of the sensory feedback on detection of the biometric information by the important biometric sensor. (4) The biological information detection system according to (3), wherein the operation control unit determines, based on the detection-related information, whether or not an operating biological sensor that is currently detecting the biological information is present among the one or more biological sensors, and controls the presentation operation if an operating biological sensor is present. (5) The biological information detection system according to (4), wherein the operation control unit determines, based on the detection-related information, an operating biological sensor among the one or more biological sensors that is currently detecting the biological information, and selects the important biological sensor from the operating biological sensors. (6) The biological information detection system according to any one of (3) to (5), wherein the detection-related information includes at least one of detection information detected by the one or more biological sensors, generated information generated based on the detection information, and application information related to an application that uses at least one of the detection information and the generated information.(7) The bioinformation detection system according to any one of (3) to (6), further comprising a wearing unit that holds the one or more biosensors and the one or more feedback devices and is configured to be wearable by the user, wherein the detection-related information includes at least one of posture information of the user wearing the wearing unit and information regarding a wearing state of the wearing unit relative to the user. (8) The bioinformation detection system according to any one of (1) to (7), wherein the operation control unit controls the presentation operation by selecting an execution feedback device that performs presentation of the sensory feedback and adjusting parameters related to the presentation operation. (9) The bioinformation detection system according to (8), wherein the operation control unit adjusts intensity of the sensory feedback as adjustment of parameters related to the presentation operation. (10) The biological information detection system according to any one of (1) to (9), wherein the detection position information includes information regarding a positional relationship between the detection position of the one or more biological sensors and a presentation position of the sensory feedback of the one or more feedback devices. (11) The biological information detection system according to (10), wherein the operation control unit determines a positional relationship between the detection position of the important biological sensor and the presentation position of the one or more feedback devices based on the detection position information, and controls the presentation operation based on the determined positional relationship. (12) The biological information detection system according to (11), wherein the operation control unit determines a distance between the detection position of the important biological sensor and the presentation position of the one or more feedback devices based on the detection position information, and adjusts intensity of the sensory feedback based on the determined distance. (13) The bioinformation detection system according to (12), wherein the one or more feedback devices apply vibration to the presentation position, and the operation control unit adjusts the intensity of the vibration based on the magnitude of the distance between the detection position of the important biosensor and the presentation position of the one or more feedback devices.(14) The biological information detection system according to any one of (1) to (13), wherein the one or more biological sensors are a plurality of biological sensors, and the one or more feedback devices are a plurality of feedback devices. (15) The biological information detection system according to any one of (1) to (14), wherein the one or more biological sensors include at least one of a myoelectricity sensor, a muscle sound sensor, a temperature sensor, a heart rate sensor, a sweat sensor, a velocity sensor, an acceleration sensor, an angular velocity sensor, an IMU (Inertial Measurement Unit) sensor, an inertial sensor, an optical sensor, an image sensor, a strain sensor, and a capacitance sensor. (16) The biological information detection system according to any one of (1) to (15), wherein the one or more biological sensors are myoelectricity sensors. (17) The biological information detection system according to any one of (1) to (16), wherein the one or more feedback devices include at least one of a tactile presentation device, an electrical stimulation device, an electromagnetic wave generation device, and a temperature presentation device. (18) A biological information detection system including an operation control unit that controls a presentation operation of sensory feedback of one or more feedback devices that present sensory feedback to a user, the operation control unit controlling the presentation operation based on detection position information regarding a detection position of the biological information of one or more biosensors that detect the user's biological information. (19) A biological information detection method comprising: detecting a user's biological information with one or more biosensors; presenting sensory feedback to the user with one or more feedback devices; and controlling the presentation operation of the sensory feedback of the one or more feedback devices based on detection position information regarding a detection position of the biological information of the one or more biosensors by a computer system.(20) A program for causing a computer system including one or more biosensors and one or more feedback devices to execute a biometric information detection method, wherein the biometric information detection method detects a user's biometric information using the one or more biosensors, presents sensory feedback to the user using the one or more feedback devices, and controls the presentation operation of the sensory feedback of the one or more feedback devices using the computer system based on detection position information regarding the detection position of the biometric information by the one or more biosensors.

[0259] DP: detection position PP: sensory feedback presentation position 1: bioinformation detection system 2: user 3: wearable device 8: wristband-type device 16: biosensor 17: feedback device 18: operation control unit 20, 30: wearing band 21: myoelectric potential sensor 22: vibrator 24: watch-type device 25: main body 38: virtual object 39: thumb 40: index finger 43: middle finger 60: computer

Claims

1. A biometric information detection system comprising: one or more biometric sensors that detect a user's biometric information; one or more feedback devices that present sensory feedback to the user; and an operation control unit that controls the presentation of the sensory feedback by the one or more feedback devices based on detection position information regarding the detection position of the biometric information by the one or more biometric sensors.

2. A bioinformation detection system as described in claim 1, wherein the operation control unit controls the presentation operation so as to suppress the influence of the presentation of the sensory feedback on the detection of the bioinformation by the one or more biosensors.

3. A bioinformation detection system as described in claim 1, wherein the operation control unit selects an important biosensor from the one or more biosensors that has a relatively high importance for the detection of the bioinformation based on detection-related information regarding the detection of the bioinformation by the one or more biosensors, including the detection position information, and controls the presentation operation so as to suppress the influence of the presentation of the sensory feedback on the detection of the bioinformation by the important biosensor.

4. A bioinformation detection system as described in claim 3, wherein the operation control unit determines whether or not there is an operating biosensor that is currently detecting the bioinformation among the one or more biosensors based on the detection-related information, and controls the presentation operation if there is an operating biosensor.

5. A biological information detection system as described in claim 4, wherein the operation control unit determines which of the one or more biological sensors is currently operating to detect the biological information based on the detection-related information, and selects the important biological sensor from the active biological sensors.

6. A bioinformation detection system according to claim 3, wherein the detection-related information includes at least one of detection information detected by the one or more biosensors, generated information generated based on the detection information, and application information relating to an application that uses at least one of the detection information and the generated information.

7. A bioinformation detection system as claimed in claim 3, further comprising a mounting unit configured to hold said one or more biosensors and said one or more feedback devices and be mountable on said user, wherein said detection-related information includes at least one of posture information of said user wearing said mounting unit and information relating to the mounting state of said mounting unit relative to said user.

8. A bioinformation detection system according to claim 1, wherein the operation control unit controls the presented operation by at least one of selecting an execution feedback device that performs the presentation of the sensory feedback and adjusting parameters related to the presented operation.

9. A biological information detection system according to claim 8, wherein the operation control unit adjusts the intensity of the sensory feedback as an adjustment of a parameter related to the presented operation.

10. A bioinformation detection system according to claim 1, wherein the detection position information includes information regarding the positional relationship between the detection position of the one or more biosensors and the presentation position of the sensory feedback of the one or more feedback devices.

11. A bioinformation detection system as described in claim 10, wherein the operation control unit determines the positional relationship between the detection position of the important biosensor and the presentation position of the one or more feedback devices based on the detection position information, and controls the presentation operation based on the determined positional relationship.

12. A bioinformation detection system as described in claim 11, wherein the operation control unit determines the magnitude of the distance between the detection position of the important biosensor and the presentation position of the one or more feedback devices based on the detection position information, and adjusts the intensity of the sensory feedback based on the magnitude of the determined distance.

13. A bioinformation detection system as described in claim 12, wherein the one or more feedback devices apply vibration to the presentation position, and the operation control unit adjusts the intensity of the vibration based on the magnitude of the distance between the detection position of the important biosensor and the presentation position of the one or more feedback devices.

14. A biological information detection system according to claim 1, wherein the one or more biological sensors are a plurality of biological sensors, and the one or more feedback devices are a plurality of feedback devices.

15. A biological information detection system according to claim 1, wherein the one or more biological sensors include at least one of a myoelectricity sensor, a muscle sound sensor, a temperature sensor, a heart rate sensor, a sweat sensor, a speed sensor, an acceleration sensor, an angular velocity sensor, an IMU (Inertial Measurement Unit) sensor, an inertial sensor, an optical sensor, an image sensor, a strain sensor, and a capacitance sensor.

16. A biological information detection system according to claim 1, wherein the one or more biological sensors are myoelectric potential sensors.

17. A biological information detection system according to claim 1, wherein the one or more feedback devices include at least one of a tactile sensation presentation device, an electrical stimulation device, an electromagnetic wave generation device, and a temperature sensation presentation device.

18. A biometric information detection system comprising an operation control unit that controls the presentation operation of one or more feedback devices that present sensory feedback to a user, wherein the operation control unit controls the presentation operation based on detection position information regarding the detection position of the biometric information of one or more biometric sensors that detect the biometric information of the user.

19. A biometric information detection method comprising: detecting a user's biometric information using one or more biometric sensors; presenting sensory feedback to the user using one or more feedback devices; and controlling the operation of the one or more feedback devices to present the sensory feedback based on detection position information regarding the detection position of the biometric information by the one or more biometric sensors using a computer system.

20. A program for causing a computer system including one or more biosensors and one or more feedback devices to execute a biometric information detection method, wherein the biometric information detection method detects a user's biometric information using the one or more biosensors, presents sensory feedback to the user using the one or more feedback devices, and controls the presentation of the sensory feedback by the computer system based on detection position information regarding the detection position of the biometric information by the one or more biosensors.

Images