Information conversion system, information processing device, information processing method, and program

Abstract

To provide an information conversion system capable of converting biological information into text information or speech information and converting sound information into text information, and capable of outputting the text information or the speech information that is converted with high accuracy.SOLUTION: An information conversion system includes: a biological information detection unit 102 for detecting biological information from one or more parts of a user; a sound information detection unit 103 for detecting sound information; a determination unit 110 for determining whether or not characteristics of the sound information satisfy a prescribed condition; and a conversion unit 106 for outputting text information or speech information converted from the biological information detected by the biological information detection unit 102 using a first conversion technique if the characteristics of the sound information detected by the sound information detection unit 103 do not satisfy the prescribed condition, and outputting text information converted from the sound information detected by the sound information detection unit 103 using a second conversion technique if the characteristics of the sound information satisfy the prescribed condition.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information conversion system, an information processing apparatus, an information processing method, and a program for converting biological information into character information.

Background Art

[0002] In recent years, using the voice information of a user, recognizing the content of speech has been performed. Obtaining biological information and voice information indicating the movement of the user's mouth, and outputting a recognition result of the content of speech based on the voice information has been performed. (For example, Patent Document 1)

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, it only outputs a recognition result of the content of speech based on speech information including the voice information of the user, and it is not a configuration for converting character information from only biological information, but a configuration for converting character information from both biological information and voice information.

[0005] An object of the present invention is to convert biological information into character information or voice information, convert sound information into character information, and output the character information or voice information converted with high accuracy.

[0006] However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. It is also possible to position the problems corresponding to the respective effects of the respective configurations shown in the embodiments described later as other problems.

Means for Solving the Problems

[0007] To achieve the objectives of the present invention, the information conversion system of the present invention includes a biometric information detection unit that detects biometric information from one or more locations on a user, a sound information detection unit that detects sound information, a determination unit that determines whether the characteristics of the sound information satisfy predetermined conditions, and, if the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, a first conversion method is used to output character information or voice information converted from the biometric information detected by the biometric information detection unit, and if the characteristics of the sound information detected by the sound information detection unit satisfy predetermined conditions, the second three Using the conversion method, the sound information Information including the biological information detected by the biological information detection unit Text information converted from or audio information It includes a conversion unit that outputs [something].

[0008] Furthermore, the information processing apparatus of the present invention includes a receiving unit that receives biological information detected by a biological information detection unit and sound information detected by a sound information detection unit; a determination unit that determines whether the characteristics of the sound information satisfy predetermined conditions; and a conversion unit that, if the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, outputs character information or voice information converted from the biological information detected by the biological information detection unit using a first conversion method, and if the characteristics of the sound information satisfy predetermined conditions, outputs character information converted from the sound information detected by the sound information detection unit using a second conversion method.

[0009] Furthermore, the information processing method of the present invention includes the steps of: receiving biological information detected by a biological information detection unit and sound information detected by a sound information detection unit; determining whether the characteristics of the sound information satisfy predetermined conditions; and, if the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, outputting character information or voice information converted from the biological information detected by the biological information detection unit using a first conversion method; and, if the characteristics of the sound information satisfy predetermined conditions, outputting character information converted from the sound information detected by the sound information detection unit using a second conversion method. [Effects of the Invention]

[0010] According to the present invention, it is possible to convert biological information into character information or voice information, convert sound information into character information, and output the character information or voice information that has been converted with high precision.

Brief Description of the Drawings

[0011] [Figure 1] A diagram showing the configuration of the information conversion system of the present invention. [Figure 2] A flowchart showing the operation of the information conversion system of the present invention. [Figure 3] A diagram showing the configuration of the information conversion system of Example 1 of the present invention. [Figure 4] A schematic diagram of the detection device of Example 1 of the present invention. [Figure 5] A flowchart showing the operation related to Example 1 of the present invention. [Figure 6] A diagram showing the configuration of the information conversion system of Example 2 of the present invention. [Figure 7] A schematic diagram of the detection device of Example 2 of the present invention. [Figure 8] A flowchart showing the operation related to Example 2 of the present invention. [Figure 9] A schematic diagram of the detection device of Example 3 of the present invention. [Figure 10] A diagram showing the configuration of the information conversion system of Example 4 of the present invention. [Figure 11] A schematic diagram of the detection device of Example 4 of the present invention. [Figure 12] A flowchart showing the operation related to Example 4 of the present invention. [Figure 13] A diagram showing the configuration of the information conversion system of Example 5 of the present invention. [Figure 14] A schematic diagram of the detection device of Example 5 of the present invention. [Figure 15] A flowchart showing the operation related to Example 5 of the present invention. [Figure 16] A schematic diagram of the detection device of Example 6 of the present invention.

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in detail. The outline of the information conversion system of the present invention is shown in FIG. 1.

[0013] The information conversion system is mainly composed of a detection device 100 and an information processing device 101. The detection device 100 includes a biological information detection unit 102 that detects biological information from one or more parts of a user, a sound information detection unit 103 that detects sound information, and a transmission unit 104 that transmits the biological information detected by the biological information detection unit 102 and the sound information detected by the sound information detection unit 103 to the information processing device 101.

[0014] In FIG. 1, a form in which the detection device 100 is one device is shown, but it may also be a plurality of devices, and the biological information detection unit 102 and the sound information detection unit 103 may be respectively composed of each detection device. Note that the detection device 100 can also be rephrased as an acquisition unit that acquires information such as biological information and sound information.

[0015] The biological information detection unit 102 is composed of sensors that detect biological information related to the movement of the user's muscles, the movement of the skin and tongue, etc. The biological information detection unit 102 is at least one of an electromyogram sensor, an acceleration sensor, an ultrasonic sensor, a tactile sensor, an optical sensor, a pressure sensor, etc. Note that the biological information detection unit 102 may be other than the above sensors as long as it is a sensor that detects biological information. Since the biological information detection unit 102 detects biological information related to the movement of the user's mouth and tongue, it is desirable to install it at sites such as the neck, lower jaw, around the mouth, and temples. As long as it can detect biological information related to the movement of the mouth and tongue, the biological information detection unit 102 may be installed at other sites than the above sites.

[0016] The sound information detection unit 103 consists of sensors (microphones) that detect external ambient sounds, the user's speech, and sounds other than speech emitted by the user, such as coughing, chewing, and swallowing sounds. The sound information detection unit 103 is at least one of the following: a condenser microphone, a bone conduction microphone, or a skin conduction microphone. The sound information detection unit 103 may be any sensor other than the microphones mentioned above that can detect sounds emitted by the user. Furthermore, in the sound information detection unit 103, the sensor (microphone) that detects external ambient sounds and the sensor (microphone) that detects the user's speech, coughing, chewing, and swallowing sounds may be installed separately.

[0017] The sound information detection unit 103 may consist of at least two detection units. The sound information detection unit 103 may include, for example, a sound information detection unit that detects sound information emitted from within the body and a sound information detection unit that detects external sounds. The sound information detection unit 103 that detects sound information emitted from within the body may be, for example, a bone conduction microphone or a skin conduction microphone.

[0018] The information processing device 101 includes a receiving unit 105 that receives biometric information and sound information transmitted from the detection device 100, a conversion unit 106 that converts the biometric information and sound information received by the receiving unit 105 into text information, voice information, or voice information, and a determination unit 110 that analyzes the sound information received by the receiving unit 105 and determines whether or not the characteristics of the sound information satisfy predetermined conditions.

[0019] The information processing device 101 may include, but is not limited to, a smartphone, a personal computer (PC), or a tablet PC. If the information processing device 101 is, for example, a personal computer, the character information converted by the conversion unit 106 is transmitted to a display unit 107, such as a display. The display unit 107 displays the character information. The information processing device 101 may also have a display control unit (not shown) that controls the display format on the display unit 107.

[0020] The conversion unit 106 can further convert the converted character information into audio information. Alternatively, the conversion unit 106 can directly convert biometric information into audio information. The audio information converted by the conversion unit 106 is transmitted to the sound information output unit 108. The sound information output unit 108 is a speaker and can play back the audio information.

[0021] The determination unit 110 analyzes the characteristics of the sound information received by the receiving unit 105 and determines whether the characteristics of the sound information satisfy predetermined conditions. For example, the determination unit 110 determines whether the volume of the sound information received by the receiving unit 105 is greater than a predetermined threshold. The determination unit 110 also determines from the characteristics of the sound information received by the receiving unit 105 whether the sound information contains noise (sound other than speech). Furthermore, the determination unit 110 can also determine from the characteristics of the sound information received by the receiving unit 105 whether or not it is a speech period in which the user is speaking (whether or not the user's voice is included). The user can arbitrarily select the above determination conditions of the determination unit 110. Noise is, for example, information unrelated to speech that is emitted from within the user's body.

[0022] Furthermore, the user can arbitrarily combine and select the above determination conditions of the determination unit 110. For example, the determination unit 110 can simultaneously determine whether the volume of the sound information received by the receiving unit 105 is greater than a predetermined threshold and whether the sound information contains noise. In addition, the determination unit 110 can simultaneously determine, for example, from the characteristics of the sound information received by the receiving unit 105 whether or not it is a period in which the user is speaking (whether or not the user's voice is included) and whether or not the sound information contains noise.

[0023] If the determination unit 110 determines that the characteristics of the sound information meet predetermined conditions, there is a high probability that the voice spoken by the user has been properly detected, so the conversion unit 106 converts the sound information detected by the sound information detection unit 103 into text information.

[0024] For example, if the determination unit 110 determines that the volume of the sound information is greater than a predetermined threshold, the conversion unit 106 converts the sound information detected by the sound information detection unit 103 into text information.

[0025] Furthermore, if the volume of the user's speech in the sound information received by the receiving unit 105 is greater than a predetermined threshold and the sound information contains little or no noise, the conversion unit 106 converts the sound information detected by the sound information detection unit 103 into text information. If the determination unit 110 determines that the signal-to-noise ratio of the sound related to the user's speech in the sound information is greater than a threshold, the conversion unit 106 may convert the sound information detected by the sound information detection unit 103 into text information. On the other hand, if the determination unit 110 determines that the characteristics of the sound information do not meet the predetermined conditions, there is a high possibility that the voice spoken by the user has not been properly detected, so the conversion unit 106 converts the biometric information detected by the biometric information detection unit 102 into text information or voice information.

[0026] If the determination unit 110 determines that the volume of the sound information is below a predetermined threshold, the conversion unit 106 converts the biometric information detected by the biometric information detection unit 102 into text information or voice information. Also, if the sound information received by the receiving unit 105 contains noise from external sounds, or if it is not a period when the user is speaking, the conversion unit 106 converts the biometric information detected by the biometric information detection unit 102 into text information or voice information. Note that if the sound information received by the receiving unit 105 contains noise emitted from within the body, such as coughing, chewing sounds, or swallowing sounds, the biometric information detected by the biometric information detection unit 102 is also likely to contain noise. Therefore, the conversion unit 106 can choose not to convert the biometric information detected by the biometric information detection unit 102 into text information or voice information. It is also possible to use two types of microphones to distinguish between external sounds and noise emitted from within the body. For example, a normal microphone can be used to capture external sounds, and a bone conduction microphone or skin conduction microphone can be used to detect sounds generated from within the body.

[0027] When the conversion unit 106 converts biometric information detected by the biometric information detection unit 102 into text information or audio information, it uses a first conversion method (first conversion algorithm) to convert the biometric information into text information or audio information. When the conversion unit 106 converts audio information detected by the audio information detection unit 103 into text information, it uses a second conversion method (second conversion algorithm) to convert the audio information into text information.

[0028] The transformation algorithms in the first and second transformation methods utilize trained models based on a neural network architecture. The information processing device 100 has a storage unit (not shown) for storing the trained models. The transformation unit 106 has the function of performing inference using the trained models.

[0029] The trained models are those generated using deep learning techniques such as CNNs (Convolutional Neural Networks) and RNNs (Recurrent Neural Networks). In addition to models derived from CNNs and RNNs, other machine learning techniques such as support vector machines, logistic regression, and random forests may also be used, as may rule-based methods.

[0030] The information processing device 101 generates a trained model for use in the first conversion method by, for example, learning to associate biometric information detected by the biometric information detection unit 102 with character information or voice information. The information processing device 101 generates a trained model for use in the second conversion method by learning to associate voice information detected by the sound information detection unit 103 with character information.

[0031] Specifically, the information processing device 101 acquires multiple datasets in which biometric information (e.g., waveform of electromyographic signals) detected by the biometric information detection unit 102 is associated with character information or voice information (e.g., a, i, u, e, o, or their sounds). The biometric information detection unit 102 has, for example, multiple electrodes on the surface that comes into contact with the user's skin (contact surface), and each electrode can measure the user's electromyographic signals.

[0032] In Figure 1, the learning unit 111 is configured in the information processing device 101. The learning unit 111 may also be configured on the cloud. If the learning unit 111 is configured on the cloud, various data are transferred via the data transfer unit (not shown) of the information processing device 101.

[0033] The learning unit 111 uses the correspondence between biometric information and textual or speech information in multiple datasets as training data, and generates a trained model to be used in the first conversion method by learning by associating biometric information with textual or speech information. Using this trained model, which has been learned by associating biometric information with textual or speech information, the conversion unit 106 can perform inference on newly input biometric information and output textual or speech information.

[0034] Similarly, the learning unit 111 acquires multiple datasets in which sound information (e.g., waveforms of sound information) detected by the sound information detection unit 103 is associated with character information (e.g., a, i, u, e, o). The learning unit 111 uses the correspondence between sound information and character information in the multiple datasets as training data, and generates a trained model to be used in the second conversion method by learning by associating the characteristics of sound information with character information. Using this trained model, which has been learned by associating sound information and character information, the conversion unit 106 can perform inference on newly input sound information and output character information.

[0035] While the conversion algorithms for the first and second conversion methods have been described, a third conversion algorithm can also be used to convert information including sound and biometric information into textual or speech information.

[0036] Specifically, the learning unit 111 acquires multiple datasets in which information containing biometric information detected by the biometric information detection unit 102 and sound information detected by the sound information detection unit 103 is associated with character information or speech information. The learning unit 111 uses the correspondence between information containing the characteristics of biometric information and sound information and character information or speech information in the multiple datasets as training data, and generates a trained model to be used in the third conversion method by training it in association with information containing biometric information and sound information and character information or speech information. Using this trained model, which has been trained in association with information containing biometric information and sound information and character information, the conversion unit 106 can perform inference on newly input information containing biometric information and sound information and output character information or speech information.

[0037] The learning unit 111 uses the correspondence between the character information converted by the second conversion method using the sound information detected by the sound information detection unit 103 and the biometric information acquired by the biometric information detection unit 102, which is acquired simultaneously with the sound information detected by the sound information detection unit 103, as training data. By learning by associating the biometric information with character information or speech information, the learning unit 111 can update the trained model used in the first conversion method.

[0038] The information processing device 101 may be configured on the cloud. The transmission unit 104 in the detection device 100 transmits the biometric information detected by the biometric information detection unit 102 and the sound information detected by the sound information detection unit 103 to the cloud. In the cloud, the biometric information and sound information are converted into text information or voice information, and the converted text information is transmitted to the display unit 107. The display unit 107 displays the text information. Alternatively, the converted text information may be further converted into voice information and transmitted to the sound information output unit 108. Alternatively, the biometric information may be directly converted into voice information and transmitted to the sound information output unit 108.

[0039] Communication between the detection device 100 and the information processing device 101 may be wired or wireless. If communication between the detection device 100 and the information processing device 101 is implemented via a wired connection, the transmitting unit 104 of the detection device 100 and the receiving unit 105 of the information processing device 101 are connected by a wired connection such as a USB cable or HDMI®.

[0040] When communication between the detection device 100 and the information processing device 101 is achieved wirelessly, the transmitting unit 104 in the detection device 100 and the receiving unit 105 in the information processing device 101 are connected wirelessly by means of a wireless LAN such as Wi-Fi, or short-range wireless communication such as Bluetooth®.

[0041] If the information processing device 101 is a smartphone or tablet PC, then the display unit 107 is a display. The sound information output unit 108 is a speaker installed in the smartphone or tablet PC, or earphones connected to the smartphone or tablet PC.

[0042] The information conversion system of the present invention comprises a biometric information detection unit 102 that detects biometric information from one or more locations on a user, a sound information detection unit 103 that detects sound information, a determination unit 110 that determines whether the characteristics of the sound information satisfy predetermined conditions, and a conversion unit 106 that, if the characteristics of the sound information detected by the sound information detection unit 103 do not satisfy predetermined conditions, outputs character information or voice information converted from the biometric information detected by the biometric information detection unit 102 using a first conversion method, and if the characteristics of the sound information satisfy predetermined conditions, outputs character information converted from the sound information detected by the sound information detection unit 103 using a second conversion method.

[0043] If the characteristics of the sound information detected by the sound information detection unit 103 do not meet predetermined conditions, the conversion unit 106 can use a first conversion method to convert the biometric information detected by the biometric information detection unit 102 into text information or voice information, and if the characteristics of the sound information meet predetermined conditions, it can use a second conversion method to convert the sound information into text information.

[0044] The conversion unit 106 can switch from the first conversion method to the second conversion method, or from the second conversion method to the first conversion method, according to the characteristics of the sound information detected by the sound information detection unit 103, and output the character information or audio information converted by the switched conversion method.

[0045] The conversion unit 106 can also output character information converted from information including sound information and biometric information using a third conversion method if the characteristics of the sound information detected by the sound information detection unit 103 satisfy predetermined conditions. Depending on the characteristics of the sound information detected by the sound information detection unit 103, the conversion unit 106 can switch from the first conversion method to the third conversion method, or from the third conversion method to the first conversion method, and output character information or audio information converted by the switched conversion method.

[0046] If the determination unit 110 determines that the sound information detected by the sound information detection unit 103 contains noise, the conversion unit 106 can choose not to convert the biometric information into either text information or audio information.

[0047] If the determination unit 110 determines that the volume of the sound information detected by the sound information detection unit 103 is greater than a threshold, the conversion unit 106 converts the sound information into text information. If the determination unit 110 determines that the volume of the sound information detected by the sound information detection unit 103 is below a threshold, the conversion unit 106 can also convert biometric information into text information.

[0048] If the determination unit 110 determines that the sound information detected by the sound information detection unit 103 is a period of user speech, the conversion unit 106 converts the sound information into text information. If the determination unit 110 determines that the sound information detected by the sound information detection unit 103 is a period of non-speech by the user, the conversion unit 106 converts the biometric information into text information or voice information.

[0049] Depending on the characteristics of the sound information detected by the sound information detection unit 103, the conversion unit 106 sets either a first conversion method for converting biometric information into character information or speech information, or a second conversion method for converting sound information into character information, and converts either biometric information or sound information into character information or speech information. The information processing device 101 of the present invention includes a receiving unit 105 that receives biometric information detected by the biometric information detection unit 102 and sound information detected by the sound information detection unit 103, a determination unit 110 that determines whether the characteristics of the sound information satisfy predetermined conditions, and a conversion unit 106 that, if the characteristics of the sound information detected by the sound information detection unit 103 do not satisfy predetermined conditions, outputs character information or speech information converted from the biometric information detected by the biometric information detection unit 102 using the first conversion method, and if the characteristics of the sound information satisfy predetermined conditions, outputs character information converted from the sound information detected by the sound information detection unit 103 using the second conversion method. If the characteristics of the sound information detected by the sound information detection unit 103 satisfy predetermined conditions, the conversion unit 106 can also output character information or voice information converted from information including sound information and biometric information using a third conversion method.

[0050] Furthermore, the information processing device 101 may also include a processing unit that evaluates the biological information detected by the biological information detection unit 102 based on predetermined evaluation criteria and deletes textual or audio information corresponding to the biological information that does not meet the predetermined evaluation criteria.

[0051] Therefore, according to the present invention, it is possible to output highly accurate converted text information or audio information.

[0052] Figure 2 shows a flowchart illustrating the operation of the information conversion system of the present invention.

[0053] The user attaches the detection device 100 to a position where biological information can be detected by the biological information detection unit 102. The user attaches the detection device 100 to the head, neck, or surrounding areas of the head and neck. The biological information detection unit 102 detects biological information including at least one of the following: electromyographic signals, acceleration information, ultrasound information, tactile information, light information, pressure information, etc. When the biological information detection unit 102 detects biological information, it acquires time information (time information). Additional information related to the time information is attached to the biological information. (S100)

[0054] The user places the detection device 100 in a position where sound information can be detected by the sound information detection unit 103. The user places the detection device 100 at any point on the user's neck or head, in a position where it can detect ambient sounds around the user. The sound information detection unit 103 detects sound information at the same time that the biometric information detection unit 102 detects biometric information. When the sound information detection unit 103 detects sound information, it acquires time information (time information). Additional information related to the time information is added to the sound information. (S101)

[0055] The transmitting unit 104 transmits sound information and biometric information to the information processing device 101. Both the biometric information and sound information are associated with time information (time information). The biometric information and sound information are linked by this associated information. Because the biometric information and sound information are linked by the associated time information (time information), the transmitting unit 104 can transmit biometric information and sound information detected at the same time period (time) to the information processing device 101. (S102)

[0056] The sound information and biometric information detected by the detection device 100 are received by the receiving unit 105 of the information processing device 101. The receiving unit 105 transmits the sound information and biometric information to the conversion unit 106. The receiving unit 105 transmits the sound information to the determination unit 110. The determination unit 110 analyzes the characteristics of the sound information received by the receiving unit 105 and determines whether the characteristics of the sound information satisfy predetermined conditions. For example, the determination unit 110 determines whether the sound information includes the user's voice. (S103)

[0057] If the characteristics of the sound information do not meet the predetermined conditions, the process proceeds to S104; if the characteristics of the sound information meet the predetermined conditions, the process proceeds to S105.

[0058] If the determination unit 110 determines that the characteristics of the sound information do not meet predetermined conditions, the conversion unit 106 converts the biometric information into text information or speech information using the first conversion method. Alternatively, the conversion unit 106 may convert the biometric information into speech information, or it may convert it into text information and then perform speech synthesis to convert it back into speech information. The conversion unit 106 outputs the text information converted from the biometric information. (S104)

[0059] Furthermore, when the biometric information detection unit 102 detects biometric information, the determination unit 110 may determine that the sound information detected by the sound information detection unit 103 at that time includes sound information emitted by the user other than speech (for example, subsalivary glands, coughs). In that case, the conversion unit 106 can be set not to convert the biometric information into either text information or voice information.

[0060] If the determination unit 110 determines that the characteristics of the sound information meet predetermined conditions, the conversion unit 106 converts the sound information into character information using a second conversion method. (S105)

[0061] If the sound information detected by the sound information detection unit 103 contains noise, the conversion unit 106 can be set not to convert the sound information into text information using the second conversion algorithm. Furthermore, if the signal-to-noise ratio (S / N) of the sound information detected by the sound information detection unit 103 is poor, the conversion unit 106 can also convert the information containing both sound information and biometric information into text information or speech information using a third conversion algorithm that converts information containing both sound information and biometric information into text information or speech information. Note that if the S / N of the sound information detected by the sound information detection unit 103 is poor, the conversion unit 106 may perform the conversion using only biometric information.

[0062] The display unit 107 displays the character information converted by the conversion unit 106, and the sound information output unit outputs sound information obtained by further converting the character information converted by the conversion unit 106. (S106)

[0063] Furthermore, the character information converted by the conversion unit 106 can be stored in the storage unit of the information processing device 101. The character information can also be transferred via a network and displayed on an external terminal.

[0064] Furthermore, by converting text or audio information into audio information, it is possible to play and record it on an external device, and the user can listen to the played audio through earphones. This allows the user to verify whether the conversion is correct. In addition, the audio information can be transferred over the network and played and recorded on other external devices. It is also possible to control external devices based on the converted text information. By repeating this series of flows, continuous communication using biometric information becomes possible. [Examples]

[0065] Next, we will describe Example 1 of the character conversion function using the information conversion system of the present invention. A schematic diagram of the information conversion system of the present invention is shown in Figure 3. A schematic diagram of the detection device 100 is shown in Figure 4.

[0066] Here, we show a configuration in which the information processing device of the information conversion system is a smartphone 300. The detection device 100 has multiple biometric information detection units 401, 402, and 403, and sound information detection units 404 and 405. The detection device is powered by a battery (not shown). In addition, the configuration of the smartphone 300 is the same as that of the information processing device 101 shown in Figure 1, except for the positions of the display unit 303 and the sound information output unit 304, so the explanation is omitted here.

[0067] As shown in Figure 4(a), the user wears the detection device 100 around their neck. The detection device 100 is a choker-type (ring-shaped) device that is wrapped around the user's neck and is made of an elastic material (fabric). The detection device 100 may be provided with connecting parts that connect both ends of the elastic material for attachment to the user's neck. As shown in Figure 4(b), the detection device 100 consists of biometric information detection units 401, 402, and 403, sound information detection units 404 and 405, and a transmission unit 406. The biometric information detection units 401, 402, and 403 and the sound information detection units 404 and 405 are connected to the transmission unit 406. The transmission unit 406 can transmit biometric information detected by the biometric information detection units 401, 402, and 403 and sound information detected by the sound information detection units 404 and 405 to the outside.

[0068] The biometric information detection units 401, 402, and 403 are, for example, 6-axis acceleration sensors. A 6-axis acceleration sensor is a sensor that can measure, for example, 3-axis translational acceleration and 3-axis angular acceleration. The biometric information detection units 401, 402, and 403 are installed on the inside (inner surface) of the choker-type detection device 100. Therefore, the biometric information detection units 401, 402, and 403 can be brought into contact with the user's skin, and biometric information (various accelerations) can be detected by the biometric information detection units 401, 402, and 403. As a result, the biometric information detection units 401, 402, and 403 can detect information related to the movement of the user's body parts.

[0069] The sound information detection unit 404 is, for example, a bone conduction microphone that detects sounds emanating from inside the user. A bone conduction microphone is a microphone that detects sound information by detecting vibrations in the bones of the neck when vibrations caused by the user's voice are transmitted to the user's neck. The sound information detection unit 404 may be a microphone that corresponds to various conduction methods, such as a skin conduction microphone that detects vibrations in the skin of the neck and detects sound information. The sound information detection unit 404 is installed on the inside (inner surface) of the choker-type detection device 100. Therefore, the sound information detection unit 404 can be brought into contact with the user's skin and can detect vibrations in the skin of the neck. Thus, the sound information detection unit 404 can detect sounds emanating from inside the user.

[0070] The sound information detection unit 405 is a microphone for capturing external ambient sounds. The sound information detection unit 405 is installed on the outside (outer surface) of the choker-type detection device 100. The sound information detection unit 405 can detect external ambient sounds. The marker 410 is a mark for alignment with the user. The user can use the marker 410 as a guide to align the biometric information detection unit 402 with the center of their neck (such as their throat), and properly attach the detection device 100 to their neck.

[0071] The biometric information detected by the biometric information detection units 401, 402, and 403, and the sound information detected by the sound information detection units 404 and 405, are transmitted by the transmission unit 406 to, for example, a Bluetooth-connected smartphone 300. The receiving unit 301 within the application of the smartphone 300 receives the data. The determination unit 305 analyzes the sound information received by the receiving unit 301 and determines whether the characteristics of the sound information meet predetermined conditions. The conversion unit 302 converts the biometric information or sound information received by the receiving unit 301 into text information or voice information according to the determination result of the determination unit 305. The text information converted by the conversion unit 302 is displayed on the display unit 303 of the smartphone. The sound information output unit 304 plays back the voice information converted by the conversion unit 302.

[0072] Figure 5 shows a flowchart illustrating the operation of the information conversion system of the present invention.

[0073] The user attaches the detection device 100 to their neck. The user attaches the detection device 100 to positions where biological information can be detected by the biological information detection units 401, 402, and 403, and where sound information can be detected by the sound information detection units 404 and 405. The biological information detection units 401, 402, and 403 acquire the user's biological information. (S500)

[0074] The sampling rate of the biological information detection units 401, 402, and 403 is set to 1 kHz.

[0075] At the same time that the biological information detection units 401, 402, and 403 detect biological information, the sound information detection units 404 and 405 detect sound information. (S501)

[0076] The sampling rate of the sound information detection units 404 and 405 is set to 16kHz.

[0077] The transmitter 406 transmits sound information and biometric information to the smartphone 300. (S502)

[0078] The sound information and biometric information are transmitted to the conversion unit 306 and the determination unit 305 via the receiving unit 305. The determination unit 305 uses a neural network model to determine whether or not there is voice information spoken by the user in relation to the sound information detected by the sound information detection unit 404. (S503)

[0079] If the sound information detected by the sound information detection unit 404 includes voice information spoken by the user, the process proceeds to S504. If the sound information detected by the sound information detection unit 404 does not include voice information spoken by the user, the process proceeds to S505.

[0080] Depending on whether the sound information detected by the sound information detection unit 404 contains speech information or not, the conversion unit 302 switches the conversion algorithm to convert the sound information or biometric information into text information. The conversion unit 302 converts the sound information or biometric information spoken by the user into text information or speech information and outputs the text information. (S504)

[0081] If the sound information detected by the sound information detection unit 404 does not contain any speech information, the determination unit 305 determines whether or not there are sounds other than those uttered by the user in the sound information, after the sound information detection unit 404 has acquired sounds related to movements not intended by the user (such as coughing or swallowing saliva). (S505)

[0082] If the sound information contains sounds other than those uttered by the user, the process proceeds to S506. If the sound information detected by the sound information detection unit 404 contains no sounds other than those uttered by the user, the process proceeds to S507.

[0083] If the sound information detection unit 404 detects any sounds other than those uttered by the user, the conversion unit 302 terminates without performing any conversion processing using that information. (S506)

[0084] If the sound information detection unit 404 detects no sounds other than those uttered by the user, the conversion unit 302 converts the biometric information when the user is not speaking into text information or voice information, and outputs the text information. (S507)

[0085] If the audio information contains no sounds other than the user's speech, the conversion unit 302 converts the biometric information into text information or audio information using the first conversion method described above. Specifically, the sensor information from the 18 axes of the 6-axis x 3-axis accelerometer is used as input data for the first conversion algorithm to convert it into text information. The display unit 303 then displays the converted text information. (S508)

[0086] Furthermore, even if the sound information detected by the sound information detection unit 404 includes sounds other than those emitted by the user, if the signal-to-noise ratio of the voice information is poor, the conversion may be performed using biometric information in addition to the voice information. It is also possible to set the system to perform the conversion using only biometric information. [Examples]

[0087] Next, an embodiment 2 of the character conversion function using the information conversion system of the present invention will be described. A schematic diagram of the information conversion system of the present invention is shown in Figure 6. A schematic diagram of the detection device 100 is shown in Figure 7.

[0088] As shown in Figure 6, the detection device 100 has a fitting detection unit 708 that detects when the biometric information detection units 701 to 705 are attached to the user. The configuration of the smartphone 300 used as an information processing device is the same as that of the smartphone 300 shown in Figure 3, so a detailed explanation is omitted here.

[0089] If the wear detection unit 708 detects that the biometric information detection units 701 to 705 are attached to the user, the conversion unit 302 can convert the biometric information detected by the biometric information detection units 701 to 705 into text information. If the wear detection unit 708 does not detect that the biometric information detection units 701 to 705 are attached to the user, it cannot convert the biometric information detected by the biometric information detection units 701 to 705 into text information or voice information.

[0090] The detection device 100 includes biometric information detection units 701 to 705, sound information detection units 706 and 707, and a transmission unit 709 that transmits biometric information detected by the biometric information detection units and sound information detected by the sound information detection units. The smartphone 300 (information processing device) includes a receiving unit 301 that receives biometric information and sound information transmitted from the transmission unit 709, a determination unit 305, and a conversion unit 302.

[0091] When the attachment detection unit 708 detects that the biometric information detection units 701 to 705 have been attached to the user, the detection device 100 and the information processing device 300 become able to communicate with each other.

[0092] As shown in Figure 7(a), the user attaches the detection device 100 to their jaw. As shown in Figure 7(b), the detection device 100 consists of biometric information detection units 701-705, sound information detection units 706 and 707, attachment detection unit 708, and transmission unit 709.

[0093] The detection device 100 is a chin mask type that is worn on the user's chin and is made of a stretchable material (fabric). The detection device 100 is formed from two ear loops 710 and a neck and chin contact portion 711. The neck and chin contact portion 711 is made of a stretchable material (fabric).

[0094] The neck and jaw contact area 711 is equipped with biometric information detection units 701-705, sound information detection units 706 and 707, and a wear detection unit 708. When the user places the ear hooks 710 over both ears and attaches the detection device 100 to the user, the biometric information detection units 701-705, sound information detection units 706 and 707, and wear detection unit 708 are positioned on the user's neck and jaw, respectively.

[0095] The transmitting unit 709 is connected to the biometric information detection units 701-705, the sound information detection units 706 and 707, and the wear detection unit 708. The transmitting unit 709 can transmit to the outside the biometric information detected by the biometric information detection units 701-705, the sound information detected by the sound information detection units 706 and 707, and the wear detection result detected by the wear detection unit 708.

[0096] The bio-information detection units 701-705 are, for example, electromyography sensors and are positioned in the neck and mandible regions. The bio-information detection units 701-705 are installed on the back surface (contact surface) of the detection device 100. Therefore, the bio-information detection units 701-705 can be brought into contact with the user's skin, and bio-information (electromyography information) can be detected by the bio-information detection units 701-705.

[0097] The sound information detection unit 706 is, for example, a bone conduction microphone for detecting sounds emanating from inside the user. The sound information detection unit 706 may also be a skin conduction microphone. The sound information detection unit 706 is installed on the back surface (contact surface) of the detection device 100. Therefore, the sound information detection unit 706 can be placed in contact with the user's skin to detect sounds emanating from inside the user. The sound information detection unit 707 is a microphone for taking in external ambient sounds. The sound information detection unit 707 is installed on the front surface (non-contact surface) of the detection device 100. The sound information detection unit 707 can detect external ambient sounds.

[0098] The attachment detection unit 708 monitors the output of the electromyography sensor and detects that the user has attached the detection device 100 to the neck and jaw. The attachment detection unit 708 detects that the detection device 100 has been attached based on the characteristics of the output waveform of the electromyography sensor. The characteristics of the output waveform of the electromyography sensor include, for example, the maximum value of the electromyography and the waveform pattern of the electromyography. If the maximum value of the electromyography exceeds a predetermined value, the unit detects that the detection device 100 has been attached. In addition, if the waveform pattern of the electromyography is similar to a predetermined waveform pattern, the unit detects that the detection device 100 has been attached.

[0099] As shown in Figure 6, the biometric information detected by the biometric information detection units 701-705, the sound information detected by the sound information detection units 706 and 707, and the wear detection result detected by the wear detection unit 708 are transmitted by the transmission unit 709 to, for example, a Wi-Fi connected smartphone 300. The receiving unit 301 in the application of the smartphone 300 receives the data. The determination unit 305 analyzes the sound information received by the receiving unit 301 and determines whether the characteristics of the sound information satisfy predetermined conditions. The determination unit 305 may also make a determination using the wear detection result detected by the wear detection unit 708.

[0100] The conversion unit 302 converts the biometric or sound information received by the receiving unit 301 into text information according to the determination result of the determination unit 305. The text information converted by the conversion unit 302 is displayed on the display unit 303 of the smartphone. The sound information output unit 304 plays back the sound information converted by the conversion unit 302.

[0101] Figure 8 shows a flowchart illustrating the operation of the information conversion system of the present invention.

[0102] The user attaches the detection device 100 to positions where biological information can be detected by the biological information detection units 701-705 and where sound information can be detected by the sound information detection units 706 and 707. After the detection device 100 is powered on, the biological information detection units 701-705 detect biological information. (S801)

[0103] The sampling rate of the biological information detection units 701-705 is 2 kHz.

[0104] The attachment detection unit 708 detects that the detection device 100 is attached based on the characteristics of the biological information (characteristics of the output waveform of the electromyography sensor). (S802)

[0105] When the attachment detection unit 708 detects that the detection device 100 has been attached, the detection device 100 and the information processing device 300 become capable of communication. The transmission unit 709 of the detection device 100 is connected to the smartphone 300 and can transmit various information. (S803)

[0106] The biometric information detection units 701-705 acquire biometric information, and the sound information detection units 706 and 707 acquire sound information (S804). The sampling rate of the sound information detection units 706 and 707 is set to 16kHz. The transmission unit 709 transmits the sound information and biometric information to the smartphone 300 (S805).

[0107] The sound information and biometric information are transmitted to the conversion unit 306 and the determination unit 305 via the receiving unit 305. The determination unit 305 uses a neural network model to determine whether or not there is voice information spoken by the user in relation to the sound information detected by the sound information detection unit 706. (S806)

[0108] If the sound information detected by the sound information detection unit 706 contains voice information spoken by the user, the process proceeds to S807. If the sound information detected by the sound information detection unit 706 does not contain voice information spoken by the user, the process proceeds to S808.

[0109] Depending on whether the sound information detected by the sound information detection unit 706 contains speech information or not, the conversion unit 302 switches the conversion algorithm to convert the sound information or biometric information into text information. The conversion unit 302 converts the sound information or biometric information spoken by the user into text information and outputs the text information. (S807)

[0110] If the sound information detected by the sound information detection unit 706 does not contain any speech information, the determination unit 305 determines whether or not there are sounds other than those uttered by the user in the sound information, by having the sound information detection unit 706 acquire sounds related to movements not intended by the user (such as coughing or swallowing saliva). (S808)

[0111] If the sound information detection unit 706 detects any sounds other than speech uttered by the user, the process proceeds to S809. If the sound information detection unit 706 detects no sounds other than speech uttered by the user, the process proceeds to S810.

[0112] If the sound information detected by the sound information detection unit 706 contains sounds other than those uttered by the user, the conversion unit 302 terminates the conversion process using that information without performing any conversion. (S809)

[0113] If the sound information detection unit 706 detects no sounds other than those uttered by the user, the conversion unit 302 converts the biometric information when the user is not speaking into text information or voice information, and outputs the text information or voice information. (S810)

[0114] If the sound information detection unit 706 detects no sounds other than speech uttered by the user, the conversion unit 302 converts the biometric information into text information or voice information using the first conversion method described above. Specifically, the electromyographic information from the electromyographic sensor is used as input data for the first conversion algorithm to convert it into text information or voice information. The display unit 303 then displays the converted text information. (S811)

[0115] Furthermore, even if the sound information detected by the sound information detection unit 706 includes voice information emitted by the user, if the signal-to-noise ratio of the voice information is poor, biometric information may be used in addition to the voice information for conversion. It is also possible to set the system to convert using only biometric information. [Examples]

[0116] Next, an embodiment 3 of the character conversion function using the information conversion system of the present invention will be described. A schematic diagram of the detection device 100 is shown in Figure 9. The detection device 100 shown in Figure 9 can be applied to the information conversion system shown in Figure 6.

[0117] The detection device 100 is a glasses-type device with detection units located on a part of the glasses. It consists of a wear detection unit 901, a transmission unit 902, biometric information detection units 903, 904, and 905, and sound information detection units 906 and 907. Biometric information detection units 903 and 904 are electromyography sensors. Biometric information detection unit 905 is a 6-axis accelerometer. The electromyography sensors of biometric information detection units 903 and 904 are installed on the user's temples. The accelerometer of biometric information detection unit 905 is installed on the user's glabella. The sound information detection unit 906 is a bone conduction microphone for detecting sounds emitted from inside the user. The sound information detection unit 907 is a microphone for capturing external ambient sounds.

[0118] The transmitting unit 902 is connected to the wear detection unit 901, the biometric information detection units 903, 904, and 905, and the sound information detection units 906 and 907. The transmitting unit 902 can transmit to the outside the wear detection result detected by the wear detection unit 901, the biometric information detected by the biometric information detection units 903, 904, and 905, and the sound information detected by the sound information detection units 906 and 907.

[0119] The attachment detection unit 901 monitors the output of the electromyography sensor or acceleration sensor and detects that the user has attached the detection device 100 to the user. The attachment detection unit 901 detects that the user has attached the detection device 100 based on the characteristics of the output waveform of the electromyography sensor or acceleration sensor.

[0120] The biometric information detected by the biometric information detection units 903, 904, and 905, the sound information detected by the sound information detection units 906 and 907, and the wear detection result detected by the wear detection unit 901 are transmitted by the transmission unit 902 to, for example, a Bluetooth-connected smartphone 300. The receiving unit 301 in the application of the smartphone 300 receives the data. The determination unit 305 analyzes the sound information received by the receiving unit 301 and determines whether the characteristics of the sound information satisfy predetermined conditions. The determination unit 305 may also make a determination using the wear detection result detected by the wear detection unit 901.

[0121] The conversion unit 302 converts the biometric information or sound information received by the receiving unit 301 into text information according to the determination result of the determination unit 305. The conversion unit 302 can convert electromyography information detected by the electromyography sensors of the biometric information detection units 903 and 904 into text information or voice information. The conversion unit 302 can also convert acceleration information detected by the acceleration sensor of the biometric information detection unit 905 into text information or voice information. Furthermore, the conversion unit 302 can convert information including electromyography information detected by the electromyography sensors of the biometric information detection units 903 and 904 and acceleration information detected by the acceleration sensor of the biometric information detection unit 905 into text information or voice information.

[0122] The character information converted by the conversion unit 302 is displayed on the smartphone's display unit 303. The sound information output unit 304 plays back the audio information converted by the conversion unit 302. [Examples]

[0123] Next, an embodiment 4 of the character conversion function using the information conversion system of the present invention will be described. A schematic diagram of the information conversion system of the present invention is shown in Figure 10. A schematic diagram of the detection device 100 is shown in Figure 11.

[0124] The detection device 100 is a ring-shaped device that is worn on the user's finger and can be fitted onto the user's finger. The detection device 100 consists of a fitting detection unit 1001, a transmission unit 1002, a biometric information detection unit 1003, and a sound information detection unit 1004. The biometric information detection unit 1003 is composed of, for example, a 6-axis tactile sensor. The 6-axis tactile sensor is a sensor that can detect force in three axes and moment in three axes. The sound information detection unit 1004 is a microphone that detects sounds emitted by the user.

[0125] The biometric information detection unit 1003 is installed on the outside (outer surface) of the ring-shaped detection device 100. Therefore, by pressing the detection device 100 against the user's skin and bringing the biometric information detection unit 1003 into contact with the user's skin, the biometric information detection unit 1003 can detect biometric information (tactile information).

[0126] The sound information detection unit 1004 is installed on the outside (outer surface) of the ring-shaped detection device 100. Therefore, the sound information detection unit 1004 can detect sounds emitted by the user.

[0127] The attachment detection unit 1001 monitors the output of the biometric information detection unit 1003 and detects when the user has brought the device into contact with the lower jaw. The attachment detection unit 1001 also detects that the detection device 100 has been attached based on the characteristics of the output waveform of the tactile sensor.

[0128] The transmitting unit 1002 is connected to the biometric information detection unit 1003, the sound information detection unit 1004, and the wear detection unit 1001 via the signal line 1010. The transmitting unit 1002 can transmit to the outside the biometric information detected by the biometric information detection unit 1003, the sound information detected by the sound information detection unit 1004, and the wear detection result detected by the wear detection unit 1001.

[0129] The biometric information detected by the biometric information detection unit 1003 and the sound information detected by the sound information detection unit 1004 are wirelessly transmitted to the smartphone 1105 by the transmission unit 1002. The receiving unit 1106 within the application of the smartphone 1105 receives the biometric information detected by the biometric information detection unit 1003 and the sound information detected by the sound information detection unit 1004. The data transfer unit 1107 transfers the biometric information and sound information to the cloud 1111, and the conversion unit 1108 on the cloud 1111 converts the biometric information into sound information. The conversion method has been explained in the above embodiment, so a detailed explanation is omitted.

[0130] The audio information converted by the conversion unit 1108 is transferred to the data transfer unit 1107. The audio information converted by the conversion unit 1108 is transferred to another smartphone 1110 and played back. The user can simultaneously play back the converted audio via the sound information output unit 1109, such as the earphones 1005, and confirm the conversion result.

[0131] Figure 12 shows a flowchart illustrating the operation of the information conversion system of the present invention.

[0132] The user attaches the detection device 100 to a position where biological information can be detected by the biological information detection unit 1003 and where sound information can be detected by the sound information detection unit 1004. After the detection device 100 is powered on, the biological information detection unit 1003 detects the biological information. (S1201)

[0133] The sampling rate of the tactile sensor is 100Hz.

[0134] The attachment detection unit 1001 detects that the detection device 100 has been attached based on the characteristics of the biological information (characteristics of the output waveform of the tactile sensor). (S1202)

[0135] When the attachment detection unit 1001 detects that the detection device 100 has been attached, the detection device 100 and the smartphone 1110 become capable of communication. Specifically, the transmission unit 1002 of the detection device 100 can connect with the smartphone 1110. (S1203)

[0136] The biological information detection unit 1003 detects biological information, and the sound information detection unit 1004 detects sound information. (S1204)

[0137] The sampling rate of the sound information detection unit 1004 was set to 16kHz. The transmission unit 1002 transmits sound information and biometric information to the smartphone 1105. (S1205)

[0138] The biometric information detected by the biometric information detection unit 1003 and the sound information detected by the sound information detection unit 1004 are transferred from the receiving unit 1106 to the conversion unit 1108 on the cloud 1111 via the data transfer unit 1107. (S1206)

[0139] The conversion unit 1108 converts biometric information into audio information. The conversion unit 1108 determines from the audio information whether the sound is related to movements unintended by the user (such as coughing or swallowing saliva). (S1207)

[0140] If the sound information detection unit 1004 detects any sounds related to movements unintended by the user, the process proceeds to S1208. If the sound information detection unit 1004 does not detect any sounds related to movements unintended by the user, the process proceeds to S1209.

[0141] If the sound information detection unit 1004 detects any sound related to movements not intended by the user, it terminates without converting the biometric information (S1208).

[0142] If the sound information detection unit 1004 does not detect any sounds related to movements unintended by the user, the conversion unit 1108 uses the information acquired by the 6-axis tactile sensor as input information and converts the biometric information into audio information using a conversion algorithm. The audio information converted by the conversion unit 1108 is then transferred to the smartphone 1105. (S1209)

[0143] The conversion algorithm used a pre-trained model based on a neural network architecture. The audio information transferred to the smartphone 1105 is played back through the user's earphones 1005 via the data transfer unit 1107. (S1210)

[0144] Simultaneously, the converted voice information is transferred to the other party (another smartphone 1110). (S1211) [Examples]

[0145] Next, Example 5 of a web conference using the information conversion system of the present invention will be described. A schematic diagram of the information conversion system of the present invention is shown in Figure 13. A schematic diagram of the detection device 100 is shown in Figure 14.

[0146] The detection device 100 is a mask type, with each detection unit located in a part of the mask. The detection device 100 consists of a transmission unit 1301, biometric information detection units 1302 to 1305, and sound information detection unit 1306. The biometric information detection units 1302 to 1305 are, for example, 6-axis acceleration sensors. The sound information detection unit 1306 is a microphone that detects sounds emitted by the user.

[0147] The transmitting unit 1301 is connected to the biometric information detection units 1302-1305 and the sound information detection unit 1306. The transmitting unit 1301 can transmit to the outside the biometric information detected by the biometric information detection units 1302-1305 and the sound information detected by the sound information detection unit 1306.

[0148] By powering on the detection device 100, the detection device 100 is connected to the information processing device 101. Subsequently, the biological information detected by the biological information detection units 1302-1305 and the sound information detected by the sound information detection unit 1306 are wirelessly transmitted to the information processing device 101 by the transmission unit 1301.

[0149] The receiving unit 1406 within the application of the information processing device 101 receives biometric information detected by the biometric information detection units 1302-1305 and sound information detected by the sound information detection unit 1306. The conversion unit 1407 converts the biometric information into sound information, and the sound information is transferred to the information processing device 1410 of the call partner (web conference partner) via the data transfer unit 1408. Simultaneously, the converted sound information is played back via the sound information output unit 1409, such as the earphones 1307.

[0150] Figure 15 shows a flowchart illustrating the operation of the information conversion system of the present invention.

[0151] The user attaches the detection device 100 to positions where biological information can be detected by the biological information detection units 1302-1305 and where sound information can be detected by the sound information detection unit 1306. After the detection device 100 is powered on, the biological information detection units 1302-1305 detect biological information. (S1500)

[0152] The sampling rate of the accelerometer is 1 kHz.

[0153] The sound information detection unit 1306 detects sound information. (S1504) The sound information detection unit 1306 is a microphone, and its sampling rate is 16kHz. The transmission unit 1301 transmits the biological information detected by the biological information detection units 1302-1305 and the sound information detected by the sound information detection unit 1306 to the information processing device 101. (S1502)

[0154] The biometric information detected by the biometric information detection units 1302-1305 and the sound information detected by the sound information detection unit 1306 are received by the receiving unit 1406 and sent to the conversion unit 1407. The conversion unit 1407 or the determination unit (not shown) uses a neural network model to determine whether or not there is voice information spoken by the user in relation to the sound information detected by the sound information detection unit 1306. (S1503)

[0155] If the sound information detected by the sound information detection unit 1306 includes voice information spoken by the user, the process proceeds to S1504. If the sound information detected by the sound information detection unit 1306 does not include voice information spoken by the user, the process proceeds to S1505.

[0156] If the sound information detected by the sound information detection unit 1306 contains voice information spoken by the user, the voice information is transferred to the information processing device 1410 of the other party in the web conference. (S1504)

[0157] If the sound information detected by the sound information detection unit 1306 does not contain any voice information spoken by the user, it determines whether the sound information contains sounds related to movements not intended by the user (such as coughing or swallowing saliva). (S1505)

[0158] If the sound information detected by the sound information detection unit 1306 contains sounds related to movements not intended by the user, the process proceeds to S1506. If the sound information does not contain sounds related to movements not intended by the user, the process proceeds to S1507.

[0159] If the sound information detected by the sound information detection unit 1306 contains sounds related to movements not intended by the user, the system terminates without performing conversion processing using biometric information. (S1506)

[0160] If the sound information detected by the sound information detection unit 1306 does not contain any sounds related to movements unintended by the user, the conversion unit 1407 converts the biometric information into audio information. (S1507)

[0161] The audio information is transferred to the user's earphones 1307 via the data transfer unit 1408 and played back. (S1508)

[0162] Simultaneously, the converted audio information is transferred to the information processing device 1410 of the other party in the web conference. (S1509) [Examples]

[0163] Next, an embodiment 6 of a web conference using the information conversion system of the present invention will be described. A schematic diagram of the detection device 100 is shown in Figure 16. The detection device 100 shown in Figure 16 can be applied to the information conversion system shown in Figure 13.

[0164] The detection device 100 is a headset type having each detection unit in a part of the headset. The detection device 100 consists of a transmitter 1601, a biometric information detection unit 1602, a sound information detection unit 1603, and an earphone 1604. The biometric information detection unit 1602 is an ultrasonic sensor. The ultrasonic sensor is a sensor that can measure the distance from the ultrasonic transmitter / receiver to the object by transmitting ultrasonic waves from the ultrasonic transmitter / receiver towards the object and receiving ultrasonic waves reflected from the object. The biometric information detection unit 1602 can use the ultrasonic sensor to detect ultrasonic information related to changes in the distance from the ultrasonic sensor to the mouth or throat when the user moves their mouth. The sound information detection unit 1603 is a microphone that detects sounds emitted by the user.

[0165] The transmitting unit 1601 is connected to a biometric information detection unit 1602 and a sound information detection unit 1603. The transmitting unit 1601 can transmit to the outside the biometric information detected by the biometric information detection unit 1602 and the sound information detected by the sound information detection unit 1603.

[0166] By powering on the detection device 100, the detection device 100 is connected to the information processing device 101. Subsequently, the biological information detected by the biological information detection unit 1602 and the sound information detected by the sound information detection unit 1603 are transferred to the information processing device 101 by the transmission unit 1601.

[0167] The receiving unit 1406 within the application of the information processing device 101 receives biometric information detected by the biometric information detection unit 1602 and sound information detected by the sound information detection unit 1603. The conversion unit 1407 converts the biometric information into sound information, and the sound information is transferred to the information processing device 1410 of the call partner (web conference partner) via the data transfer unit 1408. Simultaneously, the converted sound information is played back via the sound information output unit 1409, such as the earphone 1604.

[0168] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. This program and the computer-readable storage medium storing the program are included in the present invention.

[0169] The embodiments of the present invention described above are merely examples of how the invention can be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

Claims

1. A biometric information detection unit that detects biometric information from one or more locations on the user, A sound information detection unit that detects sound information, A determination unit that determines whether the characteristics of the sound information satisfy predetermined conditions, If the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, the first conversion method is used to output character information or voice information converted from the biometric information detected by the biometric information detection unit. An information conversion system characterized by comprising a conversion unit that, when the characteristics of the sound information satisfy predetermined conditions, outputs character information or voice information converted from information including the sound information detected by the sound information detection unit and the biometric information detected by the biometric information detection unit using a third conversion method.

2. The information conversion system according to claim 1, characterized in that the sound information detection unit is composed of at least two or more detection units.

3. The information conversion system according to claim 2, characterized in that the sound information detection unit comprises a sound information detection unit that detects sound information emitted from within the body and a sound information detection unit that detects external sounds.

4. The information conversion system according to claim 3, characterized in that the sound information detection unit that detects sound information emitted from within the body is a bone conduction microphone or a skin conduction microphone.

5. The information conversion system according to claim 1, characterized in that if the determination unit determines that the sound information detected by the sound information detection unit contains noise, the conversion unit does not convert the biometric information into text information or audio information.

6. The information conversion system according to claim 1, characterized in that, if the determination unit determines that the volume of the sound related to the user's speech in the sound information detected by the sound information detection unit is greater than a threshold, the conversion unit converts the sound information into text information or voice information.

7. The information conversion system according to claim 1, characterized in that, if the determination unit determines that the signal-to-noise ratio of the sound related to the user's speech in the sound information detected by the sound information detection unit is greater than a threshold, the conversion unit converts the sound information into text information or voice information.

8. The information conversion system according to claim 1, characterized in that, if the determination unit determines that the volume of the sound related to the user's speech in the sound information detected by the sound information detection unit is below a threshold, the conversion unit converts the biometric information into text information or voice information.

9. The information conversion system according to claim 1, characterized in that when the determination unit determines that the sound information detected by the sound information detection unit is a period of speech by the user, the conversion unit converts the sound information into text information or voice information.

10. The information conversion system according to claim 1, characterized in that when the determination unit determines that the sound information detected by the sound information detection unit is a period of non-speaking by the user, the conversion unit converts the biometric information into text information or voice information.

11. The information conversion system according to claim 1, characterized in that the biological information is at least one of electromyographic information, acceleration information, ultrasound information, tactile information, light information, and pressure information.

12. The information conversion system according to claim 1, characterized in that the conversion unit further converts the character information converted from the biometric information into speech information.

13. A biometric information detection unit that detects biometric information from one or more locations on the user, A sound information detection unit that detects sound information, A determination unit that determines whether the characteristics of the sound information satisfy predetermined conditions, If the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, the first conversion method is used to output character information or voice information converted from the biometric information detected by the biometric information detection unit. An information conversion system characterized by comprising a conversion unit that, when the characteristics of the sound information satisfy predetermined conditions, outputs character information converted from the sound information and the biometric information using a third conversion method.

14. The information conversion system according to claim 13, characterized in that, in accordance with the characteristics of the sound information detected by the sound information detection unit, the conversion unit switches from the first conversion method to the third conversion method, or from the third conversion method to the first conversion method, and outputs character information or audio information converted by the switched conversion method.

15. The system includes an attachment detection unit that detects when the biometric information detection unit is attached to the user, The information conversion system according to claim 1, characterized in that when the attachment detection unit detects that the biometric information detection unit has been attached to the user, the conversion unit is capable of converting the biometric information detected by the biometric information detection unit into text information or voice information.

16. The information conversion system according to claim 15, characterized in that the attachment detection unit analyzes the biological information detected by the biological information detection unit to detect that the biological information detection unit has been attached to the user.

17. A detection device comprising: a biological information detection unit; a sound information detection unit; and a transmission unit that transmits the biological information detected by the biological information detection unit and the sound information detected by the sound information detection unit. The information conversion system according to claim 1, characterized in that it comprises an information processing device having a receiving unit for receiving biometric information and voice information transmitted from the transmitting unit, a determination unit, and a conversion unit.

18. The system includes an attachment detection unit that detects when the biometric information detection unit is attached to the user, The information conversion system according to claim 17, characterized in that when the attachment detection unit detects that the biometric information detection unit has been attached to the user, the detection device and the information processing device become capable of communicating with each other.

19. The information conversion system according to claim 17, characterized in that the detection device is one of the following: choker type, chin mask type, glasses type, ring type, mask type, or headset type.

20. The information conversion system according to claim 1, further comprising a processing unit that evaluates the biological information detected by the biological information detection unit based on predetermined evaluation criteria and deletes text information or voice information corresponding to the biological information that does not meet the predetermined evaluation criteria.

21. A biometric information detection unit that detects biometric information from one or more locations of the user, A sound information detection unit that detects sound information, A determination unit that determines whether the characteristics of the sound information satisfy predetermined conditions, If the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, the first conversion method is used to output character information or voice information converted from the biometric information detected by the biometric information detection unit. The system includes a conversion unit that, when the characteristics of the sound information satisfy predetermined conditions, outputs character information converted from the sound information detected by the sound information detection unit using a second conversion method, An information conversion system characterized by updating the trained model used in the first conversion method based on the correspondence between character information converted by the second conversion method using sound information detected by the sound information detection unit and biometric information acquired by the biometric information detection unit, which is acquired simultaneously with the sound information detected by the sound information detection unit.

22. A receiving unit that receives biological information detected by a biological information detection unit and sound information detected by a sound information detection unit, A determination unit that determines whether the characteristics of the sound information satisfy predetermined conditions, An information processing apparatus comprising: a conversion unit that, when the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, outputs character information or voice information converted from biological information detected by the biological information detection unit using a first conversion method; and when the characteristics of the sound information satisfy predetermined conditions, outputs character information or voice information converted from information including the sound information and biological information detected by the biological information detection unit using a third conversion method.

23. The steps include receiving biological information detected by the biological information detection unit and sound information detected by the sound information detection unit, A step of determining whether the characteristics of the sound information satisfy predetermined conditions, An information processing method comprising the steps of: if the characteristics of the sound information detected by the sound information detection unit do not satisfy predetermined conditions, outputting character information or voice information converted from biological information detected by the biological information detection unit using a first conversion method; and if the characteristics of the sound information satisfy predetermined conditions, outputting character information or voice information converted from information including the sound information and biological information detected by the biological information detection unit using a third conversion method.

24. A program for causing a computer to execute the information processing method described in claim 23.

Images