Audio processing system, audio processing method, smart device, and hearable device

Abstract

The present invention provides a technique for allowing a user wearing a hearable device to easily search for a smart device. This audio processing system comprises a smart device and a hearable device connected to the smart device by wireless communication. The audio processing system transmits audio data including ringtone data generated by the smart device to the hearable device, and outputs the audio data as a sound by the hearable device. One of the smart device and the hearable device comprises an audio device that performs 3D sound conversion processing for converting the audio data into 3D sound data so as to allow a user wearing the hearable device to hear the audio data as a sound with the position of the smart device as a sound source position.

Description

Speech processing system, speech processing method, smart device, and hearable device

[0001] The technology disclosed herein relates to a voice processing system and a voice processing method for processing voice information between a smart device and a hearable device, and further to a smart device and a hearable device.

[0002] In recent years, wireless data communication between digital devices located in close proximity has become common. For example, wireless communication based on the Bluetooth (registered trademark) standard is widely used for communication between smart devices such as PCs, tablets, and smartphones and hearable devices such as speakers, headphones, and earphones.

[0003] Through this wireless communication, the smartphone sends audio information such as music to the earphones, which then play the sound, allowing the user wearing the earphones to listen to the audio information. If the earphones have a microphone, the audio information picked up by the microphone can be sent wirelessly from the earphones to the smartphone. In other words, the user wearing the earphones can send and receive phone calls between the smartphone and the earphones.

[0004] While hearable devices such as earphones that connect to smartphones via wireless communication are highly convenient, they also have the disadvantage of being easily lost. Patent Document 1 describes a technology for finding a wireless earphone as a hearable device when it is lost.

[0005] More specifically, in the technology described in Patent Document 1, the wireless earphone itself detects when the wireless earphone has fallen and transmits a dropped location notification signal to a smartphone via Bluetooth communication. The smartphone then determines the location of the wireless earphone based on the dropped location notification signal, vibrates the smartphone, and simultaneously displays the location of the wireless earphone on its display. This allows the user to quickly find the wireless earphone.

[0006] JP 2022-60000 A

[0007] Incidentally, in the relationship between a smart device such as a smartphone and a hearable device such as a wireless earphone, a situation may arise in which the user is wearing the hearable device but loses sight of the smart device.

[0008] However, Patent Document 1 does not describe or suggest any technology for a hearable device, such as a wireless earphone, worn by a user to search for a smart device, such as a smartphone.

[0009] An object of the technology disclosed herein is to provide a technology that allows a user wearing a hearable device to easily search for a smart device.

[0010] In order to solve the above problem, for example, the configuration described in the claims is adopted. The present application includes multiple means for solving the above problem, and one example thereof is an audio processing system including a smart device and a hearable device connected to the smart device via wireless communication, which transmits audio data including ringtone data generated by the smart device to the hearable device and outputs the audio data as sound from the hearable device, and one of the smart device or the hearable device includes an audio device that performs 3D sound conversion processing to convert the audio data into 3D sound data so that a user wearing the hearable device can hear the audio as sound with the position of the smart device as the sound source position.

[0011] By using the technology disclosed herein, a user wearing a hearable device can hear a ringtone (3D ringtone) through the hearable device, as if the sound source were located at the position of the smart device. From this 3D ringtone, the user can determine the approximate location of the smart device and easily find the smart device.

[0012] 1 is a schematic diagram for explaining an overview of an audio processing system according to a first embodiment. FIG. 1 is a diagram showing an example of the appearance of a smartphone according to a first embodiment. FIG. 2 is a diagram showing an example of the appearance of a smartphone according to a first embodiment. FIG. 3 is a system configuration diagram showing an example of the internal configuration of a smartphone according to a first embodiment. FIG. 4 is a system configuration diagram showing an example of the internal configuration of an earphone according to a first embodiment. FIG. 5 is a system configuration diagram showing an example of the internal configuration of a charging case according to a first embodiment. FIG. 6 is a flowchart showing an example of an audio processing procedure when answering a call in a smartphone according to a first embodiment. FIG. 7 is a flowchart showing an example of a processing procedure for Bluetooth preprocessing in a smartphone according to a first embodiment. FIG. 8 is a flowchart showing an example of a processing procedure for advertisement scanning processing in a smartphone according to a first embodiment. FIG. 9 is a flowchart showing an example of a processing procedure for advertisement scanning processing in an earphone according to a first embodiment. FIG. 10 is a diagram showing an example of a selection screen in a smartphone according to a first embodiment. FIG. 11 is a schematic diagram showing a calculation principle for calculating the relative positions of a smartphone and earphones according to a first embodiment. FIG. 12 is a diagram showing an example of an information table stored in a storage device of a smartphone according to a first embodiment. FIG. 13 is a system configuration diagram showing an example of the internal configuration of a smartphone according to a second embodiment. FIG. 14 is a system configuration diagram showing an example of the internal configuration of an earphone according to a second embodiment. FIG. 15 is a diagram showing an example of the appearance of a smartphone according to a second embodiment. FIG. 16 is a flowchart showing an example of an audio processing procedure when answering a call in a smartphone according to a second embodiment. 10 is a flowchart showing an example of a processing procedure for UWB preprocessing in a smartphone according to embodiment 2. FIG. 11 is a flowchart showing an example of a processing procedure for UWB processing in an earphone according to embodiment 2. FIG. 12 is a schematic diagram showing a calculation principle for calculating the relative position between a smartphone and an earphone according to embodiment 2. FIG. 13 is a system configuration diagram showing an example of the internal configuration of an earphone according to embodiment 3. FIG. 14 is a system configuration diagram showing an example of the internal configuration of an earphone according to embodiment 4. FIG. 15 is a schematic diagram illustrating an audio processing system according to embodiment 5. FIG. 16 is a schematic diagram illustrating an audio processing system according to embodiment 6. FIG. 17 is a schematic diagram illustrating an audio processing system according to embodiment 7.

[0013] Hereinafter, embodiments of the technology of the present disclosure will be described in detail with reference to the drawings. (Embodiment 1) <Overall Configuration of Speech Processing System> Fig. 1 is a schematic diagram illustrating an overview of a speech processing system according to embodiment 1. First, with reference to Fig. 1, the overall configuration of the speech processing system according to embodiment 1 will be described.

[0014] As shown in FIG. 1, the voice processing system 10 according to the first embodiment is configured to include a smartphone (hereinafter also referred to as a "smart phone") 100, which is an example of a smart device, and a wireless earphone (hereinafter also referred to as an "earphone") 200, which is an example of a hearable device.

[0015] As an example, the earphone 200 is composed of a left earphone 201 and a right earphone 202. In the audio processing system 10, the left earphone 201 and the right earphone 202 are treated as independent devices, and audio data including ringtone data is transmitted from the smartphone 100 to each of the left earphone 201 and the right earphone 202. In other words, the audio processing system 10 employs a so-called simultaneous left-right transmission method as the audio data transmission method. Hereinafter, the left earphone 201 and the right earphone 202 will be referred to as the earphone 200 unless there is a particular need to distinguish between them. When the earphone 200 is placed in a charging case 300, which serves as a charging device, it is charged by a rechargeable battery (not shown) provided in the charging case 300.

[0016] The smartphone 100 and the earphones 200 are connected via short-range wireless communication, for example, Bluetooth communication, and audio data is transmitted and received between the smartphone 100 and the earphones 200. The smartphone 100 can also make and receive calls and perform voice calls with other landline telephones and mobile phones via telephone network communication. For example, when the smartphone 100 receives an incoming call, the user U1 is notified of the incoming call via the earphones 200, and the call is started when the user U1 answers the call.

[0017] One feature of the technology disclosed herein is that, for example, when the smartphone 100 receives an incoming call, a telephone ringtone with the smartphone 100 as the sound source position (which can also be called a virtual sound source position) is output as a 3D ringtone from the earphones 200 worn by the user U1, thereby notifying the user U1 of the location of the smartphone 100. The procedure for notifying the user U1 of the location of the smartphone 100 will be described in detail later.

[0018] Furthermore, the smartphone 100 is connected to a network 550 to which a network server 500 is connected via an access point 400. The network server 500 may be, for example, a network server that performs various types of calculation processing or a network server that stores various types of data. The user U1 can utilize the network server 500 by connecting the smartphone 100 to the network 550 via the access point 400 as needed.

[0019] <Appearance of smartphone> Figures 2A and 2B are external views showing an example of a smartphone provided in the voice processing system according to embodiment 1, where Figure 2A is a view showing the front side of the smartphone and Figure 2B is a view showing the back side of the smartphone.

[0020] As shown in Figures 2A and 2B, smartphone 100 has a display screen 101 consisting of a touch panel, a front camera (also called an in-camera) 102 for taking selfies on the front side, and a rear camera (also called an out-camera or simply a camera) 103 on the back side.

[0021] An earphone microphone terminal (not shown) is provided on the side of the smartphone 100 on the front camera 102 side (the left side in FIG. 2A ), and an external connection terminal (not shown) is provided on the side opposite the front camera 102 (the right side in FIG. 2A ). A power key and volume keys (not shown) are provided on the upper side (the upper side in FIG. 2A ) on the front of the smartphone 100, and a card tray (not shown) is provided on the lower side (the lower side in FIG. 2A ) on the front of the smartphone 100.

[0022] The smartphone 100 also has a Bluetooth communication function, which is a short-range wireless communication function, and uses this communication function to obtain position (direction and distance) information of the earphone 200, exchange audio data, etc. Furthermore, the smartphone 100 is provided with a transmitting and receiving antenna that supports this Bluetooth communication.

[0023] As an example, the smartphone 100 is provided with three transmitting and receiving antennas: a left antenna 104, a right antenna 105, and a central antenna 106. The left antenna 104 is located at the top left corner on the front of the smartphone 100, the right antenna 105 is located at the top right corner on the front of the smartphone 100, and the central antenna 106 is located at the bottom center on the front of the smartphone 100. In the audio processing system 10 of this embodiment, as will be described later, the position of the earphone 200 is detected using these three transmitting and receiving antennas of the smartphone 100 that are located in different positions.

[0024] <Internal configuration of smartphone> Fig. 3 is a diagram showing an example of the internal configuration of a smartphone included in the audio processing system of embodiment 1. As shown in Fig. 3, the smartphone 100 according to embodiment 1 includes a control device 110, a system bus 120, a storage device 130, a sensor device 140, a communication device 150, a video device 160, an audio device 170, and an operation input device 180.

[0025] The control device 110 is a microprocessor unit that controls the entire smartphone 100 in accordance with a predetermined operation program. The system bus 120 is a data communication path for transmitting and receiving various commands and data between the control device 110 and each device within the smartphone 100. The storage device 130 stores programs for controlling the operation of the smartphone 100, operation setting values, detection values ​​from the sensor device 140, object information including video and audio content, library information, and the like.

[0026] The programs, object information, library information, etc. stored in the storage device 130 may be obtained or updated by downloading from the network server 500. The object information may be data such as videos or still images captured using the imaging functions of the in-camera 102 or out-camera 103 of the smartphone 100.

[0027] The information stored in the storage device 130 also includes information that must be retained even when the smartphone 100 is in an off state, i.e., when no external power is supplied to the smartphone 100. The area of ​​the storage device 130 where the information that must be retained even when the smartphone 100 is in an off state is stored may be, for example, a semiconductor memory such as a flash ROM or an SSD (Solid State Drive), or a magnetic disk drive such as an HDD (Hard Disk Drive).

[0028] The sensor device 140 is composed of a group of sensors including various sensors for detecting the state of the smartphone 100. As an example, the sensor device 140 includes a positioning sensor 141, a geomagnetic sensor 142, an acceleration sensor 143, a gyro sensor 144, and a light sensor 145 that detects the light and darkness around the smartphone 100. The sensor device 140 may also include other sensors, such as an altitude sensor. The sensor device 140, which is composed of this group of sensors, can appropriately detect the position, tilt, direction, movement, etc. of the smartphone 100.

[0029] Then, as will be described later, the control device 110 calculates data (first relative position data) on the relative position of the earphone 200 with respect to the smartphone 100 based on the detection results of this sensor device 140. In other words, the control device 110 according to this embodiment also functions as a position calculation device that calculates first relative position data that specifies the relative position between the earphone 200 and the smartphone 100.

[0030] The communication device 150 includes a LAN (Local Area Network) communication device 151, a telephone network communication device 152, and a Bluetooth communication device 153, which is a short-range wireless communication device. The LAN communication device 151 is connected to a network (Internet network) 550 via an access point 400, and transmits and receives data to and from a network server 500 on the network 550. The LAN communication device 151 and the access point 400 are connected wirelessly by Wi-Fi (registered trademark) or the like. Note that there may be multiple network servers 500 for each service or service content.

[0031] The telephone network communication device 152 performs telephone communication (calls) and transmission and reception of data through wireless communication with base stations of a mobile telephone communication network. Communication between the telephone network communication device 152 and base stations, etc., is performed, for example, by the LTE (Long Term Evolution) method, the 5G method (a fifth-generation mobile communication system aiming for high speed, large capacity, low latency, and multiple simultaneous connections), etc. However, the communication method between the telephone network communication device 152 and base stations, etc., is not particularly limited, and other communication methods may also be used.

[0032] The Bluetooth communication device 153 performs Bluetooth communication with other devices having a Bluetooth communication function, such as advertisement scanning processing and data communication, which will be described later. Note that the communication device 150 may also include other communication devices, such as an infrared communication device.

[0033] The LAN communication device 151, telephone network communication device 152, and Bluetooth communication device 153 that make up the communication device 150 each include an encoding circuit, a decoding circuit, an antenna, etc. Here, the antenna that is a component of the Bluetooth communication device 153 is an important part related to calculating the position of a device (e.g., earphones 200, etc.) that has a Bluetooth communication function. The left antenna 104, right antenna 105, and center antenna 106 shown in Fig. 2A are Bluetooth antennas that are components of the Bluetooth communication device 153, and are positioned in consideration of detecting the device's position.

[0034] The imaging device 160 includes an imaging sensor 161, a display 162, and a video processor 163 that processes video signals. The imaging sensor 161 converts light input from a lens into an electrical signal using an electronic device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The display 162 is a display device such as a liquid crystal display or an OLED (Organic Light Emitting Diode), and provides images, additional information, and the like to a user U1 of the smartphone 100.

[0035] The audio device 170 includes an audio sensor 171, an audio output element 172, and an audio processor 173 that processes audio signals. The audio sensor 171 is a so-called microphone that converts real-world sounds, the user's voice, and the like into audio data and inputs the data. The audio sensor 171 may be configured to allow an external microphone to be connected via an earphone microphone terminal. Furthermore, the audio sensor 171 may be connected to an external microphone via, for example, a Bluetooth communication device 153. It goes without saying that the configuration of the audio sensor 171 can be selected depending on the application.

[0036] The audio output element 172 is a so-called speaker that outputs audio information and the like required by the user U1 as sound. The audio output element 172 may be configured to allow connection of external earphones or headphones via an earphone microphone terminal. Furthermore, the audio output element 172 may be connected to the external earphones or headphones via, for example, a Bluetooth communication device 153. In this embodiment, the audio output element 172 is connected to the earphone 200 via wireless communication as described above.

[0037] It goes without saying that the configuration of the audio output element 172 can be selected depending on the application. Furthermore, a headset having both earphone and microphone functions can be connected to the earphone / microphone terminal, allowing audio signal input and output simultaneously for headset conversation. In other words, the audio sensor 171 and audio output element 172 may be configured so that a headset having both earphone and microphone functions can be connected.

[0038] The audio processor 173 processes audio signals (audio data) between the smartphone 100 and the earphones 200. The audio processor 173 performs 3D sound conversion processing as part of the audio signal processing. More specifically, the audio processor 173 performs 3D sound conversion processing for outputting audio (ringtone) from the earphones 200 worn by the user U1, with the position of the smartphone 100 as the sound source position. This 3D sound conversion processing will be described in detail later.

[0039] The operation input device 180 is configured with a touch panel or the like for inputting operation instructions and the like to the smartphone 100. In this embodiment, the smartphone 100 further includes a vibrator 190 that vibrates the smartphone 100 and an illumination light 195 that emits light. The operations of the vibrator 190 and the light 195 are controlled by, for example, the control device 110.

[0040] The hardware configuration of the smartphone 100 is not limited to the example hardware configuration shown in FIG. 3 . That is, the hardware configuration of the smartphone 100 may not include non-essential components. Even if the non-essential components are not included, the effect of the technology of the present disclosure is not impaired. Furthermore, the smartphone 100 may include other non-essential components not shown, such as an electronic money payment function.

[0041] <Example of earphone system configuration> The earphone 200 included in the audio processing system 10 according to embodiment 1 is configured with various blocks described below. Fig. 4 is a diagram showing an example of the internal configuration of the earphone according to embodiment 1. Note that the left earphone 201 and the right earphone 202 shown in Fig. 1 have the same system configuration, and therefore may be referred to simply as the earphone 200 without distinguishing between the left earphone 201 and the right earphone 202.

[0042] As shown in FIG. 4 , the earphone 200 includes a control device 210 , a system bus 220 , a storage device 230 , a communication device 240 , an audio device 250 , an operation input device 260 , a display element 270 , and a vibrator 280 .

[0043] The control device 210 is a microprocessor unit that controls the entire earphone 200 according to a predetermined operating program. The control device 210 also controls the operation of the vibrator 280, for example. The system bus 220 is a data communication path for transmitting and receiving various commands and data between the control device 210 and each device within the earphone 200.

[0044] The storage device 230 is composed of storage areas such as a storage area for storing programs for controlling the operation of the earphone 200, an area for storing various data, and a work area used for various program operations. The communication device 240 includes a Bluetooth communication device 241, which is a short-range wireless communication device. The Bluetooth communication device 241 performs Bluetooth communication with the smartphone 100, such as advertisement scanning processing and audio data communication, which will be described later. The configuration of the communication device 240 is not particularly limited.

[0045] The communication device 240 may further include, for example, a communication device for exchanging data between the left earphone 201 and the right earphone 202. Alternatively, the Bluetooth communication device 241 may be provided in only one of the left earphone 201 or the right earphone 202. In this case, audio data is transferred to the other earphone, which is not provided with the Bluetooth communication device 241, via a communication device between the left earphone 201 and the right earphone 202. The earphone 200 may also have a headphone configuration in which the left earphone 201 and the right earphone 202 are connected by wire.

[0046] The audio device 250 includes an audio sensor 251, an audio output element 252, and an audio processor 253 that processes audio signals. The audio sensor 251 is a microphone that converts real-world sounds, the user's voice, etc. into audio data and inputs it. The audio data input from the audio sensor 251 is sent to the smartphone 100 via the Bluetooth communication device 241.

[0047] Note that when the earphone 200 is connected to the smartphone 100 via wireless communication and, for example, a voice call is made, the voice sensor 171 of the smartphone 100 may be used as a microphone for a call, instead of using the voice sensor 251 of the earphone 200. If the voice sensor 171 of the smartphone 100 can be used as a microphone for a call, the earphone 200 does not need to be equipped with the voice sensor 251.

[0048] The audio output element 252 is an audio vibrator that outputs, as sound, audio data required for the user U1, including ringtone data received from the smartphone 100 via the Bluetooth communication device 241. The operation input device 260 is a device for inputting operation instructions and the like to the earphones 200. As an example, the operation input device 260 is configured with a multi-function button whose function changes depending on the number of times the button is pressed, the length of time it is pressed, etc. Operation instructions include, for example, play (PLAY), pause (PAUSE), skip forward, go back, answer a call, and end a call.

[0049] In this embodiment, the earphone 200 is turned on by removing the earphone 200 from the charging case 300, and the earphone 200 is turned off by placing the earphone 200 in the charging case 300.

[0050] The audio processor 253, like the audio processor 173 of the smartphone 100, performs audio signal processing between the smartphone 100 and the earphones 200. In the present embodiment, as part of the audio signal processing, the audio processor 253 plays audio data including ringtone data transmitted from the smartphone 100 to the earphones 200 and outputs the audio data from the audio output element 252.

[0051] The display element 270 presents the status of the earphones 200 to the user U1. As an example, the display element 270 flashes blue when the earphones 200 are removed from the charging case 300 and turned on, and lights red when the earphones 200 are placed back into the charging case 300 and turned off. Of course, the manner in which the display element 270 is used is not particularly limited. The display element 270 may present the communication status and operation status of the earphones 200 to the user U1 in other display formats. Charging of the earphones 200 is achieved by placing the earphones 200 in the charging case 300. At that time, the display element 270 lights up red and turns off when charging is complete.

[0052] <Example of system configuration of charging case> Fig. 5 is a diagram showing an example of the internal configuration of a charging case. The charging case 300 is a general-purpose case and is composed of various blocks described below. As shown in Fig. 5, the charging case 300, which is an example of a charging device, includes a control device 310, a system bus 320, a power supply device 330, and a display element 340.

[0053] The control device 310 is a microprocessor unit that controls the entire charging case 300 in accordance with a predetermined operation program. The system bus 320 is a data communication path for transmitting and receiving various commands, data, and the like between the control device 310 and each device in the charging case 300.

[0054] Here, the main function of the charging case 300 is to supply power (charge) to the set earphones 200. To supply power to the earphones 200, it is first necessary to charge the charging case 300, more specifically, to charge a rechargeable battery (not shown) such as a lithium ion battery provided in the charging case 300.

[0055] The charging case 300 is charged by the power supply device 330. Power is supplied to the power supply device 330 from the outside (commercial power via a USB port or AC adapter) via a charging cable connected to the charging terminal. In other words, the charging case 300 is charged by power supplied to the power supply device 330 from the outside. After that, power is supplied (charged) to the earphones 200 placed in the charging case 300 according to the amount of charge in the charging case 300.

[0056] Of course, it is also possible to simultaneously charge the charging case 300 and the earphones 200 placed in the charging case 300. After charging of the rechargeable battery is complete, the charging case 300 can be moved to any location by disconnecting the charging cable connected to the charging terminal from the charging terminal.

[0057] The display element 340 is used to indicate the operating state of the charging case 300. For example, the display element 340 is lit while the charging case 300 is being charged, and is turned off when charging of the charging case 300 is complete.

[0058] 6 to 8 are flowcharts illustrating the audio processing procedure in a smartphone that constitutes the audio processing system of embodiment 1. The processing procedures shown in Fig. 6 to 8 will be described with reference to the system configuration diagram in Fig. 3.

[0059] The audio processing procedure (audio processing method) in the smartphone 100 is realized by the program of the control device 110 stored in the storage device 130 using the Bluetooth communicator 153 of the communication device 150, the audio processor 173 of the audio device 170, etc. The audio processing procedure described here is an example that assumes that the smartphone 100 and the earphones 200 are wirelessly connected.

[0060] 6 is a flowchart showing the audio processing procedure when answering a call on a smartphone. As shown in FIG. 6, prior to answering a call, Bluetooth pre-processing is performed in step S401. In the Bluetooth pre-processing, the smartphone 100 detects the earphones 200, which are hearable devices connectable via Bluetooth, and establishes a Bluetooth connection between the smartphone 100 and the earphones 200. The left antenna 104, right antenna 105, and center antenna 106 provided on the smartphone 100 are used to send and receive Bluetooth communication.

[0061] 7 is a flowchart showing an example of the processing procedure of the Bluetooth pre-processing. When the Bluetooth pre-processing is started in step S401, as shown in FIG. 7, first, an advertisement scan process, which is a subroutine, is executed in step S421.

[0062] The advertise scan process is a process aimed at obtaining Bluetooth device information for the earphone 200, which is a hearable device. This advertise scan process obtains, for example, the device name of the earphone 200. Here, the earphone 200 refers to the left earphone 201 and the right earphone 202, as described above. In other words, the advertise scan process obtains the device names of the left earphone 201 and the right earphone 202.

[0063] This subroutine, the advertisement scan process, will be described in more detail with reference to Fig. 8. Fig. 8 is a flowchart showing the procedure of the advertisement scan process in a smartphone.

[0064] When the advertisement scan process in the smartphone 100 starts in step S421, as shown in Fig. 8, first, it is determined in step S431 whether new advertisement data has been received. Note that advertising is a phase in which the earphone 200, which has a Bluetooth communication function, transmits advertisement data to the outside via broadcast communication. The advertisement data includes the device name of the earphone 200.

[0065] If it is determined in the judgment process of step S431 that new advertising data has not been received (step S431: No), the process proceeds to step S432, where a time-out judgment process is performed. In step S432, it is determined whether the time during which new advertising data has not been received has reached a predetermined time. If it is determined in the judgment process of step S432 that the time during which new advertising data has not been received has not reached the predetermined time (step S432: No), the process returns to step S431, and the new advertising data reception judgment process continues. On the other hand, if it is determined in the judgment process of step S432 that the time during which new advertising data has not been received has reached the predetermined time (step S432: Yes), the new advertising data reception judgment process is terminated. In other words, the advertisement scan process of step S421 is terminated.

[0066] Furthermore, if it is determined in the determination process of step S431 that new advertisement data has been received (step S431: Yes), the process proceeds to step S433, where a scan request is transmitted. In this scan request transmission process, a scan request is transmitted from the smartphone 100 to the earphone 200 that is transmitting the new advertisement data determined in the determination process of step S431.

[0067] If it is determined in the advertisement data reception determination process (step S431) that multiple pieces of new advertisement data have been received, the new advertisement data that is first detected during the polling search of the device is selected. As an example, in step S433, the smartphone 100 transmits a scan request to the earphone 200 that is transmitting the new advertisement data that is first detected during the polling search of the earphone 200.

[0068] Next, in step S434, it is determined whether or not there has been a scan response from the target earphone 200. If it is determined in the determination process of step S434 that there has been no scan response from the target earphone 200 (step S434: No), the process proceeds to step S435, where timeout determination process is performed.

[0069] If the timeout determination process in step S435, in other words, the time elapsed since the start of transmission of the scan request in step S433, has not reached a predetermined time (step S435: No), the process returns to step S434, where the determination of whether or not a scan response has been received continues. On the other hand, if the timeout determination process in step S435 has reached a predetermined time (step S435: Yes), the determination of whether or not a scan response has been received ends. In other words, the advertising scan process (step S421) ends.

[0070] If it is determined in the determination process of step S434 that a scan response has been received from the target earphone 200 (step S434: Yes), device information for the earphone 200, including the device name, is saved in the storage device 130 (step S436).Then, the process proceeds to step S437.

[0071] In the determination process of step S437, it is determined again whether new advertisement data has been received. If it is determined in the determination process of step S437 that new advertisement data has been received (step S437: Yes), the process returns to step S433, and the scan request transmission process is performed again. For example, if there are multiple devices around the smartphone 100 that can receive new advertisement data, it is determined in step S437 that new advertisement data has been received.

[0072] On the other hand, if it is determined in the determination process of step S437 that new advertisement data cannot be received (step S437: No), that is, if it is determined that another device does not exist, the advertisement scan process (step S421) ends. Note that the determination process of step S437 is the same as the determination process of step 431.

[0073] Next, the advertise scan process in the earphone 200, which is a hearable device, will be described. Fig. 9 is a flowchart showing the procedure for the advertise scan process in the earphone. The procedure in Fig. 9 will be described with reference to the system configuration diagram in Fig. 4.

[0074] The advertising scan process in the earphone 200 is implemented by the program of the control device 210 stored in the storage device 230 using the Bluetooth communication device 241 of the communication device 240. Furthermore, if positioning is not performed from the earphone 200 side, the Bluetooth communication device 241 may have one or more Bluetooth antennas.

[0075] For example, when the earphone 200 is powered on and advertising / scanning processing in the earphone 200 is started, first, in step S441, the earphone 200 transmits advertising data by broadcast communication. Next, it is determined whether a scan request has been received from the smartphone 100 (step S442). If it is determined in the determination processing of step S442 that a scan request has not been received (step S442: No), the process proceeds to step S443, where timeout determination processing is performed.

[0076] Here, if it is determined that the time of the timeout determination process in step S443, in other words, the time during which no scan request has been received since the advertisement data was transmitted, has not reached the predetermined time (step S443: No), the process returns to step S441 and the advertisement data transmission process continues.

[0077] On the other hand, if it is determined that the timeout determination process in step S443 has reached the predetermined time (step S443: Yes), the advertise scan process in the earphone 200 (step S421) is terminated.

[0078] If it is determined in the determination process of step S442 that a scan request has been received from the smartphone 100 (step S442: Yes), the process proceeds to step S444. In step S444, a scan response is transmitted to the outside. With these steps, the advertise scan process in the earphone 200 ends.

[0079] The advertise scan process (see FIG. 8 ) in the smartphone 100 and the advertise scan process (see FIG. 9 ) in the earphone 200 described above are closely related. For example, a new advertise reception determination process (step S431) and a scan response reception determination process (step S434) in the smartphone 100 are established corresponding to the advertise transmission process (step S441) and the scan response transmission process (step S444) in the earphone 200, respectively. Also, for example, a scan request reception determination process (step S442) in the earphone 200 is established corresponding to the scan request transmission process (step S433) in the smartphone 100.

[0080] 7 , the processing procedure of the Bluetooth pre-processing in the smartphone 100 will be further described. When the subroutine advertise scan processing (step S421) ends, the process proceeds to step S422, where a device name display processing is executed. In the device name display processing, the device name obtained in the advertise scan processing is displayed on the display 162 of the video device 160.

[0081] 10 is a diagram illustrating an example of a screen of a display on which a device name is displayed by the device name display process, that is, a diagram illustrating an example of a selection screen on the smartphone according to the first embodiment.

[0082] 10 , the name of the earphone 200, which is the device detected by the advertise scan process, is displayed on the display 162 of the smartphone 100. As an example, the screen (selection screen) of the display 162 displays the device name of the left earphone 201, "EAR-X100L," and the device name of the right earphone 202, "EAR-X100R," in a selectable manner.

[0083] Next, in step S423, it is determined whether the device name display process has been completed for all devices detected in the advertisement scan process. For example, it is determined whether the device name display process has been completed for each of the left earphone 201 and the right earphone 202.

[0084] If it is determined in step S423 that the device name display process has not been completed for all of the detected devices (step S423: No), the process returns to step S422, where the device name display process continues. If it is determined in step S423 that the device name display process has been completed for all of the detected devices (step S423: Yes), the selection process of step S424 is executed.

[0085] The selection process of step S424 is a process in which the user U1 selects a device to be connected to the smartphone 100. This selection process is realized by the user U1 using the operation input device 180 to issue a selection instruction for the device name displayed on the display 162 by the device name display process of step S422.

[0086] For example, as shown in FIG. 10 , if the display 162 displays the device name of the left earphone 201, “EAR-X100L,” as number 1 and the device name of the right earphone 202, “EAR-X100R,” as number 2, the user U1 selects and instructs the numbers corresponding to the left earphone 201 and the right earphone 202 using the operation input device 180.

[0087] The selection method using the operation input device 180 is not particularly limited, and examples include methods using a UI such as a touch panel, voice recognition, or gestures. In the case of selection by voice recognition, the voice of the user U1 input from the voice sensor 171 of the audio device 170 is recognized, and a specific device is selected based on the results of the voice recognition. For example, if the user U1 utters "number one," the number one corresponds to the device name "EAR-X100L" of the left earphone 201, so the left earphone 201 is selected. Also, for example, if the user U1 utters "number two," the number two corresponds to the device name "EAR-X100R" of the right earphone 202, so the right earphone 202 is selected.

[0088] After that, Bluetooth connection processing is performed in step S425. Specifically, the earphones 200 (left earphone 201 and right earphone 202) selected in the device selection processing in step S424 are connected to the smartphone 100 via Bluetooth communication. Note that when the Bluetooth connection between the earphones 200 and the smartphone 100 is successful, the display element 270 of the earphones 200 flashes blue, indicating that pairing is in progress. With the above processing, the Bluetooth pre-processing (step S401) in the smartphone 100 ends.

[0089] 6, a further description will be given of the voice processing procedure when answering a call in the smartphone 100. Note that the voice processing procedure (voice processing method) described below forms the framework of the technology of the present disclosure.

[0090] As described above, once the Bluetooth pre-processing (step S401) is completed and the wireless connection between the smartphone 100 and the earphones 200 is completed, the process proceeds to step S402. In step S402, the telephone network communicator 152 of the communication device 150 determines whether or not there is an incoming call to the smartphone 100. If it is determined that there is no incoming call to the smartphone 100 (step S402: No), the process returns to the determination process of step S402. On the other hand, if it is determined that there is an incoming call to the smartphone 100 (step S402: Yes), the process proceeds to step S403.

[0091] From step S403 onwards, 3D sound conversion processing (also referred to as 3D audio processing) is executed by the audio device 170. In this 3D sound conversion processing, ringtone data (audio data) in the smartphone 100 is converted into predetermined 3D sound data. More specifically, in the 3D sound conversion processing, the ringtone data (audio data) in the smartphone 100 is converted into 3D sound data so that the ringtone data (audio data) in the smartphone 100 can be heard by the user U1 wearing the earphones 200 as sound whose sound source position is the position of the smartphone 100, which is a smart device.

[0092] In this embodiment, when the audio device 170 performs 3D sound conversion processing on the ringtone data, first, in step S403, for example, the control device 110 of the smartphone 100 calculates first relative position data that specifies the relative positions of the smartphone 100 and the earphones 200. The calculated first relative position data is stored in the storage device 130 (step S404).

[0093] 11 is a schematic diagram illustrating the calculation principle for calculating the relative position (first relative position data) between the smartphone and earphones according to the first embodiment. As shown in FIG. 11 , the smartphone 100 has three Bluetooth antennas: a left antenna 104, a right antenna 105, and a central antenna 106. As described above, the left antenna 104 and the right antenna 105 are provided on the upper side of the smartphone 100, and the central antenna 106 is located in the center of the side opposite the left antenna 104 and the right antenna 105 (the lower side of the smartphone 100) (see FIG. 2A ). As a result, the distance from the central antenna 106 to the left antenna 104 is equal to the distance from the central antenna 106 to the right antenna 105.

[0094] In this embodiment, radio waves are received by multiple antennas, and the position of the earphones 200 relative to the smartphone 100 is calculated using the so-called AoA (Angel of Arrival) method, which is a method of detecting the position of the radio waves based on the arrival angle, phase difference, etc. of the received radio waves. More specifically, the left antenna 104, right antenna 105, and center antenna 106 each receive radio waves transmitted from the earphones 200, and first relative position data that specifies the position of the earphones 200 relative to the smartphone 100 is calculated based on the arrival angle, phase difference, etc. of the radio waves received by each of the antennas 104, 105, and 106.

[0095] In other words, as shown in Figure 11, the position P1 where line L1 indicating the direction of the radio waves received by the left antenna 104, line L2 indicating the direction of the radio waves received by the right antenna 105, and line L3 indicating the direction of the radio waves received by the center antenna 106 intersect is calculated as the relative position of the earphone 200 with respect to the smartphone 100 (first relative position data).

[0096] In this embodiment, the first relative position data that specifies the relative position between the smartphone 100 and the earphones 200 is calculated when the smartphone 100 receives an incoming call, but the timing for calculating the first relative position data is not particularly limited. For example, the first relative position data may be calculated at predetermined intervals regardless of whether there is an incoming call. Furthermore, if the relative position calculation process for the earphones 200 is continued even during transmission of a 3D ringtone in step S406 (described later) and the result is reflected in the generation of the 3D ringtone in real time, it has the effect of making it easier for the user to find the smartphone 100.

[0097] Once the calculated first relative position data is stored in the storage device 130 (step S404), the audio processor 173 of the audio device 170 performs a 3D sound ringtone generation process in step S405, and starts generating 3D ringtone data with the smartphone 100 as the sound source. That is, based on the first relative position data and the like stored in the storage device 130, the audio device 170 starts a process of converting the ringtone data (audio data) in the smartphone 100 into 3D ringtone data (3D sound data) so that the user U1 wearing the earphones 200 can hear the sound with the smartphone 100 as the sound source. An example of this conversion is a conversion using a head-related transfer function of the user U1 described by a frequency attenuation rate, an impulse response, or the like. Furthermore, the above conversion may be simplified to simply apply attenuation and a time difference according to the relative distance between the smartphone 100 and the earphones 200.

[0098] Next, in step S406, a 3D ringtone transmission process is executed, and the 3D ringtone data generated in step S405 is transmitted from the smartphone 100 to the earphones 200 via Bluetooth communication. As a result, a 3D ringtone with the smartphone 100 as the sound source is output from the earphones 200 toward the user U1, notifying the user U1 of an incoming call. In this example, the 3D ringtone data transmitted from the smartphone 100 to the earphones 200 is played by the audio processor 253, and the 3D ringtone is output toward the user U1 wearing the earphones 200. By having the user U1 listen to this 3D ringtone, sound image position localization of the 3D smartphone 100 can be achieved.

[0099] In addition, when there is an incoming call to the smartphone 100, an incoming call presentation method such as vibrating the vibrator 190 provided on the smartphone 100 or turning on or blinking the light 195 may be adopted in addition to outputting the ringtone. Furthermore, when there is an incoming call to the smartphone 100, the vibrator 190 may not vibrate or the light 195 may not light up, but when the user U1 hears the 3D ringtone and turns towards the smartphone 100, the vibrator 190 of the smartphone 100 may vibrate or the light 195 may light up (or blink). This makes it easier for the user U1 to find the smartphone 100. For example, even if the smartphone 100 is buried under documents on a desk, the user U1 may find the smartphone 100 more easily.

[0100] Furthermore, for example, when stereo music is being played, the audio processor 173 performs 2D audio processing. That is, when stereo music is being played, 2D audio data that has been subjected to 2D audio processing is transmitted from the smartphone 100 to the earphones 200. When the smartphone 100 receives an incoming call in this state, music playback using 2D audio processing is stopped, a ringtone is played, and 3D audio conversion processing is performed to convert the ringtone data into 3D audio data. In this example, the audio processor 173 stops transmitting 2D audio data from the smartphone 100 to the earphones 200 and starts transmitting 3D audio incoming data (3D audio data) from the smartphone 100 to the earphones 200.

[0101] If the smartphone 100 is set to continue playing music even when a call comes in, when the smartphone 100 receives an incoming call, the volume of the music being played may be lowered to mono, the ringtone may be superimposed on the music and output, and 3D sound conversion processing may be performed on the ringtone data.

[0102] Also, before an incoming call or after the end of a call, the smartphone 100 may play music data received from the network server 500 through the earphones 200. If the music data is in stereo format, 2D audio processing is performed without applying 3D audio processing.

[0103] Furthermore, in step S403, if it is determined that the radio waves received by the antennas 104, 105, and 106 of the smartphone 100 are weak and the calculation accuracy of the relative positions between the smartphone 100 and the earphones 200 is low, or if it is determined that the calculation results of the relative positions of the multiple antennas 104, 105, and 106 of the smartphone 100 are inconsistent, it is not necessary to perform 3D sound conversion processing on the ringtone data. In this case, for example, monaural ringtone data may be sent from the smartphone 100 to the earphones 200 to notify the user U1 of an incoming call.

[0104] Furthermore, when the relative positions of the smartphone 100 and the earphones 200 cannot be determined, the tone, rhythm, and melody of the ringtone may be changed from those when the relative positions are determined, thereby informing the user U1 that the relative positions are not determined.

[0105] Also, for example, if it takes time to calculate the relative position of the earphone 200 with respect to the smartphone 100, 2D ringtones (monaural ringtones) of approximately equal volume can be output from the left earphone 201 and the right earphone 202 until this relative position is obtained and the 3D sound conversion process 412 is completed, thereby quickly notifying the user U1 of an incoming phone call.

[0106] Furthermore, for example, even when the radio waves received by the antennas 104, 105, and 106 of the smartphone 100 are weak and the reliability of the calculation accuracy of the relative position is determined to be low, the calculation of the relative position may be continued while sending monaural ringtone data from the smartphone 100 to the earphones 200. In other words, if the reliability of the first relative position data is lower than a preset threshold, the 3D sound conversion process may not be performed. Then, when it is determined that a reliable calculation result has been obtained, that is, when the reliability of the first relative position data is equal to or greater than a preset threshold, the process may proceed to the next step S404. This makes it possible to both notify the user U1 of an incoming call early and accurately notify the user U1 of the location of the smartphone 100.

[0107] Even when the relative position (first relative position data) between the smartphone 100 and the earphones 200 is calculated, for example, if the distance between the left earphone 201 and the right earphone 202 is greater than a preset threshold, a 2D ringtone (or a monaural ringtone) may be output from the earphones 200. In other words, if the left earphone 201 and the right earphone 202 are separated by a distance greater than the width of the user U1's head, for example, a 2D ringtone (or a monaural ringtone) may be output from the earphones 200. In other words, if the calculation result of the relative position between the smartphone 100 and the earphones 200 is determined to be inappropriate, a 2D ringtone (or a monaural ringtone) may be output from the earphones 200. This makes it possible to prevent the user U1 from being informed of an incorrect position of the smartphone 100.

[0108] Next, in step S407, it is determined whether the incoming call to the smartphone 100 is still ongoing. If it is determined in the determination process of step S407 that the incoming call to the smartphone 100 is not still ongoing (step S407: No), it is determined that the other party has hung up the phone, and transmission of the 3D ringtone data from the smartphone 100 to the earphones 200 is terminated (step S408). In other words, the processing in the smartphone 100 returns from the 3D sound conversion process to the 2D sound processing before the incoming call. In other words, the 3D sound conversion process is stopped after the incoming call ends, for example, after a predetermined period has elapsed. Also, for example, if the user U1 performs an operation to hang up the incoming call, it is determined that the incoming call is not still ongoing (step S407: No), and transmission of the 3D sound ringtone is terminated in step S408.

[0109] On the other hand, if it is determined in the determination process of step S407 that the incoming call is continuing (step S407: Yes), then the determination process of step S409 is executed. In the determination process of step S409, it is determined whether or not the user U1 has performed an answering operation. The "answering operation" refers to an operation in which the user U1 presses the multifunction button of the operation input device 260 of the earphone 200 while the ringtone (call tone) is ringing, or an operation in which the user U1 issues an instruction to answer the call to the operation input device 180 of the smartphone 100.

[0110] If it is determined that the user U1 has not performed an answering operation (step S409: No), the process returns to step S406, and the 3D ringtone data continues to be transmitted to the earphone 200. In other words, the 3D ringtone continues to be output from the earphone 200. During this time, the user U1 can hear the 3D ringtone, find the smartphone 100, and check the caller (the other party) displayed on the smartphone 100.

[0111] If it is determined in the determination process of step S409 that the user U1 has performed an answering operation (step S409: Yes), the process of step S410 is executed. When the user U1 has performed an answering operation, a process of transmitting and receiving call audio data between the smartphone 100 and the earphones 200 is performed, and the process of transmitting 3D ringtone data from the smartphone 100 to the earphones 200 is terminated (step S410). That is, the 3D sound conversion process is stopped, for example, after a predetermined period of time has elapsed after the start of a call by the smartphone 100. Thereafter, general call audio data (monaural audio data or stereo audio data) is transmitted and received between the smartphone 100 and the earphones 200 (step S411). That is, the processing by the audio processor 173 switches from 3D audio processing to 2D audio processing.

[0112] If the earphone 200 is outputting a ringtone and the vibrator 190 of the smartphone 100 is vibrating or the light 195 is turned on, the operation of the vibrator 190 and the light 195 is also stopped in step S410.

[0113] Incidentally, when the user U1 operates the operation input device 180, for example, when an answering operation is performed on the operation input device 180 in step S409, the user U1 is holding the smartphone 100. Therefore, the user U1 can talk using the audio sensor (microphone) 171 built into the smartphone 100 and hear the other party's voice reproduced by the audio output element 252 of the earphone 200 through the earphone 200. Of course, even in this case, it goes without saying that the user U1 may make a hands-free call using the audio sensor (microphone) 251 and audio output element 252 of the earphone 200.

[0114] Next, in step S412, it is determined whether an end operation has been performed. If it is determined in the determination process of step S412 that an end operation has not been performed (step S412: No), the process returns to the process of step S411, and transmission and reception of call audio data between the smartphone 100 and the earphone 200 continues.

[0115] Furthermore, if it is determined in the determination process of step S412 that an end operation has been performed (step S412: Yes), that is, if it is determined that the call has been hung up, the call process is terminated. If music playback is paused when smartphone 100 receives an incoming call, the paused music playback may be restored to the same state as before the incoming call when the call process is terminated.

[0116] The call termination operation is, for example, an operation by the user U1 to end a call, and is performed during a call using the operation input device 260 of the earphone 200 or the operation input device 180 of the smartphone 100. In addition, in the determination process of step S412, it is also determined that a call termination operation has been performed when the other party hangs up the phone.

[0117] As described above, in the voice processing system 10 of this embodiment, when a call comes in, the user U1 wearing the earphones 200 is notified of the call by a 3D ringtone with the smartphone 100 as the sound source. This makes it easier for the user U1 to identify the location of the smartphone 100 and to easily search for the smartphone 100.

[0118] Accordingly, when a call is received, the user U1 can easily make a call after checking the caller information displayed on the smartphone 100. Furthermore, by finding the smartphone 100, the user U1 can use the audio sensor (microphone) 171 of the smartphone 100 when making a call. This also has the effect of making it easier to use hearable devices such as earphones that do not have an audio sensor.

[0119] <Modification of First Embodiment> In the first embodiment, when the smartphone 100 receives an incoming call, a 3D ringtone is output from the earphones 200. However, in addition to outputting a 3D ringtone, the vibrator 280 of the earphones 200 may also be vibrated. This makes it possible to notify the user U1 of the incoming call through their sense of touch in addition to their hearing. Furthermore, the magnitude of the vibration of the vibrator 280 may be changed between the left earphone 201 and the right earphone 202 depending on the positional relationship between the smartphone 100 and the left earphone 201 and the right earphone 202. This makes it possible to more clearly indicate to the user U1 the direction of the smartphone 100.

[0120] The audio processing system 10 of this embodiment employs a so-called left-right simultaneous transmission method in which the left earphone 201 and the right earphone 202 are treated as independent devices and audio data (ringtone data) is transmitted from the smartphone 100 to each of the left earphone 201 and the right earphone 202. However, the audio data transmission method is not particularly limited. The audio data transmission method may be, for example, a relay transmission method in which one of the left earphone 201 or the right earphone 202 is treated as a single stereo device and audio data (ringtone data) is relayed to the other earphone wirelessly or via a wired connection.

[0121] In this embodiment, as an example, an example has been described in which the relative positions of the smartphone 100 and the earphones 200 are calculated when the smartphone 100 is maintained in a horizontal orientation (see Figure 11), but a similar effect can be obtained even when the smartphone 100 is maintained in any orientation, such as a vertical orientation.

[0122] In the voice processing system 10 of this embodiment, the relative positions of the smartphone 100 and the earphones 200 are calculated using Bluetooth communication, but by also using the group of sensors that make up the sensor device 140, the accuracy of calculating the relative positions of the smartphone 100 and the earphones 200 can be further improved.

[0123] In the present embodiment, a configuration including the smartphone 100 and the earphones 200 has been exemplified as the audio processing system 10, but the configuration of the audio processing system 10 is not limited to this. For example, the audio processing system 10 may include a charging case 300 in addition to the smartphone 100 and the earphones 200. In this case, the charging case 300 includes a Bluetooth communication device and an antenna, although not shown.

[0124] Then, when calculating the relative position (first relative position data) between the smartphone 100 and the earphones 200, second relative position data is calculated that specifies the relative position between the charging case 300 and at least one of the smartphone 100 and the earphones 200. In other words, the relative position (first relative position data) between the smartphone 100 and the earphones 200 is calculated based on the relative position (second relative position data) between the smartphone 100 and the earphones 200. In other words, the antenna of the charging case 300 is used as a fourth antenna linked to the left antenna 104, right antenna 105, and center antenna 106 of the smartphone 100 to calculate the first relative position data.

[0125] For example, by determining the distances between the left antenna 104, right antenna 105, and center antenna 106 of the smartphone 100 and the antenna of the charging case 300, it is possible to further improve the accuracy of detecting the direction of the earphones 200 using the AoA (Angel of Arrival) method. As a result, it is possible to more accurately identify the relative positions of the smartphone 100 and the earphones 200.

[0126] As described above, in the voice processing system 10 of this embodiment, when the smartphone 100 receives an incoming call, the ringtone data is unconditionally converted into 3D ringtone data. However, the user U1 may be allowed to select whether to convert the ringtone data into 3D ringtone data or into conventional monaural sound data or stereo sound data. In other words, the user U1 may be allowed to arbitrarily select the method for converting the ringtone data.

[0127] For example, a plurality of selection images (menu images) corresponding to each method of converting ringtone data are displayed on the display 162 of the smartphone 100, which is configured with a touch panel or the like, and the user U1 can select any conversion method via the operation input device 180 by touching one of the selection images.

[0128] In addition, if the method of converting the ringtone data into 3D ringtone data is not adopted by user U1, the processing of steps S403 to S405 in the flowchart shown in Figure 6 is not performed, and in step S406, the conventional ringtone data is sent from smartphone 100 to earphone 200.

[0129] In the present embodiment, the 3D ringtone data is faithfully generated based on the first relative position data so that a 3D ringtone with the smartphone 100 as the sound source is output from the earphones 200, but the 3D ringtone data does not necessarily have to be faithfully generated based on the first relative position data. For example, the 3D ringtone does not have to change linearly depending on the relative positions of the smartphone 100 and the earphones 200, and may change depending on a specific positional relationship between the smartphone 100 and the earphones 200. In other words, the volume and balance of the left earphone 201 and the right earphone 202 may be changed so as to deform the relative distance and direction between the earphones 200 and the smartphone 100.

[0130] However, since excessive deformation may cause the volume to become too loud and difficult to hear, it is preferable to suppress deformation below a predetermined volume level. Similarly, since the volume may become too quiet below a predetermined volume level and become inaudible to the user U1, it is preferable to suppress deformation below a predetermined volume level. In other words, when changing the volume or balance of the left earphone 201 and the right earphone 202, it is preferable to set upper and lower limits.

[0131] The interval between intermittent telephone ringtones and the frequency (which can also be considered sound quality) of the telephone ringtones may be changed according to the relative distance between the smartphone 100 and the earphones 200. As an example, the shorter the relative distance between the smartphone 100 and the earphones 200, the narrower the interval between intermittent telephone ringtones may be, or the telephone ringtone may be changed to a higher pitch. Furthermore, for example, 3D ringtone data may be generated so as to emphasize the Doppler effect, which reflects the rate of change in the relative distance between the smartphone 100 and the earphones 200.

[0132] By adopting such an emphasis method and adding variation to the 3D ringtone output from the earphone 200, it becomes easier for the user U1 to intuitively grasp the difference in the relative distance between the earphone 200 and the smartphone 100.

[0133] Furthermore, for example, the camera (imaging sensor) 161 or the light sensor 145 of the smartphone 100 may detect the brightness around the smartphone 100, and if it is determined that the smartphone 100 is blocked by an obstacle, such as if the smartphone 100 is in a desk drawer, the sound quality of the 3D ringtone output from the earphones 200 may be changed to take the obstacle into consideration. That is, the ringtone data may be converted into different 3D sound data when it is determined that the smartphone 100 is not blocked and when it is determined that the smartphone 100 is blocked. For example, by converting the ringtone data so that a muffled 3D ringtone is output from the earphones 200, the user U1 can understand that the smartphone 100 is blocked by an obstacle, making it easier for the user U1 to find the smartphone 100.

[0134] In this embodiment, when the smartphone 100 receives an incoming call, 3D ringtone data is transmitted from the smartphone 100 to the earphones 200. However, instead of the 3D ringtone data, a voice message indicating the location of the smartphone 100 may be transmitted. That is, when the smartphone 100 receives an incoming call, a voice message indicating the location of the smartphone 100 may be output from the earphones 200 along with a normal ringtone. As an example, a voice message such as "2 meters ahead in the 11 o'clock direction" may be output from the earphones 200, expressing the position from the user U1 as the starting point.

[0135] Even in this case, the user U1 can know the location of the smartphone 100 from the voice message, and can easily search for the smartphone 100. Of course, it goes without saying that the 3D ringtone and the voice message can be used together.

[0136] In addition, in the present embodiment, a case where the smartphone 100 receives an incoming call has been described, but the timing at which the technology of the present disclosure can be applied is not limited to when a call is received. It goes without saying that the technology of the present disclosure can be applied at any timing when the user U1 needs the smartphone 100.

[0137] As an example, the user U1 operates (e.g., presses and holds) the multi-function button of the operation input device 260 of the earphones 200, thereby setting a search mode for searching for the smartphone 100. When the search mode is set, the earphones 200 transmit a signal to the smartphone 100 indicating that the search mode has been set. When the smartphone 100 receives this signal, 3D sound data representing the sound source position is transmitted from the smartphone 100 to the earphones 200, and 3D sound is output from the earphones 200. As a result, the user U1 can easily search for the smartphone 100. It goes without saying that the 3D sound output from the earphones 200 in the search mode may be different from the 3D ringtone output when a normal phone call is received.

[0138] It is preferable that the selection of the emphasis method for the 3D sound (3D ringtone), the magnitude of the emphasis effect, etc. can be adjusted by the user U1 as appropriate using the display 162 (touch panel, etc.) of the smartphone 100. For example, it is preferable that the selection of the collaboration method, the magnitude of the collaboration effect, etc. be stored in the storage device 130 as user setting information, and that this user setting information can be changed as appropriate on the display 162 of the smartphone 100. An example of user setting information is described below.

[0139] <Example of User Setting Information> In this embodiment, device information and user setting information of the earphone 200 are stored in advance in the storage device 130 of the smartphone 100. Fig. 12 is a diagram showing an example of an information table including device information and user setting information stored in the storage device of the smartphone.

[0140] 12 is composed of information items 601 and information content corresponding to the information items 601. The information content is information content related to each hearable device connected via Bluetooth to the smartphone 100. In the example shown in FIG. 12 , the information content includes information content 602 related to the left earphone 201 and information content 603 related to the right earphone 202.

[0141] The information items 601 are categorized into device information items 610 and user setting information items 620. The device information items 610 include a device name item 611 and multiple device position information items. As an example, the device position information items include three pieces of information: a distance (r) item 612 from the smartphone 100 to each earphone 200, an up-down angle (Θ) item 613 relative to the position of each earphone 200, and a left-right angle (Φ) item 614 relative to the position of each earphone 200. The device position information items may also include other information items related to each earphone 200 as necessary.

[0142] Three-dimensional position coordinates with the position of the central antenna 106 of the smartphone 100 as the origin are obtained based on the device position information including the distance item 612, the up-down angle item 613, and the left-right angle item 614. From these three-dimensional position coordinates, the relative positions (first relative position data) between the earphone 200 and the smartphone 100 can also be calculated.

[0143] The user setting information items 620 in the upper table T600 include a plurality of information items set by the user U1. The user setting information items 620 are displayed on the display 162 of the smartphone 100 and are set by the user U1 using the operation input device 180.

[0144] The user setting information item 620 includes, for example, the following items: The 3D sound processing item 621 is an item for setting whether or not to set the telephone ringtone as a 3D ringtone. When this item is "ON," the telephone ringtone is set to a 3D ringtone, and when this item is "OFF," the telephone ringtone is not set to a 3D ringtone (not set).

[0145] The ringtone intermittent interval change item 622 is an item for setting whether to change the ringtone intermittent interval according to the relative distance between the smartphone 100 and the earphone 200. When this item is "ON," the shorter the relative distance, the shorter the intermittent interval is set; when this item is "OFF," the setting is such that the ringtone intermittent interval is not changed (not set). In practice, the ringtone intermittent interval is changed based on a correspondence table or formula between the relative distance and the intermittent interval that is separately created.

[0146] The ringtone frequency change item 623 is an item for setting whether or not to change the frequency of the ringtone according to the relative distance between the smartphone 100 and the earphone 200. When this item is "ON," the ringtone frequency is set to increase as the relative distance decreases, and when this item is "OFF," the ringtone frequency is set not to change (not set). In practice, the ringtone frequency is changed based on a separately created correspondence table or formula between the relative distance and the frequency increase rate (or decrease rate).

[0147] The 3D ringtone deformation strength item 624 is an item for setting the deformation strength of the ringtone. In this example, the 3D ringtone deformation strength item 624 is set to one of three levels: "strong," "weak," and "OFF (not set)." Note that the 3D ringtone deformation strength item 624 may be set using a quantitative or intuitive number such as ±%.

[0148] The Doppler effect strength item 625 is an item for setting the Doppler effect strength of the ringtone. In this example, the Doppler effect strength item 625 is set to one of three levels: "strong," "weak," and "OFF (not set)." The Doppler effect strength item 625 may be set quantitatively, such as ±%, or as an intuitive number.

[0149] The occlusion response item 626 is an item for setting whether to take a predetermined response when the smartphone 100 is determined to be occluded by the light sensor 145 of the smartphone 100. When this item is "ON", the predetermined response is taken, and when this item is "OFF", the predetermined response is not taken (not set).

[0150] The voice message item 627 is an item for setting whether to notify user U1 of an incoming call with a voice message. In this example, the voice message item 627 can be set to one of three types: "single," "combined," and "OFF (not set)." When this item is set to "single," the setting is to notify user U1 of an incoming call with only a voice message, and when this item is set to "combined," the setting is to notify user U1 of an incoming call with both a ringtone and a voice message. When this item is set to "OFF," the setting is to notify user U1 of an incoming call with only a ringtone.

[0151] The vibrator item 628 is an item for setting whether or not to vibrate the vibrator 190 of the smartphone 100 when the orientation of the user U1 is toward the smartphone 100. When this item is "ON", the vibrator 190 is set to vibrate when the orientation of the user U1 is toward the smartphone 100, and when this item is "OFF", the vibrator 190 is set not to vibrate (not set).

[0152] The light item 629 is an item for setting whether to turn on (or blink) the light 195 of the smartphone 100 when the user U1 is facing the smartphone 100. In this example, the light item 629 is set to one of three types: "on," "blinking," and "off (not set)."

[0153] The earphone vibrator item 630 is an item for setting whether or not to vibrate the vibrator 280 of the earphone 200 when a call is received on the smartphone 100. When this item is "ON", the vibrator 280 of the earphone 200 is set to vibrate when a call is received on the smartphone 100, and when this item is "OFF", the vibrator 280 is set not to vibrate (not set).

[0154] The earphone vibrator change intensity item 631 is an item for setting whether to change the vibration intensity of the left earphone 201 and the right earphone 202 depending on the position of the smartphone 100. When this item is "ON", the setting is to change the vibration intensity of the vibrator 280, and when this item is "OFF", the setting is to not change the vibration intensity of the vibrator 280 (not set).

[0155] It should be noted that additional settings can be made as needed in the user setting information item 620. For example, items for setting ON / OFF of the 3D mail ringtone, ON / OFF of the 3D alarm notification sound, etc. can be added.

[0156] In addition, this information table T600 can be displayed on the display 162 of the smartphone 100 as described above and presented to the user U1. Furthermore, this information table T600 can also be stored in the network server 500 via the network 550. This completes the description of the information table T600 shown in FIG. 12.

[0157] In the first embodiment, a case is assumed in which a user U1 wearing the earphones 200 searches for the smartphone 100, but the technology of the present disclosure can also be applied to cases other than searching for the smartphone 100. As an application example, in a configuration in which the audio processing system 10 includes the smartphone 100, the earphones 200, and the charging case 300, a case is assumed in which the user U1 wearing the earphones 200 needs to search for the charging case 300. For example, the technology of the present disclosure can also be applied in a case in which the charging case 300 is not within the user U1's line of sight, the charging capacity of the earphones 200 is running low, and the user U1 wants to search for the charging case 300.

[0158] In this case, the charging case 300 is equipped with a Bluetooth communication device as described above, and the smartphone 100 can calculate the antenna position of the charging case 300 by utilizing the three antennas 104, 105, and 106 of the smartphone 100.

[0159] As a specific example of an operation to search for the charging case 300, the user U1 operates (e.g., presses multiple times) the multi-function button of the operation input device 260 of the earphones 200. In response to this operation, the earphones 200 notify the smartphone 100 that a mode for searching for the charging case 300 has been set.

[0160] Then, the smartphone 100 that has received the notification calculates the relative position of the charging case 300 with respect to the smartphone 100, and outputs a 3D notification sound from the earphones 200, with the charging case 300 as the sound source position, according to the calculated relative position. As a result, even if the charging case 300 is located out of the user U1's line of sight, the user U1 can easily find the charging case 300. Note that the 3D notification sound at this time may be the same as or different from the 3D ringtone when a phone call is received.

[0161] Furthermore, in this case, the smartphone 100 can grasp both the relative position of the earphone 200 and the relative position of the charging case 300, and can therefore also grasp the relative positions of the earphone 200 and the charging case 300. Therefore, for example, when the smartphone 100 determines that the distance between the earphone 200 and the charging case 300 is equal to or greater than a certain distance, a 3D notification sound that is different from the 3D ringtone that is generated when a call is received may be output from the earphone 200. The user U1 can recognize that the distance between the earphone 200 and the charging case 300 is equal to or greater than a certain distance by the generation of a 3D notification sound that is different from the 3D ringtone that is generated when a call is received, without having to operate the multi-function button of the operation input device 260 of the earphone 200.

[0162] (Embodiment 2) Fig. 13 is a diagram showing an example of the internal configuration of a smartphone included in a voice processing system of embodiment 2. Fig. 14 is a diagram showing an example of the internal configuration of earphones included in the voice processing system of the embodiment. Fig. 15 is a diagram showing an example of the external appearance of a smartphone included in a voice processing system according to embodiment 2.

[0163] The audio processing system 10 of the first embodiment uses Bluetooth communication to calculate first relative position data that identifies the relative positions between the smartphone 100 and the earphones 200. In contrast, the audio processing system 10 of the second embodiment uses ultra-wide band (UWB) wireless communication to calculate the first relative position that identifies the relative positions between the smartphone 100 and the earphones 200. Note that position calculation using UWB wireless communication is generally said to be more accurate than position calculation using Bluetooth communication.

[0164] The basic hardware and software configurations of the devices constituting the speech processing system 10 of the second embodiment are the same as those of the first embodiment. Therefore, the following description will focus on the differences between the second embodiment and the first embodiment, and overlapping descriptions will be omitted.

[0165] As shown in Fig. 13 , the smartphone 100 according to the second embodiment includes, in addition to the configuration of the first embodiment, a UWB communicator 154 that constitutes a communication device 150. Also, as shown in Fig. 14 , the earphone 200 according to the second embodiment includes, in addition to the configuration of the first embodiment, a UWB communicator 242 that constitutes a communication device 240.

[0166] 15 , the smartphone 100 according to the second embodiment has a left UWB antenna 107, a right UWB antenna 108, and a center UWB antenna 109 as UWB antennas required for UWB wireless communication. The left UWB antenna 107 is provided at the leftmost end of one side of the smartphone 100, and the right UWB antenna 108 is provided at the rightmost end of one side of the smartphone 100. The center UWB antenna 109 is provided at the center position (the middle position between the left UWB antenna 107 and the right UWB antenna 108) of the other side of the smartphone 100 (the side opposite the left UWB antenna 107 and the right UWB antenna 108).

[0167] 16 and 17 are flowcharts illustrating a voice processing procedure in a smartphone constituting a voice processing system according to embodiment 2. The processing procedures shown in Fig. 16 and Fig. 17 will be described with reference to the system configuration diagram in Fig. 13 .

[0168] Fig. 16 is a flowchart showing a voice processing procedure when answering a call in a smartphone according to embodiment 2. The voice processing procedure (voice processing method) of embodiment 2 differs from the voice processing procedure of embodiment 1 (see Fig. 6) in that UWB preprocessing (step S450) is executed, as shown in Fig. 16. Specifically, after Bluetooth preprocessing is completed in step S401, UWB preprocessing is then executed in step S450.

[0169] 17 is a flowchart showing an example of the processing procedure for UWB preprocessing (step S450), which is an added subroutine. When UWB preprocessing starts in step S450, as shown in FIG. 17, first, in step S451, it is determined whether a UWB polling signal has been received from the earphone 200, which is a UWB device. The UWB polling signal from the earphone 200 contains the UWB device ID of the earphone 200. When the smartphone 100 receives a UWB polling signal from the earphone 200, it can obtain the UWB device ID information of the earphone 200.

[0170] If it is determined in the determination process of step S451 that a UWB polling signal has not been received from the earphone 200 (step S451: No), the process proceeds to step S460, which will be described later, and a timeout determination process is executed.

[0171] If it is determined in step S451 that a UWB polling signal has been received from the earphone 200 (step S451: Yes), the process proceeds to step S452, which determines whether the acquired UWB device ID of the earphone 200 is new.

[0172] If it is determined in step S452 that the acquired UWB device ID of the earphone 200 is not new (step S452: No), a timeout determination process is executed in step S460. If it is determined in step S452 that the acquired UWB device ID of the earphone 200 is new (step S452: Yes), a UWB device information storage process is executed in step S453.

[0173] The UWB device information saving process in step S453 is a process of saving the acquired UWB device information in the storage device 130. In the UWB device information saving process in this embodiment, for example, the UWB device ID acquired from the UWB polling signal of the received device, the reception time of the UWB polling signal, etc. are saved in the storage device 130 as the UWB device information.

[0174] Next, in step S454, a UWB response transmission process is performed to transmit a UWB response to the earphone 200. Next, in step S455, a UWB response transmission time storage process is performed, and the UWB response transmission time, which is the transmission time of the response transmitted to the earphone 200, is stored in storage device 130.

[0175] Next, in step S456, it is determined whether or not a UWB message signal has been received from the earphone 200. If it is determined in the determination process of step S456 that a UWB message signal has not been received from the earphone 200 (step S456: No), a timeout determination process of step S457 is executed.

[0176] If it is determined in the timeout determination process of step S457 that the time waiting for reception of a UWB message signal has not reached the predetermined time and therefore no timeout has occurred (step S457: No), the process returns to step S456, and the UWB message reception determination process is performed again. On the other hand, if it is determined in the timeout determination process of step S457 that the time waiting for reception of a UWB message signal has reached the predetermined time and therefore a timeout has occurred (step S457: Yes), the process proceeds to the timeout determination process of step S460. Note that the timeout determination process of step S456 is rarely necessary (at most several tens of seconds) because it simply waits for reception of a UWB message signal, but it is performed to avoid system deadlock.

[0177] Furthermore, if it is determined in the determination process of step S456 that a UWB message signal has been received from earphone 200 (step S456: Yes), a process of saving the UWB message reception time and UWB device information is executed in step S458. The saving process of step S458 is a process of saving the reception time of the UWB message signal received by smartphone 100 and the UWB device information acquired from the UWB message signal in storage device 130. The UWB device information includes, for example, the UWB polling transmission time, the UWB response reception time, and the UWB message transmission time.

[0178] Next, in step S459, the external device distance is calculated and saved, as will be described in detail later. That is, the relative distance between the smartphone 100 and the earphones 200 (the relative distance to the device) is calculated, and the calculated relative distance is saved in the storage device 130. Thereafter, a timeout determination process is executed in step S460.

[0179] If it is determined in the timeout determination process of step S460 that the timeout has not occurred (step S460: No), the process returns to the UWB polling reception determination process of step S451. On the other hand, if it is determined in the timeout determination process of step S460 that the timeout has occurred (step S460: Yes), the UWB pre-processing (step S450) is terminated.

[0180] In the timeout determination process of step S460, the time until a timeout is determined is set to be sufficiently longer than the generation period of the UWB polling signal. Of course, the UWB preprocessing can also be terminated at any time by the user U1 or the like. This concludes the description of the processing procedure of the UWB preprocessing (step S450) in the smartphone 100 according to the second embodiment.

[0181] Next, a description will be given of the UWB processing procedure in the earphone 200 according to embodiment 2. Fig. 18 is a flowchart showing an example of the UWB processing procedure in the earphone according to embodiment 2. The UWB processing procedure shown in Fig. 18 will be described with reference to the system configuration diagram in Fig. 14.

[0182] When the above-described UWB preprocessing (step S450) in the smartphone 100 starts, first, in step S471, a UWB polling signal is transmitted by broadcast communication in the earphone 200. At that time, the time when the UWB polling signal was transmitted (UWB polling transmission time) is stored in the storage device 230 (step S472).

[0183] Next, in step S473, it is determined whether a UWB response signal has been received from smartphone 100. If it is determined in the determination process of step S473 that a UWB response signal has not been received from smartphone 100 (step S473: No), a determination process of whether an end instruction has been given in step S477, which will be described later, is executed. On the other hand, if it is determined in the determination process of step S473 that a UWB response signal has been received from smartphone 100 (step S473: Yes), the time at which the UWB response signal was received (UWB response reception time) is saved in storage device 230 in step S474.

[0184] Next, in step S475, the time at which the UWB message signal is transmitted (UWB message transmission time) is stored in storage device 230. Subsequently, in step S476, the UWB message signal is transmitted.

[0185] Then, the process proceeds to step S477, where it is determined whether an end instruction has been issued. If it is determined in step S477 that an end instruction has not been issued (step S477: No), the process returns to step S471, and the UWB polling transmission process is executed again. If it is determined in step S477 that an end instruction has been issued (step S477: Yes), the UWB connection process for this device (earphone 200) ends.

[0186] In other words, the UWB polling transmission process of step S471 is performed at regular intervals until it is determined in step S477 that an end instruction has been received. Note that the process of determining whether an end instruction has been received in step S477 is added for convenience and is, of course, based on the device specifications. This concludes the description of the UWB processing procedure in the earphone 200 according to embodiment 2.

[0187] The UWB pre-processing procedure in the smartphone 100 (see FIG. 17 ) and the UWB processing procedure in the earphone 200 (see FIG. 18 ) are closely related. For example, UWB polling reception determination processing (step S451) and UWB message reception determination processing (step S456) in the smartphone 100 correspond to the UWB polling transmission processing (step S471) and UWB message transmission processing (step S476) in the earphone 200, respectively. Similarly, UWB response reception determination processing (step S473) in the earphone 200 corresponds to the UWB response transmission processing (step S454) in the smartphone 100.

[0188] Here, the distance D from the smartphone 100 to the earphone 200 can be calculated based on the arrival time of the UWB signal. In the audio processing system 10 of the second embodiment, the distance D is calculated based on this distance D.

[0189] In detail, the arrival time T of the UW signal can be calculated using the following formula, taking into account that the time system of the smartphone 100 and the time system of the earphone 200 are not synchronized (they are not the same time system).

[0190] T={(Trr-Tsp)-(Tsr-Trp)+ (Trf-Tsr)-(Tsf-Trr)} / 4...(1)

[0191] In the above formula (1), the UWB response reception time is represented as Trr, the UWB polling transmission time is represented as Tsp, the UWB response transmission time is represented as Tsr, the UWB polling reception time is represented as Trp, the UWB message reception time is represented as Trf, and the UWB message transmission time is represented as Tsf.

[0192] If the arrival time T of the UWB signal can be calculated, the distance D from the smartphone 100 to the earphone 200 can be calculated using the following equation (2), since the propagation speed of the UWB signal is the speed of light c.

[0193] D=T×c…(2)

[0194] More specifically, the distance D from the smartphone 100 to the earphone 200 calculated by the above formula (2) is the distance from the UWB antenna included in the smartphone 100 to the earphone 200. The smartphone 100 according to the second embodiment has three UWB antennas (left UWB antenna 107, right UWB antenna 108, and center UWB antenna 109), and therefore the distance from each of these UWB antennas to the earphone 200 is calculated as the distance D by the above formula (2).

[0195] Next, the calculation principle for calculating the relative position (first relative position data) of the earphone 200 with respect to the smartphone 100 from the distance D from each UWB antenna provided in the smartphone 100 to the earphone 200 will be described.

[0196] FIG. 19 is a schematic diagram showing the calculation principle for calculating the relative position (first relative position data) between the smartphone and the earphone in the audio processing system according to the second embodiment.

[0197] As described above, the smartphone 100 according to the second embodiment has the left UWB antenna 107, the right UWB antenna 108, and the center UWB antenna 109 (see FIG. 15 ). The center UWB antenna 109 is disposed at the midpoint between the opposite sides of the left UWB antenna 107 and the right UWB antenna 108. Therefore, as shown in FIG. 19 , when the position Pa of the center UWB antenna 109 is taken as the origin, the distance from the position Pa of the center UWB antenna 109 to the position Pb of the left UWB antenna 107 is equal to the distance from the position Pa of the center UWB antenna 109 to the position Pc of the right UWB antenna 108.

[0198] Then, when a circle 702 with a radius equal to the linear distance 701 from position Pc of the right UWB antenna 108 to the earphone 200, a circle 712 with a radius equal to the linear distance 711 from position Pa of the central UWB antenna 109 to the earphone 200, and a circle 722 with a radius equal to the linear distance 721 from the left UWB antenna 107 to the earphone 200 are drawn, the point (intersection) Pd where these three circles 702, 712, and 722 intersect is the position of the earphone 200 (specifically, the position of the UWB antenna of the earphone 200). In the example shown in Fig. 19, position Pd of the earphone 200 is a distance D1 to the left of position Pa of the central UWB antenna 109 and a distance D2 upward in the figure.

[0199] Furthermore, the angle θ1 of the position Pd of the earphone 200 relative to the position Pa of the central UWB antenna 109, that is, the angle θ1 formed by the line L11 connecting the positions Pa and Pd in ​​FIG. 19 and the line L12 in the left-right direction in the figure, can be calculated using the following formula.

[0200] θ1=arctan(D2 / D1)...(3)

[0201] Using the above calculation principle, it is possible to calculate the relative position of the earphone 200 with respect to the smartphone 100. In other words, using the above calculation principle, it is possible to calculate first relative position data that specifies the relative positions of the smartphone 100 and the earphone 200. The obtained information on the relative position of the earphone 200 with respect to the smartphone 100 (first relative position data) is saved in the storage device 130 of the smartphone 100.

[0202] Returning to the explanation of the flowchart in Figure 16, when the UWB pre-processing subroutine ends in step S450, the processes of steps S402 to S412 are then executed, as in embodiment 1. However, in step S403, the relative position of the earphone 200 is calculated based on the calculation principle described above (see Figure 19).

[0203] It goes without saying that the protocol related to UWB is not limited to the protocol described in embodiment 2, and other protocols may also be used. Also, in embodiment 2, if the relative positions of the smartphone 100 and the earphone 200 cannot be clearly determined, for example, if the three circles 702, 712, and 722 shown in FIG. 19 do not intersect at a single point, the vicinity of the center of multiple intersections (for example, three points) of these three circles 702, 712, and 722 is set as position Pd of the earphone 200, and the relative position of the smartphone 100 and the earphone 200 is calculated.

[0204] In addition, in this embodiment, the relative position between the smartphone 100 and the earphone 200 is calculated using UWB wireless communication, but it goes without saying that the accuracy of calculating the relative position can be improved by using, for example, the sensor group of the sensor device 140 of the smartphone 100 in combination.

[0205] By equipping the charging case 300 with a UWB communication device, the UWB antenna of the charging case 300 can also be used as a fourth UWB antenna linked to the three UWB antennas of the smartphone 100 (left UWB antenna 107, right UWB antenna 108, and center UWB antenna 109).

[0206] Furthermore, by knowing the distance between the UWB antennas (107, 108, 109) of the smartphone 100 and the UWB antenna of the charging case 300, it is possible to further improve the accuracy of detecting the direction and distance of the earphone 200. As a result, it is possible to more accurately calculate the relative position of the earphone 200 with respect to the smartphone 100 (first relative position data).

[0207] In the second embodiment described above, when a call comes in, a 3D ringtone with the smartphone 100 as the sound source is presented to the user U1 wearing the earphones 200. Therefore, even if the smartphone 100 is located out of the user U1's line of sight, the user U1 can easily find the smartphone 100.

[0208] Furthermore, while Bluetooth communication is used in the first embodiment and UWB wireless communication is used in the second embodiment to calculate the relative positions of the smartphone 100 and the earphones 200, both may be used in combination. This increases the reliability of the relative positions of the smartphone 100 and the earphones 200. Furthermore, other short-range wireless communication technologies or Wi-Fi technology may be used to calculate the relative positions of the smartphone 100 and the earphones 200, for example, or position search using ultrasound, millimeter waves, gamma rays, or the like may be used. If the smartphone 100 and the earphones 200 each have a positioning device such as a GPS or gyro, they may exchange positioning results and calculate the relative positions of the smartphone 100 and the earphones 200 from the difference between the results.

[0209] Third Embodiment In the above-described embodiment, the relative position (first relative position data) between the smartphone 100 and the earphone 200 is calculated in the smartphone 100. In contrast, a third embodiment is an example in which the relative position (first relative position data) between the smartphone 100 and the earphone 200 is calculated in the earphone 200.

[0210] The basic hardware and software configurations of each device constituting the voice processing system 10 of embodiment 3 are the same as those of embodiment 1, and the following mainly describes the differences between embodiment 3 and embodiment 1, and omits redundant explanations.

[0211] Fig. 20 is a diagram showing an example of the internal configuration of an earphone constituting the audio processing system of embodiment 3. As shown in Fig. 20, the earphone 200 according to embodiment 3 includes a sensor device 290 in addition to the configuration of embodiment 1.

[0212] The sensor device 290 follows the group of sensors (141 to 145) in the sensor device 140 of the smartphone 100 according to embodiment 1. The group of sensors in the sensor device 290 added to the earphone 200 is a group of various sensors for detecting the state of the earphone 200. Specifically, the sensor device 290 is composed of a group of sensors including a positioning sensor 291, a geomagnetic sensor 292, an acceleration sensor 293, a gyro sensor 294, and a light sensor 295 that detects the light and darkness around the earphone 200.

[0213] The sensor device 290, which is composed of a group of these sensors, can detect the position, tilt, direction, movement, etc. of the earphone 200. Therefore, when calculating the relative position between the earphone 200 and the smartphone 100 via Bluetooth communication, the detection results by the sensor device 290 can be used in addition to or instead of the results.

[0214] In the third embodiment, the earphone 200 is provided with a plurality of (for example, three) Bluetooth antennas in a Bluetooth communication device 241 so that the earphone 200 can calculate the relative position between the earphone 200 and the smartphone 100. Note that the connection procedure for Bluetooth communication and the method for calculating the relative position between the earphone 200 and the smartphone 100 via Bluetooth communication are explained in the first embodiment, and therefore will not be explained here.

[0215] In addition to the calculation results of the relative positions between the earphones 200 and the smartphone 100 calculated by the positioning sensor 291 and the Bluetooth communication device 241, the head movements of the user U1 are detected by the acceleration sensor 293 and the gyro sensor 294 and the detection results are taken into consideration, thereby improving the calculation accuracy and responsiveness of the relative positions between the earphones 200 and the smartphone 100.

[0216] In embodiment 3, the audio processor 253 of the earphone 200 generates 3D ringtone data with the smartphone 100 as the sound source position based on the relative position between the earphone 200 and the smartphone 100 calculated using the Bluetooth communication of the earphone 200.

[0217] The left earphone 201 and the right earphone 202 each have the configuration of the earphone 200 shown in Fig. 20. The left earphone 201 generates 3D ringtone data for the left ear and plays back the 3D ringtone for the left ear. The right earphone 202 generates 3D ringtone data for the right ear and plays back the 3D ringtone for the right ear. As a result, a 3D ringtone whose sound source is the smartphone 100 can be presented to the user U1 wearing the earphone 200.

[0218] In this way, in embodiment 3, the left earphone 201 and the right earphone 202 are treated as independent devices, and ringtone data is sent directly from the smartphone 100 to each of the left earphone 201 and the right earphone 202, a so-called simultaneous left and right transmission method is adopted.

[0219] However, the method for transmitting the audio data (ringtone data) is not particularly limited. For example, the audio data may be transmitted by a relay transmission method in which 3D ringtone data is generated by either the left earphone 201 or the right earphone 202 and the 3D ringtone data (audio data) is relayed wirelessly or via a wired connection from one earphone to the other. In this case, the earphone 200 that generates the 3D ringtone data has the configuration shown in FIG. 20 .

[0220] Also in the third embodiment, the charging case 300 may be provided with a Bluetooth communication device, and the Bluetooth antenna in the charging case 300 may be used in conjunction with multiple Bluetooth antennas in the earphones 200.

[0221] For example, it is possible to improve the accuracy of detecting the direction of the smartphone 100 using AoA (Angel of Arrival) by determining the distance between the Bluetooth antenna of the earphone 200 and the Bluetooth antenna of the charging case 300. Furthermore, for example, when the distance between the charging case 300 and the earphone 200 is equal to or greater than a predetermined distance, the earphone 200 may emit a 3D warning sound whose sound source is the charging case 300, instead of a 3D ringtone whose sound source is the smartphone 100.

[0222] In addition, in the third embodiment, the left earphone 201 and the right earphone 202 are treated as independent devices, and each of the left earphone 201 and the right earphone 202 is equipped with multiple (for example, three) Bluetooth communication antennas necessary to calculate the relative positions of the earphone 200 and the smartphone 100.

[0223] However, multiple Bluetooth communication antennas may be provided separately for the left earphone 201 and the right earphone 202. For example, one Bluetooth communication antenna may be provided for the left earphone 201, and two Bluetooth communication antennas may be provided for the right earphone 202. In this case, by setting the distance between the two ears in advance, it is possible to improve the accuracy of detecting the direction of the smartphone 100 using AoA (Angel of Arrival).

[0224] Furthermore, in the third embodiment, the earphone 200 calculates the relative position between the earphone 200 and the smartphone 100 (first relative position data), and generates 3D ringtone data with the smartphone 100 as the sound source position based on the first relative position data. However, the 3D ringtone data may be generated on the smartphone 100 side. For example, the relative position between the earphone 200 and the smartphone 100 (first relative position data) calculated by the earphone 200 may be transmitted to the smartphone 100, and the 3D ringtone data may be generated by the audio processor 173 of the smartphone 100. The generated 3D ringtone data is transmitted from the smartphone 100 to the earphone 200 via Bluetooth communication or the like, as in the above-described embodiments. Similarly, relative position information between the earphone 200 and the smartphone 100 calculated by the smartphone 100 may be transmitted to the earphone 200, and the 3D ringtone data may be generated by the audio processor 253 of the earphone 200, so that the smartphone 100 and the earphone 200 can perform distributed processing as appropriate.

[0225] (Embodiment 4) Embodiment 4 is a modification of embodiment 3. In embodiment 3, the relative position of the earphone 200 with respect to the smartphone 100 is calculated using Bluetooth communication. In contrast, in embodiment 4, the relative position of the smartphone 100 with respect to the earphone 200 is calculated using ultra-wide band (UWB) wireless communication.

[0226] The basic hardware and software configurations of the devices constituting the voice processing system of embodiment 4 are the same as those of embodiment 3. The following mainly describes the differences between embodiment 4 and embodiment 3, and omits redundant explanations.

[0227] Fig. 21 is a diagram showing an example of the internal configuration of earphones constituting the audio processing system of embodiment 4. As shown in Fig. 21 , the earphones 200 according to embodiment 4 include a UWB communicator 242 constituting a communication device 240 in addition to the configuration of embodiment 3.

[0228] Note that the configuration other than the UWB communication device 242 is the same as in the third embodiment, and therefore a description thereof will be omitted here. Also, the connection procedure for UWB wireless communication, the method for calculating the relative positions of the earphone 200 and the smartphone 100 via UWB wireless communication, and the like have been described in the second embodiment, and therefore a description thereof will be omitted here.

[0229] Although not shown, the UWB communication device 242 is equipped with multiple (e.g., three) UWB antennas. These multiple UWB antennas are used to calculate the relative positions of the earphone 200 and the smartphone 100. In the fourth embodiment, the audio processor 253 of the earphone 200 generates 3D ringtone data in which the smartphone 100 is the sound source position, based on the relative positions of the earphone 200 and the smartphone 100 calculated using the UWB wireless communication of the earphone 200.

[0230] Also in the fourth embodiment, the charging case 300 can be equipped with a UWB communication device, and the UWB antenna in the charging case 300 can be used in conjunction with multiple UWB antennas in the earphones 200. For example, by determining the distance between the UWB antenna of the earphones 200 and the UWB antenna of the charging case 300, it is possible to improve the accuracy of detecting the direction and distance of the smartphone 100. As a result, it is possible to more accurately calculate the relative position of the smartphone 100 with respect to the earphones 200.

[0231] Fifth Embodiment In the above-described embodiments, an example of a voice processing system including one smartphone as a smart device has been described. In contrast, a fifth embodiment is an example of a voice processing system including multiple smartphones 100 at the same time.

[0232] The earphones 200 constituting the audio processing system 10 of the fifth embodiment have a multi-pairing function and a multi-point function, and can be paired with multiple smart devices. As an example, the earphones 200 according to the fifth embodiment can be paired with up to eight Bluetooth devices (smart devices).

[0233] The basic hardware and software configurations of the devices constituting the speech processing system 10 of the fifth embodiment are the same as those of the above-described embodiments. The following mainly describes the differences between the fifth embodiment and the above-described embodiments, and redundant explanations will be omitted.

[0234] Fig. 22 is a schematic diagram illustrating a voice processing system according to embodiment 5. As shown in Fig. 22, in the voice processing system 10 according to embodiment 5, a plurality of (e.g., three) smartphones 100A, 100B, and 100C are paired with earphones 200 via Bluetooth communication.

[0235] The calculation of the relative positions (first relative position data) between the earphone 200 and each of the smartphones 100A, 100B, and 100C is realized by applying the method of the above-described embodiment. Also, as described in the above-described embodiment, the 3D ringtone data may be generated by the audio processor 173 of the smartphone 100 or the audio processor 253 of the earphone 200.

[0236] Furthermore, the earphones 200 according to the fifth embodiment have a multipoint function as described above. Therefore, when an incoming call is received by any of the three smartphones 100A, 100B, and 100C, 3D sound incoming call data is generated and a 3D ringtone is output from the earphones 200. At that time, a 3D ringtone whose sound source is the smartphone that has received the incoming call among the smartphones 100A, 100B, and 100C is output from the earphones 200. Therefore, the user U1 can easily search for the smartphone that has received the incoming call.

[0237] It is also possible to set different telephone ringtones (melody, message audio, bell, etc.) for each of the smartphones 100A, 100B, and 100C. In this case, user U1 can easily determine which smartphone received the incoming call just by listening to the telephone ringtone output from the earphones 200. Furthermore, since a 3D ringtone with the smartphone that received the incoming call as the sound source position is output from the earphones 200, user U1 can easily search for the smartphone that received the incoming call.

[0238] Sixth Embodiment In the above-described embodiment, the voice processing system 10 includes the smartphone 100 as a smart device. In contrast, a sixth embodiment is an example in which the voice processing system 10 includes a smart device other than the smartphone 100.

[0239] The basic hardware and software configurations of the devices constituting the speech processing system 10 of the sixth embodiment are the same as those of the above-described embodiments. The following mainly describes the differences between the sixth embodiment and the above-described embodiments, and redundant explanations will be omitted.

[0240] Fig. 23 is a schematic diagram illustrating a voice processing system according to a sixth embodiment. As shown in Fig. 23 , in the voice processing system 10 according to the sixth embodiment, in addition to the smartphone 100, a personal computer (e.g., a laptop computer) 800 and a tablet 900 are paired with the earphones 200 via Bluetooth communication as smart devices. Note that the earphones 200 have a multi-pairing function and a multi-point function, similar to those of the fifth embodiment, and can be paired with multiple smart devices.

[0241] The calculation of the relative positions (first relative position data) between the earphone 200 and each smart device (100, 800, 900) is realized by applying the method of the above-described embodiment. Furthermore, the generation of the 3D ringtone data may be performed by the audio processor 173 of the smartphone 100 or the audio processor 253 of the earphone 200.

[0242] There are various methods for receiving calls on the personal computer 800 or the tablet 900. As an example, a method for receiving a call on the personal computer 800 will be described. The personal computer 800 uses, for example, a communication tool to respond to the call. There are various communication tools, such as "Skype" provided by Microsoft, and the user can choose the communication tool that best suits their preference.

[0243] Furthermore, the personal computer 800 and the smartphone may be paired, and incoming calls may be received by the personal computer 800 through so-called "smartphone synchronization." The smartphone paired with the personal computer 800 may be the smartphone 100 constituting the voice processing system 10, or any other smartphone. In this case, when an incoming call is received by the smartphone paired with the personal computer 800, a notification window indicating the incoming call is displayed on the desktop screen of the personal computer 800. User U1 performs operations such as answering the call on the personal computer 800 in accordance with the notification window.

[0244] The earphone 200 according to the sixth embodiment also has a multipoint function. Therefore, as shown in Fig. 23 , when the earphone 200 is paired with multiple smart devices and one of the smart devices receives an incoming call, a 3D ringtone is output from the earphone 200. In this case, the earphone 200 outputs a 3D ringtone whose sound source is the smart device that received the incoming call, allowing the user U1 to easily search for the smart device that received the incoming call.

[0245] Seventh Embodiment In the sixth embodiment described above, the speech processing system 10 is described, which includes, as smart devices, the smartphone 100, the personal computer 800, and the tablet 900. In contrast, the seventh embodiment is an example in which the technology of the present disclosure is applied to multiple windows (multi-windows) displayed on the desktop screen of the personal computer 800 included in the speech processing system 10.

[0246] In the seventh embodiment, the smart devices included in the audio processing system 10 include the smartphone 100 and the like, as well as multiple windows (multi-windows) displayed on the desktop screen of the personal computer 800. Then, a 3D sound conversion process is performed to convert the audio data into 3D sound data so that the user U1 wearing the earphones 200 can hear the audio data as sound with the position of one of the multiple windows as the sound source position.

[0247] The basic hardware and software configurations of the devices constituting the speech processing system 10 of the seventh embodiment are the same as those of the above-described embodiments. The following mainly describes the differences between the seventh embodiment and the above-described embodiments, and omits redundant explanations.

[0248] Fig. 24 is a schematic diagram illustrating a voice processing system according to embodiment 7. As shown in Fig. 24, in this embodiment, a plurality of (two in this example) windows (also referred to as multi-windows) 821 and 822 are displayed on a display screen (desktop screen) 810 of a personal computer 800. In the example shown in Fig. 24, window 821 is displayed overlapping window 822 and is in an active state.

[0249] When a notification occurs in window 821, for example, a pop-up notification 830 is displayed in the lower right corner of window 821, and a notification sound (audio) is output from earphones 200 worn by user U1. At this time, the notification sound data (audio data) is converted into 3D sound data so that user U1 can hear the sound with the display position of window 821 as a smart device as a virtual sound source position. In other words, 3D notification sound data (audio data) with the display position of window 821 as the sound source position is generated.

[0250] For example, in this embodiment, 3D notification sound data is generated with the display position of pop-up notification 830 in window 821 as the sound source position. Note that the 3D audio conversion process, such as generating the 3D notification sound data, is executed by, for example, an audio processor included in personal computer 800. Then, a 3D ringtone with the display position of pop-up notification 830 as the sound source position is output from earphones 200. This allows user U1 to easily recognize the display position of pop-up notification 830.

[0251] The relative position between the earphone 200 and the display position of the pop-up notification 830 can be calculated based on information such as the relative position between the earphone 200 and the antenna (not shown) of the personal computer 800, the relative position between the earphone 200 and the pop-up notification 830, etc.

[0252] The relative position between the earphones 200 and the antenna of the personal computer 800 can be calculated by applying the method of the above-described embodiment. The relative position between the earphones 200 and the pop-up notification 830 can be calculated based on the relative position between the antenna of the personal computer 800 and the display screen 810, the relative display position of the pop-up notification 830 on the display screen 810, etc.

[0253] Information on the relative positions of the antenna of the personal computer 800 and the display screen 810 is stored in advance in, for example, a storage device provided in the personal computer 800, and can be obtained from the storage device as needed. The relative display position of the pop-up notification 830 on the display screen 810 can be obtained from, for example, the OS (operating system) or an app of the personal computer 800.

[0254] Based on the relative position between the earphone 200 and the display position of the pop-up notification 830 calculated in this manner, 3D notification sound data with the display position of the pop-up notification 830 as the sound source position is generated by the personal computer 800, and the generated 3D notification sound data is transmitted to the earphone 200. As a result, a 3D notification sound with the display position of the pop-up notification 830 as the sound source position is output from the earphone 200. Therefore, the user U1 can quickly grasp the display position of the pop-up notification 830 and understand the content of the pop-up notification 830.

[0255] The 3D notification sound output from earphones 200 may also be changed depending on the arrangement of the multiple windows. For example, when window 821 is active and a notification sound corresponding to window 822 behind window 821 is output from earphones 200, the notification sound is made a slightly muffled 3D notification sound. This makes it possible to more clearly notify user U1 that a notification sound corresponding to window 822, which is the rear window, has been generated.

[0256] The above describes an example in which the audio processor provided in the personal computer 800 performs 3D sound conversion processing of the notification sound, but as described in embodiments 2 and 4, the audio processor 253 in the earphone 200 may also perform 3D sound conversion processing.

[0257] Furthermore, the detection of the relative position between the earphone 200 and the antenna of the personal computer 800 can be performed by either the earphone 200 or the personal computer 800, but the detected relative position must be notified to the audio processor that performs the 3D sound conversion process. If the detection of the relative position between the antenna of the earphone 200 and the personal computer 800 and the 3D sound conversion process are performed by the earphone 200, information about the relative position between the antenna of the personal computer 800 and the pop-up notification 830 must be transmitted from the personal computer 800 to the earphone 200.

[0258] In this embodiment, the notification sound when the pop-up notification 830 is displayed in the windows 821 and 822 is the target of the 3D sound conversion process, but it goes without saying that the 3D sound conversion process may also be performed on sound sources other than the notification sound.

[0259] (Other Embodiments) While the embodiments of the technology of the present disclosure have been described above, the technology of the present disclosure is not limited to the above-described embodiments. For example, the smart device constituting the voice processing system is not limited to a smart device that supports one telephone number (one SIM), but may be a smart device that supports multiple telephone numbers (multiple SIMs). In this case, by changing the telephone ringtone for each telephone number, user U1 can easily determine which telephone number is receiving an incoming call.

[0260] Similarly, the smart device that constitutes the voice processing system is not limited to one that corresponds to one email address, but may correspond to multiple email addresses. In this case, by changing the email ringtone for each email address, user U1 can easily determine which email address the incoming email is from.

[0261] The smart devices constituting the voice processing system are not particularly limited as long as they have the same functions as the smart devices according to the above-described embodiments. Examples of other smart devices include smart watches and smart televisions.

[0262] The hearable device constituting the audio processing system is not particularly limited as long as it has the same functions as the hearable device (earphone 200) according to the above-described embodiment. Examples of other hearable devices include headphones, hearing aids, hearing devices using bone conduction, and head-mounted displays (HMDs) with built-in headphones.

[0263] Furthermore, in the above-described embodiment, the output of 3D sound is realized by an audio processor, but it goes without saying that it may be realized by software alone or by software used in combination with hardware.

[0264] Furthermore, the configuration for realizing the technology of the present disclosure is not limited to the above-described embodiment, and various modifications are possible. For example, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these fall within the scope of the technology of the present disclosure.

[0265] Furthermore, the numerical values ​​and messages that appear in the text and figures are merely examples, and the same effects as those of the above-described embodiment can be obtained even if different numerical values ​​and messages are used.

[0266] Furthermore, some or all of the functions of the technology disclosed herein may be implemented in hardware, for example, by designing them as integrated circuits, or in software, by a microprocessor unit or the like interpreting and executing a program that implements each function. Hardware and software may also be used together. The software may be pre-stored in the smartphone's storage device or the like at the time of product shipment, or may be obtained from various server devices or the like on the Internet after product shipment, or may be provided on a memory card, optical disc, or the like.

[0267] Furthermore, the control lines and information lines shown in the diagram are those considered necessary for explanation, and do not necessarily represent all of the control lines and information lines on the product. In reality, it can be assumed that almost all components are interconnected.

[0268] 10...Speech processing system, 100...Smartphone, 101...Display screen, 102...Front camera (in-camera), 103...Rear camera (out-camera), 104...Left antenna, 105...Right antenna, 106...Central antenna, 107...Left UWB antenna, 108...Right UWB antenna, 109...Central UWB antenna, 110...Control device, 120...System bus, 130...Storage device, 140...Sensor device, 141...Positioning sensor, 142...Geomagnetic sensor, 1 43...acceleration sensor, 144...gyro sensor, 145...light sensor, 150...communication device, 151...LAN communication device, 152...telephone network communication device, 153...Bluetooth communication device, 154...UWB communication device, 160...video device, 161...imaging sensor (camera), 162...display, 163...video processor, 170...audio device, 171...audio sensor (microphone), 172...audio output element, 173...audio processor, 180...operation input device, 190...vibrator, 1 95...light, 200...wireless earphone (earphone), 201...left earphone, 202...right earphone, 210...control device, 220...system bus, 230...storage device, 240...communication device, 241...Bluetooth communication device, 242...UWB communication device, 250...audio device, 251...audio sensor (microphone), 252...audio output element, 253...audio processor, 260...operation input device, 270...display element, 280...vibrator, 290...sensor device, 291...positioning sensor sensor, 292...geomagnetic sensor, 293...acceleration sensor, 294...gyro sensor, 295...light sensor, 300...charging case, 310...control device, 320...system bus, 330...power supply device, 340...display element, 400...access point, 500...network server, 550...network, 800...personal computer, 810...display screen (desktop screen), 821, 822...window (multi-window), 830...pop-up notification, 900...tablet

Claims

1. A sound processing system comprising a smart device and a hearable device connected to the smart device via wireless communication, transmitting sound data including ringtone data generated by the smart device to the hearable device and outputting the sound data as sound from the hearable device, wherein one of the smart device or the hearable device comprises an audio device that performs 3D sound conversion processing to convert the sound data into 3D sound data so that the sound data can be heard by a user wearing the hearable device as sound with the position of the smart device as the sound source position.

2. A voice processing system as described in claim 1, wherein one of the hearable device or the smart device has a position calculation device that calculates first relative position data that identifies the relative position between the hearable device and the smart device, and the voice device performs the 3D sound conversion processing based on the first relative position data.

3. The audio processing system according to claim 2, wherein the audio device does not perform the 3D sound conversion process when the reliability of the first relative position data is lower than a preset threshold value.

4. A voice processing system as described in claim 2, further comprising a charging device for charging the hearable device, wherein the position calculation device calculates second relative position data that identifies a relative position between the charging device and at least one of the smart device and the hearable device, and calculates the first relative position data based on the second relative position data.

5. A voice processing system according to claim 2, wherein the position calculation device calculates the first relative position data by using short-range wireless communication or ultra-wideband wireless communication.

6. A voice processing system as described in claim 1, wherein the voice device starts the 3D sound conversion process when the smart device receives an incoming call from the telephone network or a server, and stops the 3D sound conversion process a predetermined period of time after the smart device ends the incoming call from the telephone network or starts a call, or after a predetermined period of time has passed since the smart device received an incoming call from the server.

7. A voice processing system according to claim 1, wherein the voice device changes the intermittent interval of the voice data according to the relative distance between the smart device and the hearable device through the 3D sound conversion process.

8. The audio processing system according to claim 1, wherein the audio device changes the frequency of the audio data according to the relative distance between the smart device and the hearable device through the 3D sound conversion process.

9. An audio processing system as described in claim 1, wherein the smart device has a light sensor that detects the surrounding brightness, and the audio device determines whether the smart device is occluded based on the detection result of the light sensor, and converts the audio data into separate 3D audio data when it is determined that the smart device is not occluded and when it is determined that the smart device is occluded by the 3D audio conversion process.

10. The audio processing system according to claim 1, wherein the audio device performs the 3D sound conversion processing so as to emphasize the Doppler effect according to the rate of change of the relative distance between the smart device and the hearable device.

11. A voice processing system as described in claim 1, comprising a plurality of smart devices to which the hearable device is connected via wireless communication, and the voice device starts the 3D sound conversion process when any one of the plurality of smart devices receives an incoming call from a telephone network or a server.

12. A voice processing system as described in claim 1, wherein the smart device includes each of a plurality of windows displayed on a desktop screen of a personal computer, and the audio device performs the 3D sound conversion process so as to present the voice data to a user wearing the hearable device as sound whose sound source position is the position of one of the plurality of windows.

13. A sound processing method using a sound processing system comprising a smart device and a hearable device connected to the smart device via wireless communication, which transmits sound data including ringtone data generated by the smart device to the hearable device and outputs the sound data as sound from the hearable device, and performs a 3D sound conversion process to convert the sound data into 3D sound data so that the sound data can be heard by a user wearing the hearable device as sound with the position of the smart device as the sound source position.

14. The audio processing method according to claim 13, further comprising: calculating first relative position data that identifies a relative position between the hearable device and the smart device; and performing the 3D sound conversion processing based on the calculated first relative position data.

15. A smart device that is connected via wireless communication to a hearable device worn by a user and transmits audio data including ringtone data to the hearable device, and is equipped with an audio device that performs 3D sound conversion processing to convert the audio data into 3D sound data so that the user wearing the hearable device can hear the audio data as sound with the position of the smart device as the sound source position.

16. A smart device as described in claim 15, comprising: a light for illumination; and a control device for controlling the operation of the light; wherein the audio device performs the 3D sound conversion process when a call is received from a telephone network or a server; and the control device turns on or blinks the light when a call is received from the telephone network or a server and the user wearing the hearable device turns towards the smart device.

17. A smart device as described in claim 15, comprising: a vibrator; and a control device that controls the operation of the vibrator, wherein the audio device performs the 3D sound conversion process when a call is received from a telephone network or a server, and the control device vibrates the vibrator when a call is received from a telephone network or a server and when a user wearing the hearable device turns towards the smart device.

18. A hearable device that is connected to a smart device via wireless communication and outputs audio data including ringtone data received from the smart device as sound, the hearable device comprising: an audio device that performs 3D sound conversion processing to convert the audio data into 3D sound data so that the audio data can be heard by a user wearing the hearable device as sound with the position of the smart device as the sound source position.

19. A hearable device as described in claim 18, comprising: a vibrator; and a control device that controls the operation of the vibrator, wherein the control device vibrates the vibrator while the 3D sound conversion process is being performed by the audio device and when a user wearing the hearable device turns towards the smart device.

20. A hearable device as described in claim 18, comprising: a vibrator; and a control device that controls the operation of the vibrator, wherein the control device vibrates the vibrator while the 3D sound conversion process is being performed by the audio device, and changes the vibration intensity of the vibrator between the left and right sides of the user's head depending on the orientation of the smart device relative to the user wearing the hearable device.

Images