Audio processing device and audio processing method

Abstract

To provide a voice processing device and a voice processing method capable of automatically outputting a voice according to user's preference.SOLUTION: A voice processing device acquires a collection voice collected by a sound collection device at a place at which a first user is present, stores a favorability level of a second user existing at a place different from a place at which the first user is present with respect to a prescribed sample voice, estimates a favorability level of the second user to the collection voice on the basis of the favorability level of the second user to the sample voice, generates a processing voice obtained by processing the collection voice by emphasizing or suppressing the collection voice according to the favorability level of the second user to the collection voice, and outputs the processing voice to an output device for outputting the voice to the second user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a voice processing device and a voice processing method. [Background technology]

[0002] A vehicle acoustic system including an acoustic detector that detects ambient noise outside the vehicle is known (for example, Patent Document 1). The vehicle acoustic system described in Patent Document 1 generates a privacy zone in the vehicle cabin by filtering the ambient noise to identify continuous sounds in the ambient noise, processing the continuous sounds into masking sounds, and outputting the masking sounds from a speaker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-86784 A Summary of the Invention [Problem to be solved by the invention]

[0004] The vehicle audio system described in Patent Document 1 has a problem in that, because audio processing is performed on the audio inside the vehicle, it is not possible to automatically output audio that matches the user's preferences.

[0005] An object of the present invention is to provide a voice processing device and a voice processing method capable of automatically outputting a voice that matches the preferences of a user. [Means for solving the problem]

[0006] The present invention solves the above problem by obtaining collected audio collected by a sound collection device at a location where a first user is located, storing a liking level of a second user who is located at a location different from the first user for a predetermined sample audio, estimating the second user's liking level for the collected audio based on the second user's liking level for the sample audio, generating a processed audio by processing the collected audio by emphasizing or suppressing the collected audio depending on the second user's liking level for the collected audio, and outputting the processed audio to an output device that outputs the audio to the second user. Effect of the Invention

[0007] According to the present invention, a processed voice that has been processed according to the user's liking for a collected voice can be output to the user, so that a voice that matches the user's preferences can be automatically output. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of a communication system according to an embodiment of the present invention. [Diagram 2] 5 is a flowchart showing a voice output process by the communication system according to the present embodiment. [Diagram 3] 5 is a flowchart showing a voice output process by the communication system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, a voice processing device and a voice processing method according to the present invention will be described using an example in which they are applied to a communication system.

[0010] 1 is a block diagram of a communication system 100 according to this embodiment. The communication system 100 according to this embodiment is used in a situation where a user A and a user B who are not in the same space converse with each other. In this embodiment, a user A who is in a vehicle 1 and in an interior space 10 is taken as an example, and a user B who is in a remote space 20 that is a place away from the vehicle 1 is taken as an example. The location of user A and the location of user B are different.

[0011] The communication system 100 may be used in a situation where the location of user A and the location of user B are different, and the locations of user A and user B are not particularly limited. For example, user A does not need to be in the vehicle 1. In the following description, for convenience, user A in the vehicle space 10 is referred to as in-vehicle user A, and user B in the remote space 20 is referred to as remote user B.

[0012] In this embodiment, the state of each user when the in-vehicle user A and the remote user B have a conversation is not particularly limited. For example, in FIG. 1, the in-vehicle user A is shown steering the vehicle 1, but the in-vehicle user A may not be a driver, but may be a passenger sitting in the passenger seat or the back seat. For example, in FIG. 1, the remote user B is shown sitting, but the remote user B may be standing. In this embodiment, the in-vehicle user A and the remote user B have a conversation, but the communication system 100 can be used in situations other than conversation. For example, the communication system 100 can be used in a situation where the in-vehicle user A talks unilaterally to the remote user B, that is, where the voice flow is one-way.

[0013] In this embodiment, the communication system 100 is used in a system that uses virtual reality technology. As shown in FIG. 1, the in-car user A and the remote user B are in different spaces, so the in-car user A and the remote user B cannot talk to each other in the same space. However, in the virtual reality technology that allows a user to perceive a virtual space, which is a world created by a computer, as reality, the computer can create a virtual space for the remote user B as if he were in the in-car space 10. In the virtual space, the remote user B perceives as if he were in the in-car space 10 at least through his vision and hearing. For example, the remote user B can recognize the virtual space as the in-car space 10 through his vision by wearing a device using virtual reality technology known at the time of filing of this application, such as VR goggles. In addition, the remote user B can recognize the virtual space as the in-car space 10 through his hearing by the communication system 100 according to this embodiment. Hereinafter, each configuration of the communication system 100 will be described with reference to FIG. 1.

[0014] 1, the communication system 100 includes an in-vehicle space 10, a remote space 20, and a server 30. The remote space 20 is a space separated from the in-vehicle space 10. The remote space 20 may be, for example, the interior of a vehicle other than the vehicle 1 or a room in a building, but the remote space 20 is not particularly limited as long as it is a space separate from the in-vehicle space 10.

[0015] The interior space 10 is a section showing the interior of the vehicle 1. The interior space 10 is provided with an in-vehicle microphone 11, an in-vehicle controller 12, and an in-vehicle communication device 14. The in-vehicle microphone 11, the in-vehicle controller 12, and the in-vehicle communication device 14 are mounted on the vehicle 1. The devices provided in the interior space 10 are connected to each other by, for example, a Controller Area Network (CAN) or other in-vehicle networks (such as Ethernet (registered trademark)) in order to transmit and receive information to and from each other.

[0016] The in-vehicle microphone 11 is a sound collecting device that collects sound at a location where the in-vehicle user A is located. The in-vehicle microphone 11 includes an in-vehicle microphone 11a for collecting sound inside the vehicle 1 and an exterior microphone 11b for collecting sound outside the vehicle 1. The in-vehicle microphone 11a may be a device or equipment mounted on the vehicle 1 that collects sound inside the vehicle 1, and the shape, number, characteristics, installation position, etc., of the in-vehicle microphone 11a are not particularly limited. Similarly, the exterior microphone 11b may be a device or equipment mounted on the vehicle 1 that collects sound outside the vehicle 1, and the shape, number, characteristics, installation position, etc. of the exterior microphone 11b are not particularly limited.

[0017] The in-vehicle microphone 11a collects sounds that flow in the in-vehicle space 10 and are generated from a human being or a device or system installed in the vehicle 1. Examples of sounds that the in-vehicle microphone 11a collects include the voice of the in-vehicle user A or other occupants of the vehicle 1, sounds indicating that the vehicle 1 is running (engine sounds, motor sounds), sounds indicating that the direction indicators are operating (blinker sounds), sounds indicating that the wipers are operating (wiper sounds), sounds indicating that the hazard lights are operating (hazard light sounds), sounds output from the car navigation system (audio guidance sounds), sounds played from the audio system (music, television sounds), and the like.

[0018] The exterior microphone 11b collects sounds that are emitted in the environment surrounding the vehicle 1 and are generated from humans, other vehicles other than the vehicle 1, or devices other than devices mounted on the vehicle 1. Examples of sounds that the exterior microphone 11b collects include sounds related to other vehicles other than the vehicle 1 (such as the running sounds of other vehicles and horn sounds), sounds related to emergency vehicles (such as sirens), and sounds related to advertising vehicles (such as advertising sounds).

[0019] The voice collected by the in-vehicle microphone 11 includes at least one of the voice of the in-vehicle user A and the background voice of the in-vehicle user A. The background voice of the in-vehicle user A is a voice other than the voice uttered by the in-vehicle user A (for example, the engine sound of the vehicle 1 and the horn sound of another vehicle, etc.). Generally, some background voice exists inside or outside the vehicle 1. Therefore, in this embodiment, when the in-vehicle user A speaks, the voice collected by the in-vehicle microphone 11 includes the voice of the in-vehicle user A and the background voice, and on the other hand, when the in-vehicle user A does not speak, the voice collected by the in-vehicle microphone 11 includes the background voice. In addition, hereinafter, for convenience of explanation, the voice collected by the in-vehicle microphone 11 will be referred to as the collected voice. The collected voice collected by the in-vehicle microphone 11 is output to the in-vehicle controller 12 as collected voice data.

[0020] The in-vehicle controller 12 is a computer including a ROM (Read Only Memory) storing a program for transmitting and receiving voice data to and from the remote space 20, a CPU (Central Processing Unit) executing the program stored in the ROM, and a RAM (Random Access Memory) functioning as an accessible storage device. The in-vehicle controller 12 has a voice acquisition unit 13. The in-vehicle controller 12 can execute the function of the voice acquisition unit 13 by executing the program stored in the ROM.

[0021] The voice acquisition unit 13 acquires collected voice collected by the in-vehicle microphone 11 at the location where the in-vehicle user A is located. When the voice acquisition unit 13 acquires the collected voice, it transfers the collected voice as is to the in-vehicle communication device 14. The voice acquisition unit 13 also outputs a transmission command including a transmission destination of the collected voice data to the in-vehicle communication device 14. The voice acquisition unit 13 is an example of the "voice acquisition unit" of the present invention.

[0022] The in-vehicle communication device 14 is a device having a wireless communication function for wirelessly transmitting and receiving data to and from the outside of the vehicle 1. For example, the in-vehicle communication device 14 is a telematics control unit (TCU). The in-vehicle communication device 14 uses a communication standard such as 4G / LTE or Wifi (registered trademark) by the wireless communication function to connect to the Internet and transmit and receive various data to and from a server or a system provided outside the vehicle 1. A transmission command is input to the in-vehicle communication device 14 from the in-vehicle controller 12 together with the collected voice data. The in-vehicle communication device 14 transmits the collected voice data to a destination indicated in the transmission command. In this embodiment, the in-vehicle communication device 14 transmits the collected voice data to the remote communication device 21 in accordance with the transmission command from the in-vehicle controller 12.

[0023] The remote space 20 will now be described. As shown in Fig. 1, the remote space 20 is provided with a remote communication device 21, a remote camera 22, a remote microphone 23, a remote input device 24, a remote output device 25, and a remote controller 26. The devices provided in the remote space 20 are connected to each other by, for example, Ethernet (registered trademark) in order to transmit and receive information to and from each other.

[0024] The remote communication device 21 is a device equipped with a wireless communication function for wirelessly transmitting and receiving data with the outside of the remote space 20. An example of the remote communication device 21 is a router. The remote communication device 21 uses the wireless communication function to connect to the Internet using communication standards such as 4G / LTE and Wifi (registered trademark), and transmits and receives various data with servers and systems provided outside the remote space 20. In this embodiment, the remote communication device 21 transmits and receives data with the in-vehicle communication device 14 and the server communication device 31. The data transmitted and received between each communication device will be described later.

[0025] The remote camera 22 is an imaging device that captures an image of the remote user B in the remote space 20. The remote camera 22 is preferably installed in a position where it can capture an image of the facial expression of the remote user B. The captured image including the facial expression of the remote user B captured by the remote camera 22 is output to the remote controller 26 as captured image data of the remote user B. The remote camera 22 is not particularly limited in terms of its form, number, characteristics, installation location, etc. For example, the remote camera 22 may be composed of multiple cameras or multiple types of cameras.

[0026] The remote microphone 23 is a sound collecting device that collects sound at the location (remote space 20) where the remote user B is located. The remote microphone 23 is preferably installed at a position where it can collect the sound of the remote user B. The collected sound collected by the remote microphone 23 is output to the remote controller 26. The remote microphone 23 is not particularly limited in terms of its shape, number, characteristics, installation location, etc. For example, the remote microphone 23 may be composed of multiple microphones or multiple types of microphones.

[0027] The remote input device 24 is a device that functions as a human interface that can be operated by the remote user B. The remote user B can input settings related to the communication system 100 by operating the remote input device 24. The remote input device 24 may be configured with each device, such as a keyboard and a mouse, separately, or may be configured with a display and an input unit integrated together, such as a touch panel. The form, characteristics, installation location, etc. of the remote input device 24 are not particularly limited.

[0028] In this embodiment, the remote device user B sets the voice processing mode via the remote input device 24 as a setting related to the communication system 100. For example, the remote user B sets the voice processing mode by selecting one mode from a plurality of modes displayed on the screen. The method of setting the voice processing mode by the remote user B is one example, and other setting methods may be used. The setting information of the voice processing mode inputted to the remote input device 24 is outputted to the remote controller 26.

[0029] Here, the voice processing mode according to the present embodiment will be described. The voice processing mode is a mode that specifies the viewpoint from which the collected voice collected by the vehicle-mounted microphone 11 is processed. In the present embodiment, two types of modes are set as the voice processing mode.

[0030] One of the two voice processing modes is called the preference mode, which places emphasis on the preference of remote user B. The preference of remote user B is the taste of remote user B. The taste of remote user B includes the tendency of remote user B to like things and the tendency of remote user B to dislike things. When the voice processing mode is set to the preference mode, a voice processing unit 28 of a remote controller 26, which will be described later, processes the collected voice according to the preference of remote user B for the collected voice. The voice processing unit 28 will be described later.

[0031] The other of the two types of voice processing modes is called an importance mode, which is a mode that emphasizes the importance of the collected voice. The importance of the collected voice is the degree to which the user wants to hear the collected voice. When the voice processing mode is set to the importance mode, a voice processing unit 28 of a remote controller 26 described later determines whether or not processing of the collected voice is required according to the importance of the collected voice, and processes the collected voice if it is determined that processing is required.

[0032] The remote output device 25 is a device that outputs voice to the remote user B. The remote output device 25 is preferably installed in a position where the remote user B can easily hear the voice output from the remote output device 25. The remote output device 25 receives collected voice data or processed voice data from the remote controller 26. The processed voice data is data of processed voice obtained by processing the collected voice by the voice processing unit 28 of the remote controller 26.

[0033] The remote output device 25 generates a playback sound based on collected audio data or processed audio data output from the remote controller 26, and outputs the generated playback sound. An example of the remote output device 25 is a speaker. The remote output device 25 is not particularly limited in its form, number, installation location, characteristics, etc. For example, the remote output device 25 may be composed of multiple speakers or multiple types of speakers. The remote output device 25 may also have a stereophonic function that outputs the playback sound as stereophonic sound. For example, when the audio data input from the remote controller 26 includes a binaurally recorded signal, the remote output device 25 may output the playback sound as stereophonic sound using a binaural method using the stereophonic function.

[0034] The remote controller 26 is a computer consisting of a ROM (Read Only Memory) storing a program for processing the voice heard by the remote user B, a CPU (Central Processing Unit) that executes the program stored in the ROM, and a RAM (Random Access Memory) that functions as an accessible storage device. The remote controller 26 has a target sample voice identification unit 27, a voice processing unit 28, and a voice output unit 29. The remote controller 26 can execute the functions of the target sample voice identification unit 27, the voice processing unit 28, and the voice output unit 29 by executing the program stored in the ROM. Each function of the remote controller 26 will be described.

[0035] The target sample voice identification unit 27 receives the collected voice data transmitted from the in-vehicle communication device 14 via the remote communication device 21. The target sample voice identification unit 27 sequentially performs a voice recognition process, a clustering process, and a sample voice identification process on the collected voice data transmitted from the in-vehicle communication device 14 to identify a sample voice that is the same as or similar to the collected voice as a target sample voice. The sample voice is a sample voice used to estimate the remote user B's favorability rating for the collected voice.

[0036] The target sample voice identifying unit 27 executes a voice recognition process on the collected voice data to determine the type of voice included in the collected voice. For example, when the collected voice includes a voice uttered by the in-vehicle user A, the engine sound of the vehicle 1, and the horn sound of another vehicle, the target sample voice identifying unit 27 extracts the voice of the in-vehicle user A, the engine sound of the vehicle 1, and the horn sound of another vehicle from the collected voice. The target sample voice identifying unit 27 also distinguishes the engine sound of the vehicle 1 and the horn sound of another vehicle from the voice of the in-vehicle user A, treating these sounds as background sounds.

[0037] Next, the target sample voice identification unit 27 classifies the various types of voice into preset classes by performing a clustering process on the determined types of voice. For example, the target sample voice identification unit 27 classifies the extracted voice of the in-vehicle user A, the engine sound of the vehicle 1, and the horn sound of the other vehicle into preset classes. As an example, the target sample voice identification unit 27 classifies the voice of the in-vehicle user A into "user voice", the engine sound of the vehicle 1 into "driving sound of the vehicle 1", and the horn sound of the other vehicle into "voice of the other vehicle". Note that the class classification method, the depth of the class hierarchy, the number of classes, etc. in the clustering process are not particularly limited, and the above is merely an example.

[0038] Furthermore, the target sample sound identification unit 27 identifies a sample sound that is the same as or similar to a sound included in the classified class as a target sample sound. The target sample sound identification unit 27 identifies a target sample sound for each sound included in the classified class, thereby identifying the target sample sound for all sounds included in the collected sound.

[0039] Here, the concepts of identical and similar in this embodiment will be described. For example, the frequency components (frequency spectrum) of the voice can be given as parameters for defining the voice as the same or similar. For example, the target sample voice specification unit 27 specifies, as the target sample voice, a sample voice having the same frequency components as the frequency components of the voice included in the classified class from among a plurality of sample voices stored in advance. In addition, for example, the target sample voice specification unit 27 specifies, as the target sample voice, a sample voice having frequency components whose frequency band is within a predetermined range or whose peak value is within a predetermined range, although the frequency components do not completely match those of the voice included in the classified class, from a plurality of sample voices stored in advance. Note that the parameters for specifying the target sample voice are not limited to frequency components, and may be other parameters related to voice.

[0040] The target sample voice identifying unit 27 outputs the identified target sample voice to the remote communication device 21. Note that each process by the target sample voice identifying unit 27 is an example, and is not limited to the above processes. The processes executed by the target sample voice identifying unit 27 may include voice recognition processes, clustering processes, and voice identification processes known at the time of filing of this application. The target sample voice identifying unit 27 is an example of the "target sample voice identifying unit" of the present invention.

[0041] The voice processing unit 28 receives input of collected voice data from the in-vehicle communication device 14 via the remote communication device 21. Also, setting information of a voice processing mode is input to the voice processing unit 28 from the remote input device 24. Also, information on the remote user B's favorable rating for the collected voice is input to the voice processing unit 28 from the server communication device 31 via the remote communication device 21.

[0042] When the voice processing mode is set to the likeability mode, the voice processing unit 28 executes voice processing on the collected voice, and does not transfer the collected voice input to the voice processing unit 28 to the voice output unit 29. When the voice processing mode is set to the importance mode, the voice processing unit 28 executes a process for determining whether voice processing is necessary for the collected voice, and does not transfer the collected voice input to the voice processing unit 28 to the voice output unit 29. On the other hand, when the voice processing mode is not set to either the likeability mode or the importance mode, the voice processing unit 28 transfers the collected voice input to the voice processing unit 28 to the voice output unit 29.

[0043] The voice processing by the voice processing unit 28 will be described. First, the case where the voice processing mode is set to the likeability mode will be described. When the voice processing mode is set to the likeability mode, the voice processing unit 28 identifies the likeability of the remote user B for each voice included in the collected voice, and compares the likeability of the remote user B with a predetermined first threshold for each voice included in the collected voice. When the likeability of the remote user B for a specific voice is higher than the predetermined first threshold, the voice processing unit 28 processes the collected voice so that the specific voice included in the processed voice is relatively emphasized compared to the specific voice included in the collected voice. On the other hand, when the likeability of the remote user B for a specific voice included in the collected voice is lower than the predetermined first threshold, the voice processing unit 28 processes the collected voice so that the specific voice included in the processed voice is relatively suppressed compared to the specific voice included in the collected voice. In this way, by performing the emphasis process or the suppression process for each voice included in the collected voice, a processed voice can be generated in which a voice that the user relatively likes is emphasized and a voice that the user relatively does not like is suppressed. The above-mentioned predetermined first threshold is a threshold for determining whether to emphasize or suppress the sound, and is a threshold determined in advance.

[0044] For example, if the remote user B likes music output from a car navigation system and thus the remote user B's preference for the music included in the collected sound is higher than a predetermined first threshold, the sound processing unit 28 processes the collected sound so that the music included in the processed sound is relatively emphasized compared to the music included in the collected sound. Also, if the remote user B does not like the horn sounds of other vehicles and thus the remote user B's preference for the horn sounds of other vehicles included in the collected sound is lower than a predetermined first threshold, the sound processing unit 28 processes the collected sound so that the horn sounds of other vehicles included in the processed sound are relatively suppressed compared to the horn sounds of other vehicles included in the collected sound.

[0045] It should be noted that the voice emphasis processing and voice suppression processing by the voice processing unit 28 described by way of example can be any voice emphasis processing and voice suppression processing known at the time of filing of this application.

[0046] Next, a case where the voice processing mode is set to the importance mode will be described. When the voice processing mode is set to the importance mode, the voice processing unit 28 specifies the level of importance of each voice included in the collected voice, and compares the importance of the voice with a predetermined second threshold for each voice included in the collected voice. When the importance of at least one voice among the voices included in the collected voice is higher than the predetermined second threshold, the voice processing unit 28 determines that the importance of the collected voice is higher than the predetermined second threshold and that voice processing is necessary for the collected voice. Specifically, the voice processing unit 28 determines that voice processing is necessary for a voice whose importance is higher than the predetermined second threshold. On the other hand, when the importance of each voice included in the collected voice is lower than the predetermined second threshold, the voice processing unit 28 determines that the importance of the collected voice is lower than the predetermined second threshold and that processing of the collected voice is unnecessary. Note that the above-mentioned predetermined second threshold is a threshold for determining whether or not voice processing is necessary based on the importance of the collected voice, and is a threshold that is determined in advance.

[0047] The importance of the collected voice is set according to the contents of the collected voice. For example, the siren sound of an emergency vehicle is classified in advance into a high importance class as a sound that should be emphasized and transmitted to the remote user B. The voice processing process of the voice processing unit 28 will be described using an example in which the collected voice includes the siren sound of an emergency vehicle, that is, an emergency vehicle is present in the vicinity of the vehicle 1.

[0048] The voice processing unit 28 uses voice recognition processing to distinguish each sound included in the collected voice, thereby identifying that the collected voice includes a siren sound of an emergency vehicle. The voice processing unit 28 compares the importance of the siren sound of the emergency vehicle with a predetermined second threshold, and determines that processing of the siren sound of the emergency vehicle is necessary if the importance of the siren sound of the emergency vehicle is higher than the predetermined second threshold. In this case, the voice processing unit 28 processes the collected voice so that the siren sound of the emergency vehicle included in the processed voice is emphasized relatively more than the siren sound of the emergency vehicle included in the collected voice. The voice processing unit 28 is an example of the "voice processing unit" of the present invention.

[0049] The audio output unit 29 outputs the collected audio or processed audio input from the audio processing unit 28 to the remote output device 25. When the audio processing mode is set to the likeability mode, the processed audio is input from the audio processing unit 28 to the audio output unit 29. In this case, the audio output unit 29 outputs the input processed audio to the remote output device 25. When the audio processing mode is set to the importance mode and the audio processing unit 28 executes audio processing, the processed audio is input from the audio processing unit 28 to the audio output unit 29. In this case, the audio output unit 29 outputs the input processed audio to the remote output device 25. When the audio processing mode is not set to either the likeability mode or the importance mode, the collected audio is input from the audio processing unit 28 to the audio output unit 29. In this case, the audio output unit 29 outputs the input collected audio to the remote output device 25. The audio output unit 29 is an example of the "audio output unit" of the present invention.

[0050] Next, the server 30 will be described. As shown in Fig. 1, the server 30 includes a server communication device 31, a database 32, and a server controller 33. The devices included in the server 30 are connected to each other by, for example, Ethernet (registered trademark) in order to transmit and receive information to and from each other.

[0051] The server communication device 31 is a device equipped with a wireless communication function for wirelessly transmitting and receiving data to and from the remote communication device 21. An example of the server communication device 31 is a router. The server communication device 31 uses the wireless communication function to connect to the Internet using communication standards such as 4G / LTE and Wifi (registered trademark), and transmits and receives various data to and from other servers and systems provided outside the server 30. In this embodiment, the server communication device 31 transmits and receives data to and from the remote communication device 21.

[0052] The following describes data transmitted and received by the server communication device 31. The server communication device 31 receives captured image data including facial expressions of remote user B captured by the remote camera 22 and voice data of remote user B collected by the remote microphone 23 from the remote communication device 21. The server communication device 31 outputs the received captured image data and voice data to the database 32 and the server controller 33. In this embodiment, it is assumed that the server communication device 31 receives the captured image data and voice data before in-car user A and remote user B use the communication system 100.

[0053] Furthermore, the server communication device 31 receives data of the target sample voice identified by the target sample voice identification unit 27 from the remote communication device 21. The server communication device 31 outputs the received data of the target sample voice to the server controller 33. In this embodiment, it is assumed that the server communication device 31 receives the data of the target sample voice after the in-vehicle user A and the remote user B start using the communication system 100.

[0054] Further, information on the remote user B's favorable rating for the collected voice is input to the server communication device 31 from the server controller 33. The server communication device 31 transmits the information on the remote user B's favorable rating for the collected voice to the remote communication device 21.

[0055] The database 32 stores various data of users who use the communication system 100. The database 32 stores the likeability rating of the remote user B for a given sample voice. There are multiple types of sample voices, and the database 32 stores the likeability rating of the remote user B for each sample voice. The likeability rating of the remote user B for the sample voices stored in the database 32 is estimated by a likeability rating estimation unit 34 of the server controller 33, which will be described later.

[0056] The database 32 also stores information on the user's expression when listening to a predetermined sample voice as data used in the popularity rating estimation process of the popularity rating estimation unit 34. The user's expression when listening to a predetermined sample voice includes at least one of the following: the facial expression of the user when listening to the predetermined sample voice, the sound quality of the voice uttered by the user when listening to the predetermined sample voice, and the content of the user's speech when listening to the predetermined sample voice. The facial expression of the user when listening to the predetermined sample voice is information acquired from the data of the captured image including the facial expression of the remote user B captured by the remote camera 22. Moreover, the sound quality of the voice uttered by the user when listening to the predetermined sample voice and the content of the user's speech when listening to the predetermined sample voice are information acquired from the voice data of the remote user B collected by the remote microphone 23.

[0057] For example, assume that an opportunity to obtain information on the expression mode of the remote user B when listening to the sample voice is provided before the in-car user A and the remote user B start using the communication system 100. On this occasion, the reaction (so-called reaction) of the remote user B when listening to the sample voice is obtained as data by the remote camera 22 and the remote microphone 23. For example, when the remote user B listens to the engine sound of a specific model of vehicle as a sample voice, the remote camera 22 captures the facial expression of the remote user B at that time, and the remote microphone 23 collects the voice of the remote user B at that time. Every time the sample voice changes, the remote camera 22 and the remote microphone 23 obtain the reaction of the remote user B when listening to the sample voice as data. When the remote user B finishes listening to all the sample voices, the image data captured by the remote camera 22 and the voice data collected by the remote microphone 23 are transmitted to the server communication device 31 via the remote communication device 21. This allows the database 32 to store sample voices in association with the expression of remote user B when listening to the sample voices, thereby realizing a database of sample voices and the expression of remote users. Note that in this embodiment, there are no particular limitations on the type and number of sample voices, or the timing at which the user listens to the sample voices. The various data stored in the database 32 are used by a favorability rating estimation unit 34 of the server controller 33. The database 32 is an example of the "storage unit" of the present invention.

[0058] The server controller 33 is a computer including a ROM (Read Only Memory) storing programs for executing a sample voice likability estimation process for estimating a user's likability for a sample voice and a collected voice likability estimation process for estimating a user's likability for a collected voice, a CPU (Central Processing Unit) for executing the programs stored in the ROM, and a RAM (Random Access Memory) functioning as an accessible storage device. The server controller 33 has a likability estimation unit 34. The server controller 33 can execute the function of the likability estimation unit 34 by executing the program stored in the ROM. The function of the likability estimation unit 34 will be described.

[0059] The likability estimation unit 34 estimates the likability of the remote user B for the sample by the sample voice likability estimation process based on the information of the expression mode of the user when listening to a certain sample voice stored in the database 32. For example, the likability estimation unit 34 estimates the likability of the remote user B for the sample based on the change in the facial expression of the remote user before and after listening to the sample voice. For example, the likability estimation unit 34 estimates the likability of the remote user B for the sample based on the change in the intonation of the remote user's voice before and after listening to the sample voice. For example, the likability estimation unit 34 estimates the likability of the remote user B for the sample based on the content uttered by the remote user B after listening to the sample voice. The content uttered by the remote user B includes, for example, content that directly expresses the preference of the remote user B, such as "like" and "dislike", or content that suggests the preference of the remote user B. The likability estimation unit 34 stores the estimated likability of the remote user B for the sample voice in the database 32.

[0060] An example of the likeability estimation unit 34 is a trained model that has been trained by machine learning to estimate the likeability of the remote user B for a sample voice from the remote user B's expression style. The trained model has a trained neural network. The trained neural network is a neural network that has been subjected to machine learning processing using teacher data so as to estimate the remote user B's favorability rating for a given sample voice when the user's expression when listening to a given sample voice is input. When the favorability rating estimation unit 34 receives the remote user B's expression when listening to a given sample voice via the server communication device 31, the favorability rating estimation unit 34 uses the trained neural network to estimate the remote user B's favorability rating for the given sample voice.

[0061] When the data of the target sample voice is input, the likeability estimation unit 34 estimates the likeability of the remote user for the collected voice by the collected voice likeability estimation process based on the likeability of the remote user B for a predetermined sample voice stored in the database 32. Specifically, the likeability estimation unit 34 extracts a sample voice corresponding to the target sample voice from among the sample voices stored in the database 32. The database 32 stores the sample voices and the likeability of the remote user for the sample voice in association with each other. The likeability estimation unit 34 specifies the likeability of the remote user B for the identified sample voice by referring to the database 32. The likeability estimation unit 34 estimates the likeability of the remote user B for the identified sample voice as the likeability of the remote user B for the collected voice. The likeability estimation unit 34 transmits the estimated likeability of the remote user B for the collected voice to the remote communication device 21 via the server communication device 31.

[0062] 2 and 3, a flow will be described below, in which voice collected at the location of in-vehicle user A is output to remote user B by communication system 100. FIGS. 2 and 3 are flowcharts showing a voice output process by communication system 100. As shown in FIGS. 2 and 3, each step of the flowchart is executed by any one of in-vehicle controller 12, remote controller 26, or server controller 33.

[0063] In step S101, the in-vehicle controller 12 judges whether or not the use of the communication system 100 has started. For example, the use of the communication system 100 is started when the remote user B wearing the VR goggles performs an operation to start the communication system 100. When information on the operation by the remote user B is transmitted from the remote space 20 to the vehicle 1 and input to the in-vehicle controller 12, the in-vehicle controller 12 judges that the use of the communication system 100 has started. If the in-vehicle controller 12 judges that the use of the communication system 100 has started, the process proceeds to step S102. On the other hand, if the information on the operation by the remote user B is not input to the in-vehicle controller 12 and ... is not started, the process proceeds to step S102.

[0064] In step S102, the in-vehicle controller 12 determines whether or not it has acquired collected sound collected by the in-vehicle microphone 11. If collected sound data is input from the in-vehicle microphone 11, the in-vehicle controller 12 determines that it has acquired collected sound, and if collected sound data is not input from the in-vehicle microphone 11, it determines that it has not acquired collected sound. If the in-vehicle controller 12 makes a positive determination, the process proceeds to step S103, and if the in-vehicle controller 12 makes a negative determination, the process returns to step S101.

[0065] If the vehicle controller 12 makes a positive determination in step S102, the process proceeds to step S103. In step S103, the vehicle controller 12 transmits the collected voice data acquired in step S102 to the remote space 20 via the vehicle communication device 14.

[0066] In step S104, the remote controller 26 receives the collected voice data transmitted in step S103 from the in-vehicle communication device 14 via the remote communication device 21. In step S105, the remote controller 26 specifies a sample voice that is the same as or similar to the collected voice received in step S104 as a target sample voice.

[0067] In step S106, the remote controller 26 acquires the setting information of the voice processing mode input by the remote user B to the remote input device 24, and determines whether the voice processing mode is the importance mode. If the remote controller 26 determines that the voice processing mode is an importance mode, the process proceeds to step S107, and if the remote controller 26 determines that the voice processing mode is an importance mode, the process proceeds to step S108.

[0068] If the remote controller 26 makes an affirmative determination in step S106, the process proceeds to step S107. In step S107, the remote controller 26 determines whether the importance of the collected sound received in step S104 is higher than a predetermined second threshold value.

[0069] For example, the remote controller 26 executes a voice recognition process on the collected voice to distinguish each voice included in the collected voice. The remote controller 26 compares the importance of each voice included in the collected voice with a predetermined second threshold. If the importance of at least one voice among the voices included in the collected voice is higher than the predetermined second threshold, the remote controller 26 determines that the importance of the collected voice is higher than the predetermined second threshold. On the other hand, if the importance of all voices included in the collected voice is lower than the predetermined second threshold, the remote controller 26 determines that the importance of the collected voice is lower than the predetermined second threshold. If the remote controller 26 makes a positive determination, the process proceeds to step S114, and if the remote controller 26 makes a negative determination, the process proceeds to step S108.

[0070] In step S107, if the remote controller 26 makes a negative determination, the process proceeds to step S108. In step S108, the remote controller 26 determines whether the voice processing mode is the favorable level mode. If the remote controller 26 makes a positive determination, the process proceeds to step S109, and if the remote controller 26 makes a negative determination, the process proceeds to step S115.

[0071] If the remote controller 26 makes an affirmative determination in step S108, the process proceeds to step S109. In step S109, the remote controller 26 transmits data of the target sample voice identified in step S105 (target sample voice data) to the server 30 via the remote communication device 21.

[0072] In step S110, the server controller 33 receives the target sample voice data transmitted in step S109 from the remote communication device 21 via the server communication device 31. In step S111, the server controller 33 estimates the remote user B's likeability rating for the collected voice based on the remote user B's likeability rating for the sample voice stored in the database 32.

[0073] In step S112, the server controller 33 transmits information on the remote user B's favorability rating for the collected sound (the estimation result of the favorability rating estimated in step S111) to the remote space 20 via the server communication device 31.

[0074] In step S113, the remote controller 26 receives, via the remote communication device 21, the information on the estimation result of the favorability rating transmitted from the server communication device 31 in step S112.

[0075] In step S114, the remote controller 26 executes a voice processing process on the collected voice data received in step S104. In this step, the remote controller 26 executes a voice processing process in which the voice to be processed and the processing method differ according to the set voice processing mode.

[0076] When the process proceeds from step S107 to step S114, the remote controller 26 executes an emphasis process for emphasizing the voice determined in step S107 to have a higher importance than the predetermined second threshold, and generates a processed voice by processing the collected voice. On the other hand, when the process proceeds from step S113 to step S114, the remote controller 26 executes an emphasis process for emphasizing or a suppression process for suppressing the collected voice received in step S104 according to the remote user B's favorability rating for the collected voice received in step S113, and generates a processed voice by processing the collected voice.

[0077] In step S115, the remote controller 26 outputs the voice to the remote user B via the remote output device 25. The remote controller 26 outputs different voices to the remote user B according to the set voice processing mode.

[0078] When the voice processing mode is set to the preference mode, the remote controller 26 outputs the processed voice in which the collected voice is emphasized or suppressed in step S114 to the remote output device 25. This makes it possible to automatically output to the remote user B the processed voice that reflects the preferences of the remote user B, unlike the collected voice.

[0079] Furthermore, when the voice processing mode is set to the importance mode and the voice processing is executed in step S114, the remote controller 26 outputs, as the processed voice, a processed voice in which voices having importance higher than a predetermined second threshold value are emphasized with respect to the collected voice, to the remote output device 25. This makes it possible to automatically output to the remote user B a processed voice in which voices having importance higher than a predetermined second threshold value are emphasized, unlike the collected voice.

[0080] Furthermore, if the voice processing mode is not set to either the likeability mode or the importance mode, the remote controller 26 outputs the collected voice received in step S104 to the remote user B. This allows the voice collected at the location where the in-vehicle user A is located to be output to the remote user B as is.

[0081] When the process of step S115 ends, the process returns to step S101, and the voice output process by the communication system 100 shown in FIG. 2 and FIG. 3 is repeatedly executed until a negative determination is made in step S102.

[0082] In this way, according to the voice output process by the communication system 100 of the present embodiment, different voices can be output to the remote user B depending on the setting of the voice processing mode, the type of voice included in the collected voice, the importance of the collected voice, and the likeability of the collected voice by the remote user B. In addition, when the voice processing mode is set to the likeability mode or the importance mode and a processed voice in which a part or all of each voice included in the collected voice is processed is output to the remote user B, the emphasized voice among the voices included in the collected voice becomes easy for the remote user B to hear, and the suppressed voice among the voices included in the collected voice becomes difficult for the remote user B to hear. In other words, according to the voice output process by the communication system 100 of the present embodiment, a voice can be selected from each voice included in the collected voice and played to the remote user B.

[0083] As described above, in this embodiment, the in-vehicle controller 12 includes a voice acquisition unit 13 that acquires the collected voice collected by the in-vehicle microphone 11 at the location where the in-vehicle user A is located, the server 30 includes a database 32 that stores the likeability of the remote user B, who is located at a location other than the location where the in-vehicle user A is located, for a predetermined sample voice, and a likeability estimation unit 34 that estimates the likeability of the remote user B for the collected voice based on the likeability of the remote user B for the sample voice, and the remote controller 26 includes a voice processing unit 28 that generates a processed voice by processing the collected voice by emphasizing or suppressing the collected voice according to the likeability of the remote user B for the collected voice, and a voice output unit 29 that outputs the processed voice to the remote output device 25 that outputs the voice to the remote user B. As a result, the processed voice processed according to the likeability of the remote user B for the collected voice can be output to the remote user B, so that the voice according to the user's preferences can be automatically output. For example, for a user who does not like deep bass engine sounds, it is possible to automatically output processed sound in which the engine sound of the vehicle 1 is selectively cancelled. As a result, it is possible to reduce the number of occasions where the user needs to adjust the sound to suit his / her own preferences.

[0084] In the present embodiment, the likability estimation unit 34 estimates the likability of the remote user B for a given sample voice based on the expression of the remote user B when listening to the sample voice. This allows the likability of the user for the sample voice to be estimated with high accuracy.

[0085] Furthermore, in this embodiment, the remote controller 26 includes a target sample voice identification unit 27 that identifies a sample voice identical or similar to the collected voice as a target sample voice, and the likability estimation unit 34 estimates the likability of the remote user B for the collected voice based on the likability of the remote user B for the target sample voice. Since the user's likability for the collected voice is estimated from the user's likability for a sample voice identical or similar to the collected voice, the user's likability for the collected voice can be estimated with high accuracy.

[0086] Additionally, in this embodiment, the in-vehicle user A is a user who is in the vehicle 1, and the remote user B is a user who is in a remote space 20 that is located away from the vehicle 1. This allows automatic output of audio that matches the preferences of the listener, even in a situation where a conversation is held with a user who is in a space where various types of audio are played, such as the interior of a vehicle. As a result, audio that matches the preferences of the listener, can be automatically output, regardless of the sound field environment where the user is located.

[0087] In addition, in this embodiment, when the remote user B's favorability rating for the collected voice is higher than a predetermined first threshold, the voice processing unit 28 processes the collected voice so that the processed voice is emphasized relatively compared to the collected voice. This allows the user to hear the processed voice in which the user's preferred voice is emphasized, thereby reducing the opportunity for the user to adjust the voice and improving user convenience.

[0088] Furthermore, in this embodiment, when a preference level mode in which the collected voice is processed according to the preference level of the remote user B for the collected voice is set as the voice processing mode for processing the collected voice, the voice output unit 29 outputs the processed voice according to the preference level of the remote user B for the collected voice to the remote output device 25. This allows the user to hear the processed voice or the collected voice depending on whether the voice processing mode is set to the preference level mode. The user can hear the actual voice, not limited to the voice processed according to his / her preference. For example, even if the listener does not like the engine sound of a vehicle or the horn sound of another vehicle, it is possible to prevent the listener from falling into a state where he / she cannot grasp the situation of the conversation partner when some kind of trouble occurs in the vehicle in which the conversation partner is riding.

[0089] In addition, in this embodiment, when the importance of the collected voice is higher than a predetermined second threshold, the voice processing unit 28 processes the collected voice so that the processed voice is emphasized compared to the collected voice. This allows the user to hear sounds that the user cannot ignore, such as the siren of an emergency vehicle, in an emphasized state, preventing the user from falling into a state where he or she cannot grasp the surrounding situation of the conversation partner.

[0090] In addition, in this embodiment, when an importance mode in which the collected voice is processed according to the importance of the collected voice is set as the voice processing mode for processing the collected voice, the voice output unit 29 outputs the processed voice according to the importance of the collected voice to the remote output device 25. This allows the user to hear the processed voice or the collected voice depending on whether the voice processing mode is set to the importance mode. Since the user can hear the actual voice, not limited to the voice processed according to the importance of the voice, it is possible to prevent the user from falling into a state where he / she cannot grasp the situation of the conversation partner.

[0091] Furthermore, in this embodiment, the expression mode of the remote user B includes at least one of the facial expression of the remote user B when listening to a predetermined sample voice, the sound quality of the voice uttered by the remote user when listening to the sample voice, and the contents of the voice uttered by the remote user when listening to the sample voice. This makes it possible to estimate the user's liking for the sample voice based on the user's reflexive reaction when listening to the sample voice, thereby improving the accuracy of estimating the user's liking for the sample voice.

[0092] In addition, in this embodiment, when the expression mode of the remote user B when listening to a predetermined sample voice is input, the likability rating estimation unit 34 estimates the likability rating of the remote user B for the sample voice using the trained neural network. The trained neural network is a neural network that has been subjected to machine learning processing using training data so as to estimate the likability rating of the remote user B for the sample voice when the expression mode of the remote user B when listening to the sample voice is input. This makes it possible to estimate the user's likability rating for the sample voice with higher accuracy than estimating the user's likability rating for the sample voice based on rules.

[0093] The above-described embodiments are described for the purpose of facilitating understanding of the present invention, and are not described for the purpose of limiting the present invention. Therefore, the elements disclosed in the above embodiments are intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0094] For example, in the above embodiment, the voice processing mode is set to the likeability mode or the importance mode as an alternative example, but the present invention is not limited to this, and the voice processing mode may be set to the likeability mode and the importance mode. In this case, the voice processing unit 28 generates a processed voice by emphasizing or suppressing the collected voice according to the likeability of the remote user B to the collected voice and the importance of the collected voice. This allows the user to hear a voice according to the user's state or situation, and also to hear a voice that reflects the user's preferences. For example, if the turn signal sound of the vehicle 1 is unnecessary for the remote user B, and the music played in the vehicle 1 is preferred by the remote user B, a processed voice in which the turn signal sound is suppressed and the music is emphasized can be output to the remote user B.

[0095] For example, in the above embodiment, the estimation method using a trained neural network has been described as an example of the method of estimating the likeability of the sample voice by the likeability estimation unit 34, but the likeability of the sample voice may be estimated by other methods. For example, the likeability estimation unit 34 may include a neural network including an input layer and an output layer, in which input data to the input layer is the expression manner of the remote user B when listening to the sample voice and output data from the output layer is the likeability of the remote user B for the sample voice, a machine learning unit that trains the neural network using the actual values ​​of the input data and the output data as teacher data, and an estimation unit that inputs new input data to the neural network trained by the machine learning unit and estimates the likeability of the remote user B for the sample voice as new output data for the input data. As a result, the likeability estimation unit 34 learns the expression manner of the remote user B every time the expression manner of the remote user B when listening to the sample voice is input, so that the estimation accuracy of the likeability of the remote user B for the sample voice can be further improved.

[0096] In addition, for example, in the above-mentioned embodiment, the case where the collected voice collected at the location where the in-vehicle user A is present is output to the remote user B in the remote space 20 has been described as an example, but the present invention is not limited thereto. For example, the voice processing device and the voice processing method of the present invention can be applied even when the collected voice collected at the location where the remote user B is present is output to the in-vehicle user A in the in-vehicle space 10. In this case, the database 32 stores the in-vehicle user A's favorability rating for a predetermined sample voice estimated by the favorability rating estimation unit 34, and the favorability rating estimation unit 34 estimates the in-vehicle user A's favorability rating for the collected voice based on the in-vehicle user A's favorability rating for the target sample voice. The remote controller 26 executes a voice processing process for the collected voice according to the in-vehicle user A's favorability rating for the collected voice, and the processed voice is transmitted to the vehicle 1. Then, the in-vehicle controller 12 outputs the processed voice to an output device such as a speaker mounted on the vehicle 1.

[0097] For example, in the above-mentioned embodiment, the vehicle controller 12 includes the voice acquisition unit 13, the server 30 includes the database 32 and the favorability rating estimation unit 34, and the remote controller 26 includes the target sample voice identification unit 27, the voice processing unit 28, and the voice output unit 29. However, the configuration of the voice processing device of the present invention is not limited to this. For example, the voice acquisition unit, the memory unit, the favorability rating estimation unit, the target sample voice identification unit, the voice processing unit, and the voice output unit may be configured to be included in a specific controller or device. For example, the vehicle controller 12 may include these blocks, the remote controller 26 may include these blocks, or the server 30 may include these blocks. In the case of such a configuration in which each block of the voice processing device is aggregated, a device including each block can transmit and receive data between a device that does not include any of the blocks via a communication device, thereby obtaining the same effect as the above-mentioned embodiment.

[0098] For example, in the above embodiment, voice emphasis and suppression are described as an example of the voice processing method of the present invention, but other voice processing methods may be used. Voice processing may also be performed by combining parameters other than the likeability rating and importance. For example, in the emphasis processing, stereophonic processing may be added so that a user who hears the voice can recognize the direction of the sound source of the voice and the distance to the sound source.

[0099] Also, for example, instead of the emphasis process and suppression process that assume the output of the voice, a process of eliminating the voice itself may be performed. In this case, it is preferable to perform this process on voices that are considered to have a particularly low importance. Also, whether or not the voice is output may be determined according to the state of the user. For example, when the listener is sleeping or talking to a user other than the speaker, the voice may be processed so that it is not output regardless of the user's favorability rating for the collected voice or the importance of the collected voice. In this case, the state of the user can be grasped from a camera or the like.

[0100] Also, for example, the processing may be a sound processing for adding a sound different from the collected sound. For example, the processing may be a sound processing for adding a sound effect indicating that a user or a device has operated to the collected sound. For example, a sound effect that exaggerates that in-vehicle user A is operating the air conditioner may be added to the collected sound when the air conditioner mounted in the vehicle 1 is operated. [Explanation of symbols]

[0101] 100…Communication Systems 10…Interior space 11…Car microphone 11a…Car microphone 11b…Exterior microphone 12…In-vehicle controller 13…Audio acquisition section 14...In-vehicle communication device 20...Remote Space 21...Remote communication equipment 22...Remote camera 23…Remote microphone 24...Remote input device 25...Remote output device 26...Remote controller 27…Target sample voice identification section 28…Audio processing section 29…Audio output section 30…Server 31...Server communication device 32…Database 33…Server controller 34…Likeability estimation section

Claims

1. a voice acquisition unit that acquires collected voice collected by a sound collection device at a location where the first user is located; A storage unit that stores a favorability rating of a second user located at a location different from the location of the first user for a predetermined sample voice; a preference rating estimation unit that estimates a preference rating of the second user for the collected voice based on the preference rating of the second user for the sample voice; a sound processing unit that generates a processed sound by processing the collected sound by emphasizing or suppressing the collected sound according to a favorability rating of the second user for the collected sound; an audio output unit that outputs the processed audio to an output device that outputs audio to the second user; The voice processing unit processes the collected voice so that the processed voice is relatively emphasized compared to the collected voice when the second user's likability for the collected voice is higher than a predetermined first threshold.

2. The voice processing device according to claim 1 , wherein the likability estimation unit estimates the second user's likability for the sample voice based on an expression mode of the second user when listening to the sample voice.

3. a target sample voice identification unit that identifies the sample voice that is the same as or similar to the collected voice as a target sample voice; The voice processing device according to claim 1 , wherein the likability rating estimation unit estimates the second user's likability rating for the collected voice based on the second user's likability rating for the target sample voice.

4. The first user is an in-vehicle user who is in a vehicle, 4. The voice processing device according to claim 1, wherein the second user is a remote user located at a location away from the vehicle.

5. The voice processing unit generates the processed voice by emphasizing or suppressing the collected voice according to the second user's likability for the collected voice and the importance of the collected voice set according to the content of the collected voice. The voice processing device described in any one of claims 1 to 4.

6. The voice processing device according to claim 5 , wherein the voice processing unit processes the collected voice so that the processed voice is relatively emphasized compared to the collected voice when the importance of the collected voice is higher than a predetermined second threshold value.

7. The voice processing device of claim 2, wherein the expression manner of the second user includes at least one of the facial expression of the second user when listening to the sample voice, the sound quality of the voice uttered by the second user when listening to the sample voice, and the content of the speech uttered by the second user when listening to the sample voice.

8. the likability estimation unit, when receiving the expression manner of the second user when listening to the sample voice, estimates a likability rating of the second user for the sample voice using a trained neural network; The voice processing device according to claim 2 or 7, wherein the trained neural network is a neural network that has been subjected to machine learning processing using training data so as to estimate the second user's liking for the sample voice when the expression manner of the second user when listening to the sample voice is input.

9. The likability estimation unit is a neural network including an input layer and an output layer, the input data to the input layer being the expression manner of the second user when listening to the sample voice, and the output data from the output layer being the favorability rating of the second user for the sample voice; a machine learning unit that trains the neural network using actual values ​​of the input data and the output data as training data; The voice processing device according to claim 2 or 7, further comprising an estimation unit that inputs new input data into the neural network trained by the machine learning unit and estimates the second user's liking for the sample voice as new output data corresponding to the input data.

10. A voice processing method executed by a controller, comprising: The controller: Acquire a collected voice collected by a sound collecting device at a location where the first user is present; storing a favorability rating of a second user located at a location different from the location of the first user for a predetermined sample voice; estimating a favorable rating of the second user for the collected voice based on the favorable rating of the second user for the sample voice; generating a processed sound by processing the collected sound by emphasizing or suppressing the collected sound according to a favorable rating of the second user for the collected sound; outputting the processed voice to an output device that outputs voice to the second user; The voice processing method further comprises processing the collected voice so that the processed voice is emphasized relatively compared to the collected voice when the second user's favorability rating for the collected voice is higher than a predetermined first threshold.

11. A voice acquisition unit that acquires collected voice collected by a sound collection device at a location where a first user is present; A storage unit that stores a favorability rating of a second user located at a location different from the location of the first user for a predetermined sample voice; a preference rating estimation unit that estimates a preference rating of the second user for the collected voice based on the preference rating of the second user for the sample voice; a sound processing unit that generates a processed sound by processing the collected sound by emphasizing or suppressing the collected sound according to a favorability rating of the second user for the collected sound; an audio output unit that outputs the processed audio to an output device that outputs audio to the second user; The voice processing unit generates the processed voice by emphasizing or suppressing the collected voice according to a favorability rating of the second user for the collected voice and an importance of the collected voice set according to a content of the collected voice, The audio processing unit processes the collected audio so that the processed audio is emphasized relatively compared to the collected audio when the importance of the collected audio is higher than a predetermined second threshold.

12. A voice acquisition unit that acquires collected voice collected by a sound collection device at a location where a first user is present; A storage unit that stores a favorability rating of a second user located at a location different from the location of the first user for a predetermined sample voice; a preference rating estimation unit that estimates a preference rating of the second user for the collected voice based on the preference rating of the second user for the sample voice; a sound processing unit that generates a processed sound by processing the collected sound by emphasizing or suppressing the collected sound according to a favorability rating of the second user for the collected sound; an audio output unit that outputs the processed audio to an output device that outputs audio to the second user; The likability estimation unit estimates a likability level of the second user for the sample voice based on an expression mode of the second user when listening to the sample voice, A voice processing device, wherein the expression manner of the second user includes at least one of the facial expression of the second user when listening to the sample audio, the sound quality of the voice uttered by the second user when listening to the sample audio, and the content of the speech uttered by the second user when listening to the sample audio.

Images