Conversation support device and conversation support method

Abstract

To support a conversation between users of a plurality of mobile bodies.SOLUTION: A controller 20 being a conversation support device for respective users boarding on a plurality of mobile bodies includes: a user voice acquisition section 21, a peripheral voice acquisition section 22; a voice output control section 39 for controlling output of user voice; an other-user recognition section 31 for recognizing another user positioned in the periphery of the mobile body and boarding on another mobile body; a peripheral state acquisition section 32; a distance determination section 33 for determining whether a distance between the mobile body and the other user is equal to or less than a distance threshold; and a peripheral state determination section 34 for determining whether or not a present position of the mobile body is suitable for a conversation between the users boarding on the mobile bodies based on whether or not the present position is in an area with noise regulation or whether or not in an area crowded with people. The voice output control section 39 performs control to output the user voice by changing a voice output mode based on a determination result of the peripheral state determination section.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a conversation support device and a conversation support method. [Background technology]

[0002] There is known a technology for supporting conversations between passengers in a vehicle (see, for example, Patent Document 1). In the technology described in Patent Document 1, a speaker and a microphone are provided for each passenger to support the conversation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-068237 A Summary of the Invention [Problem to be solved by the invention]

[0004] When users of the same group travel on moving objects that are not covered by a body frame, such as multiple electric kickboards, they may bring the moving objects close to each other and talk to each other. If the noise around the moving objects is loud, it is difficult to hear the users' voices. If users talk loudly, the sound may leak to the surroundings and the users' voices may become noise. When using an intercommunication system, it may be bothersome to wear the system. Compared to talking directly, the way the voices sound when talking through an intercommunication system may be different, which may cause a sense of discomfort. As such, there is room for improvement in supporting conversations between users of multiple moving objects.

[0005] The present disclosure has been made in consideration of the above, and aims to support conversations between users of multiple mobile objects. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, a conversation support device according to the present disclosure is a conversation support device between users who are respectively riding on multiple moving bodies, and includes a user voice acquisition unit that acquires user voice from a user sound collection unit that collects voices of users riding on the moving bodies, a peripheral sound acquisition unit that acquires peripheral sound from a peripheral sound collection unit that collects sound around the moving body, a sound output control unit that controls output of the user voice from a sound output unit that can output the user voice from the moving body to the surroundings, a different user recognition unit that recognizes other users riding on other moving bodies located in the vicinity of the moving body, and a different user recognition unit that recognizes other users riding on other moving bodies located in the vicinity of the moving body. The device comprises a surrounding situation acquisition unit that acquires the surrounding situation, a distance determination unit that determines whether the distance between the moving body and the other user recognized by the other user recognition unit is equal to or less than a distance threshold, and a surrounding situation determination unit that determines whether the current location of the moving body is unsuitable for conversation between users on board the moving body depending on whether the current location is in an area where noise is regulated or an area where people are densely populated, based on the surrounding situation acquired by the surrounding situation acquisition unit, and the audio output control unit controls to change the audio output form and output user voice based on the determination result of the surrounding situation determination unit.

[0007] The conversation assistance method according to the present disclosure is a conversation assistance method between users who are respectively aboard a plurality of moving bodies, and includes a user voice acquisition step of acquiring user voice from a user sound collection unit that collects voice of the user aboard the moving body, a surrounding voice acquisition step of acquiring surrounding voice from a surrounding sound collection unit that collects voice around the moving body, a voice output control step of controlling output of the user voice from a voice output unit that can output the user voice from the moving body to the surroundings, a other user recognition step of recognizing other users aboard other moving bodies located in the vicinity of the moving body, and a surrounding situation acquisition step of acquiring the surrounding situation of the moving body. The method includes a step of determining whether the distance between the moving body and the other user recognized by the other user recognition step is equal to or less than a distance threshold, and a surrounding situation determination step of determining whether the current location of the moving body is unsuitable for conversation between users on board the moving body depending on whether the current location is in an area where noise is regulated or an area where people are densely populated, based on the surrounding situation acquired by the surrounding situation acquisition step, and the audio output control step controls to change the audio output form and output user voice based on the determination result of the surrounding situation determination step. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an advantage of being able to support conversations between users of multiple mobile objects. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing a state in which a conversation support system according to a first embodiment is being used. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a conversation support system including a conversation support device according to the first embodiment. [Diagram 3] FIG. 3 is a flowchart showing a flow of processing performed by the conversation support device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of a conversation support system including a conversation support device according to the second embodiment. [Diagram 5] FIG. 5 is a flowchart showing a flow of processing performed by the conversation support device according to the second embodiment. [Figure 6] FIG. 6 is a flowchart showing a flow of processing performed by the conversation support device according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. An embodiment of a conversation support device and a conversation support method according to the present disclosure will be described in detail. Note that the present invention is not limited to the following embodiments.

[0011] [First embodiment] (Conversation support system) Fig. 1 is a schematic diagram showing a state in which a conversation support system according to a first embodiment is being used. The conversation support system 10 supports conversations between users who are riding on multiple moving objects. In Fig. 1, the conversation support system 10 is disposed on each of the kick scooters on which user U1 is riding and the kick scooter on which user U2 is riding.

[0012] A mobile body is a vehicle in which the portion on which a passenger sits is not enclosed by a body frame. Examples of mobile bodies include bicycles, micromobility vehicles such as kickboards, and wheelchairs.

[0013] 2 is a block diagram showing a configuration example of a conversation support system including a conversation support device according to the first embodiment. The conversation support system 10 includes a user microphone 11, a front microphone (surrounding sound collection unit) 12, a rear microphone (surrounding sound collection unit) 13, a camera unit 14, a sensor unit 15, a GNSS (Global Navigation Satellite System) receiver 16, a map information storage unit 17, a front speaker (audio output unit) 18, a rear speaker (audio output unit) 19, and a control unit (conversation support device) 20.

[0014] The user microphone 11 is a microphone that collects the voice of a user riding on a moving object. The user microphone 11 is arranged, for example, facing the user's face. The user microphone 11 collects the user's voice when the user talks with another user. The user microphone 11 may start collecting the user's voice, for example, when an operation unit (not shown) receives an ON operation. The user microphone 11 outputs the collected user voice to the user voice acquisition unit 21.

[0015] The front microphone 12 is a microphone that picks up environmental sounds, which are sounds around the moving body. The front microphone 12 picks up sounds in front of the moving body. The front microphone 12 is arranged facing the front of the moving body. The front microphone 12 outputs the picked up ambient sounds to the ambient sound acquisition unit 22.

[0016] The rear microphone 13 is a microphone that picks up environmental sounds, which are sounds around the moving body. The rear microphone 13 picks up sounds behind the moving body. The rear microphone 13 is disposed facing the rear of the moving body. The rear microphone 13 outputs the picked up ambient sounds to the ambient sound acquisition unit 22.

[0017] The camera unit 14 has multiple cameras that capture images of the periphery of the moving object. The multiple cameras capture images in different directions relative to the moving object. In this embodiment, the camera unit 14 has a front camera that can capture images in front and a rear camera that can capture images in the rear. The front camera is disposed, for example, in front of the moving object. The rear camera is disposed, for example, behind the moving object. In the following description, when it is not necessary to distinguish between the multiple cameras, they will be described as the camera unit 14. The camera unit 14 outputs captured image data to the image acquisition unit 23.

[0018] The sensor unit 15 includes a plurality of sensors installed around the moving object. The sensor unit 15 is capable of detecting a person present in the vicinity of the moving object. In this embodiment, the sensor unit 15 is provided with a front sensor, a rear sensor, a right side sensor, and a left side sensor. Since each sensor has the same configuration, only the front sensor will be described, and the description of the other sensors will be omitted.

[0019] The forward sensor is disposed in front of the moving body and detects a person in front of the moving body. The forward sensor is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or a combination of these. The forward sensor outputs forward sensor data, which is the detection result, to the sensor data acquisition unit 24.

[0020] The GNSS receiving unit 16 is configured with a GNSS receiver that receives GNSS signals from GNSS satellites, etc. The GNSS receiving unit 16 outputs the received GNSS signals to the position information acquisition unit 25.

[0021] The map information storage unit 17 stores map information including roads. The map information storage unit 17 is, for example, a semiconductor memory element such as a ROM (Read Only Memory) or a flash memory, or a recording unit such as a memory card. Alternatively, the map information storage unit 17 may be an external recording unit wirelessly connected via a communication device (not shown).

[0022] The map information may include, for example, information indicating noise-regulated areas or information indicating areas where people are concentrated. The noise-regulated areas may include areas where silence is particularly required, such as areas where medical facilities or social welfare facilities are concentrated. The areas where people are concentrated may include areas where people are particularly concentrated, such as shopping streets or commercial areas in downtown areas.

[0023] The front speaker 18 is an audio output unit that outputs user voice in the conversation support system 10. The front speaker 18 can output user voice from the moving body toward the surroundings. The front speaker 18 is arranged facing the front of the moving body. The front speaker 18 outputs user voice picked up by the user microphone 11 to other users in front of the moving body. The front speaker 18 outputs user voice based on an audio signal from the audio output control unit 39.

[0024] The rear speaker 19 is an audio output unit that outputs user voice in the conversation support system 10. The rear speaker 19 can output user voice from the moving body toward the surroundings. The rear speaker 19 is arranged facing the rear of the moving body. The rear speaker 19 outputs user voice picked up by the user microphone 11 to other users behind the moving body. The rear speaker 19 outputs user voice based on an audio signal from the audio output control unit 39.

[0025] The volume of the user's voice output from the front speaker 18 and the rear speaker 19 is approximately the same as the volume of the user's voice heard by other users when the user talks with another user at a distance of about 1 meter.

[0026] The effective distances that the sounds output from the front speaker 18 and the rear speaker 19 can reach are stored in advance in a storage unit (not shown). The effective distances are about several meters. The effective distances of the front speaker 18 and the rear speaker 19 may be the same or different. In the following description, it is assumed that they are the same.

[0027] (Conversation support device) The control unit 20 controls the conversation support system 10. The control unit 20 controls the output of the user voice from the front speaker 18 and the rear speaker 19. The control unit 20 is an arithmetic processing device constituted by, for example, a CPU (Central Processing Unit) and the like. The control unit 20 loads a program stored in a storage unit (not shown) into a memory and executes instructions included in the program. The control unit 20 includes an internal memory (not shown) that is used for temporary storage of data, etc. The control unit 20 includes a user voice acquisition unit 21, a surrounding voice acquisition unit 22, a video acquisition unit 23, a sensor data acquisition unit 24, a position information acquisition unit 25, a map information acquisition unit 26, a mobile object information acquisition unit 27, a different user recognition unit 31, a surrounding situation acquisition unit 32, a distance determination unit 33, a surrounding situation determination unit 34, and a voice output control unit 39.

[0028] The user voice acquisition unit 21 acquires the user voice that is collected by the voice of the user who is riding on the moving object. The user voice acquisition unit 21 acquires the user voice that is collected by the user microphone 11.

[0029] The ambient sound acquisition unit 22 acquires ambient sound data by collecting ambient sounds. The ambient sound acquisition unit 22 acquires ambient sounds collected by the front microphone 12 and the rear microphone 13.

[0030] The image acquisition unit 23 acquires the image captured by the camera unit 14 .

[0031] The sensor data acquisition unit 24 acquires the sensor data acquired by the sensor unit 15 .

[0032] The position information acquisition unit 25 acquires position information indicating the current position of the mobile object. The position information acquisition unit 25 calculates the position information of the current position of the mobile object based on the radio waves received by the GNSS receiving unit 16 by a known method.

[0033] The map information acquisition unit 26 acquires map information from the map information storage unit 17 .

[0034] The mobile object information acquisition unit 27 acquires mobile object information from the mobile object via the CAN or the like.

[0035] The moving object information includes, for example, information indicating the speed of the moving object.

[0036] The other user recognition unit 31 recognizes other users who are conversation partners of the user. The other user recognition unit 31 recognizes other users who are located around the moving object and are riding on other moving objects. The other user recognition unit 31 recognizes other users, for example, by performing image processing on the image acquired by the video acquisition unit 23. The other user recognition unit 31 recognizes other users, for example, based on sensor data acquired by the sensor data acquisition unit 24. The method of recognizing other users can be any known method and is not limited.

[0037] The other user recognition unit 31 recognizes whether or not a moving object on which another user is riding is equipped with the conversation support system 10. For example, the other user recognition unit 31 recognizes that the moving object is equipped with the conversation support system 10 by recognizing a predetermined marker affixed to the moving object from an image acquired by the video acquisition unit 23. For example, the other user recognition unit 31 recognizes that the moving object is equipped with the conversation support system 10 by recognizing a predetermined shape arranged on the moving object from sensor data acquired by the sensor data acquisition unit 24.

[0038] The other user recognition unit 31 further recognizes the direction, distance, and speed of the recognized other users as viewed from the moving object.

[0039] The surrounding situation acquisition unit 32 acquires the surrounding situation of the mobile object. The surrounding situation acquisition unit 32 acquires whether the current position of the mobile object is an area where noise is regulated or an area where people are densely packed as the surrounding situation. The surrounding situation acquisition unit 32 may acquire information indicating a noise regulated area or an area where people are densely packed, which is linked to the current position of the mobile object, based on map information acquired by the map information acquisition unit 26, for example. The surrounding situation acquisition unit 32 may recognize the level of surrounding noise from the surrounding sound acquired by the surrounding sound acquisition unit 22, and acquire whether the area is an area where noise is regulated or an area where people are densely packed. The surrounding situation acquisition unit 32 may recognize people and acquire information indicating an area where people are densely packed, for example, from an image acquired by the video acquisition unit 23 or sensor data acquired by the sensor data acquisition unit 24. The surrounding situation acquisition unit 32 may acquire attributes of the area or area at the current position of the mobile object as the surrounding situation or surrounding information.

[0040] The area attribute refers to a specific area, such as a residential area, a commercial area, a downtown area, a theme park, or the like.

[0041] The distance determination unit 33 determines whether the distance between the moving body and the other user recognized by the other user recognition unit 31 is equal to or less than the distance threshold. The distance determination unit 33 determines whether the distance between the moving body and the other user recognized by the other user recognition unit 31 is equal to or less than the effective distance of the front speaker 18 or the rear speaker 19 used for conversation. In this embodiment, when the distance determination unit 33 determines that the distance between the moving body and the other user is equal to or less than the distance threshold, the conversation between the users is validated. When the distance determination unit 33 determines that the distance between the moving body and the other user is farther than the distance threshold, the conversation between the users is invalidated. More specifically, when the distance determination unit 33 determines that the distance between the moving body and the other user is equal to or less than the distance threshold, the surrounding situation determination unit 34 performs a determination as a subsequent process.

[0042] The distance threshold is a distance at which the sound output from the front speaker 18 and the rear speaker 19 can be heard by other users in the front or rear. The distance threshold is, for example, approximately the same as the effective distance of the front speaker 18 and the rear speaker 19, and is set to about several meters. The distance threshold is, for example, a distance at which feedback does not occur between the microphone and the speaker.

[0043] Distance determination unit 33 may further determine, based on the speed of the moving body and the speed of the other user, whether the distance between the moving body and the other user during a predetermined period of about several seconds corresponding to the time for conversation is equal to or less than the effective distance of front speaker 18 or rear speaker 19 used for conversation. This is to prevent the conversation from being erroneously validated when the difference in speed is large.

[0044] The surrounding situation determination unit 34 determines whether the situation around the moving object is not suitable for conversation between users aboard the moving object. The surrounding situation determination unit 34 determines whether the situation around the moving object is not suitable for conversation between users aboard the moving object depending on whether the surrounding area is a noise regulated area or a crowded area, based on the surrounding situation acquired by the surrounding situation acquisition unit 32. When the surrounding area of ​​the moving object is a noise regulated area or a crowded area, the surrounding situation determination unit 34 determines that the situation around the moving object is not suitable for conversation between users aboard the moving object.

[0045] In this embodiment, the surrounding situation determination unit 34 may determine whether or not the moving body and another moving body have a predetermined positional relationship.

[0046] The predetermined positional relationship is a positional relationship in which the sounds output from the front speaker 18 and the rear speaker 19 can be heard by other users in front or behind. The predetermined positional relationship is, for example, a positional relationship in which the moving body and another moving body are lined up front-to-back or left-to-right at a distance equal to or less than a distance threshold.

[0047] The audio output control unit 39 controls the output of the user voice from the front speaker 18 or the rear speaker 19. The audio output control unit 39 controls the user voice to be output from the front speaker 18 or the rear speaker 19. The audio output control unit 39 controls the user voice to be output from the front speaker 18 or the rear speaker 19 facing the direction in which the other user is recognized. The audio output control unit 39 controls the user voice to be output by changing the audio output form based on the determination result of the surrounding situation determination unit 34.

[0048] The change in the audio output mode is, for example, a change in volume, a change in sound quality, a change in voice quality, etc. In this embodiment, a change in volume will be described as an example of a change in the audio output mode.

[0049] In this embodiment, the audio output control unit 39 controls the user voice to be output at a minimum volume or muted when the current location of the mobile object is in an area where noise is regulated or in an area where people are densely populated. In this embodiment, the audio output control unit 39 controls the user voice to be output at a normal volume when the current location of the mobile object is not in an area where noise is regulated or is not in an area where people are densely populated.

[0050] In this embodiment, when the surrounding situation determination unit 34 determines that the moving body and another moving body are in a predetermined positional relationship, the audio output control unit 39 may control the front speaker 18 and the rear speaker 19 to change their directivity and output the user's voice.

[0051] The predetermined positional relationship is, for example, a positional relationship in which the moving body and another moving body are lined up in twos, front to back or left to right, at or below a distance threshold, and the directivity of the front speaker 18 and the rear speaker 19 may be controlled to be unidirectional to output the user voice in the direction of the other user. The predetermined positional relationship is, for example, a positional relationship in which the moving body and another moving body are lined up in fours, front to back or left to right, at or below a distance threshold, and the directivity of the front speaker 18 and the rear speaker 19 may be controlled to be omnidirectional to output the user voice so that it can be heard by multiple other users.

[0052] The audio output control unit 39 determines which of the front speaker 18 and the rear speaker 19 to use for conversation and outputs the user voice, based on the direction of the other user as seen from the moving body recognized by the other user recognition unit 31. When the other user is in the front, the audio output control unit 39 controls to output the user voice from the front speaker 18. When the other user is in the rear, the audio output control unit 39 controls to output the user voice from the rear speaker 19.

[0053] (Conversation support method) Next, information processing in the conversation support system 10 will be described with reference to FIG. 3. FIG. 3 is a flowchart showing a flow of processing in the conversation support device according to the first embodiment. When the power supply of the conversation support system 10 is ON and an ON operation of the user microphone 11 is accepted, the processing of the flowchart shown in FIG. 3 is started. When the processing of the flowchart in FIG. 3 is started, the user's voice is picked up by the user's microphone 11. The picked up user voice is output to the user voice acquisition unit 21. The front microphone 12 and the rear microphone 13 pick up ambient voice. The picked up ambient voice is output to the ambient voice acquisition unit 22. The camera unit 14 captures the surroundings. The captured image is acquired by the image acquisition unit 23. The sensor unit 15 detects a person present in the vicinity of the moving object. The detected sensor data is acquired by the sensor data acquisition unit 24.

[0054] The control unit 20 determines whether or not another user has been recognized (step S101). More specifically, the control unit 20 determines whether or not another user who is on board another moving object and located in the vicinity of the moving object has been recognized by the other user recognition unit 31. When the control unit 20 determines that another user has been recognized (Yes in step S101), the control unit 20 proceeds to step S102. When the control unit 20 does not determine that another user has been recognized (No in step S101), the control unit 20 proceeds to step S107.

[0055] When it is determined that another user has been recognized (Yes in step S101), the control unit 20 determines whether or not the distance between the moving body and the other user is equal to or less than the distance threshold (step S102). More specifically, the distance determination unit 33 determines whether or not the distance between the moving body and the other user recognized by the other user recognition unit 31 is equal to or less than the distance threshold. When the control unit 20 determines that the distance is equal to or less than the distance threshold (Yes in step S102), the control unit 20 proceeds to step S103. When the control unit 20 does not determine that the distance is equal to or less than the distance threshold (No in step S102), the control unit 20 proceeds to step S107.

[0056] When it is determined that the distance is equal to or less than the distance threshold (Yes in step S102), the control unit 20 acquires the surrounding conditions (step S103). More specifically, the control unit 20 acquires, via the surrounding conditions acquisition unit 32, whether the current position of the moving object is in an area where noise is regulated or in an area where people are densely packed, as the surrounding conditions. The control unit 20 proceeds to step S104.

[0057] The control unit 20 determines whether the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed (step S104). When the surrounding situation determination unit 34 determines that the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed (Yes in step S104), the control unit 20 proceeds to step S105. When the surrounding situation determination unit 34 does not determine that the current location of the moving object is in an area where noise is regulated and does not determine that the area is in an area where people are densely packed (No in step S104), the control unit 20 proceeds to step S106. The control unit 20 may determine whether or not to limit the volume output from the front speaker 18 or the rear speaker 19 based on attribute information of the area or region where the moving object is located.

[0058] When it is determined that the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed (Yes in step S104), the control unit 20 outputs the user voice at the minimum volume or at muted volume (step S105). More specifically, the control unit 20 controls the audio output control unit 39 to output the user voice at the minimum volume or at muted volume from the front speaker 18 or the rear speaker 19 facing the direction in which the other user is recognized. The control unit 20 proceeds to step S107.

[0059] If the current location of the moving object is not determined to be in an area where noise is regulated and is not determined to be in an area where people are densely packed (No in step S104), the control unit 20 outputs the user voice at a normal volume (step S106). More specifically, the control unit 20 controls the audio output control unit 39 to output the user voice at a normal volume from the front speaker 18 or the rear speaker 19 facing the direction in which the other user is recognized. The control unit 20 proceeds to step S107.

[0060] The control unit 20 determines whether or not to end this process (step S107). The control unit 20 determines to end this process when the power of the conversation support system 10 is turned off, or when an operation to turn off the user microphone 11 is received, etc. If the control unit 20 determines to end the process (Yes in step S107), it ends the process of this flowchart. If the control unit 20 does not determine to end the process (No in step S107), it executes the process of step S101 again.

[0061] (effect) As described above, in this embodiment, when the distance between the moving body and another user is equal to or less than the distance threshold and the current location of the moving body is in an area where noise is regulated or in an area where people are densely packed, the user voice is output with a changed volume from the front speaker 18 or the rear speaker 19 facing the direction recognized by the other user. According to this embodiment, it is possible to support conversations between users of multiple moving bodies.

[0062] In this embodiment, when the distance between the moving body and another user is equal to or less than the distance threshold and the current location of the moving body is determined to be in an area where noise is regulated or in an area where people are densely populated, the user's voice can be output at minimum volume or muted from the front speaker 18 or rear speaker 19 facing the direction recognized by the other user.

[0063] In this embodiment, if the distance between the moving body and another user is less than or equal to the distance threshold, and the current location of the moving body is not determined to be in an area where noise is regulated, and is not determined to be in an area where people are densely populated, the user's voice can be output at normal volume from the front speaker 18 or rear speaker 19 facing the direction recognized by the other user.

[0064] In this embodiment, when it is determined that a moving body and another moving body are in a specified positional relationship, the front speaker 18 or the rear speaker 19 facing the direction recognized by the other user is controlled to be omnidirectional and output the user's voice.

[0065] In this embodiment, the user's voice is output from the front speaker 18 or the rear speaker 19 facing the direction in which the other user is present. According to this embodiment, it is possible to make the voice audible from the direction of the person with whom the conversation is taking place. According to this embodiment, it is possible to have a natural conversation. According to this embodiment, it is possible to make the voice less audible to anyone other than the person with whom the conversation is taking place.

[0066] [Second embodiment] A conversation support system 10A according to this embodiment will be described with reference to Figs. 4 and 5. Fig. 4 is a block diagram showing a configuration example of a conversation support system including a conversation support device according to a second embodiment. Fig. 5 is a flowchart showing a processing flow of the conversation support device according to the second embodiment. The conversation support system 10A has a basic configuration similar to that of the conversation support system 10 of the first embodiment. In the following description, components similar to those of the conversation support system 10 are given the same or corresponding reference numerals, and detailed description thereof will be omitted. The conversation support system 10A differs from the first embodiment in that it includes a recording unit 41A and a sound recording control unit 35A, and in the processing in the audio output control unit 39A.

[0067] The recording unit 41A is used for temporary storage of data in the conversation support system 10. The recording unit 41A is, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a recording medium such as a memory card. The recording unit 41A may be an external recording unit that is wirelessly connected via a communication device (not shown). The recording unit 41A records the user's voice based on a control signal output from the sound recording control unit 35A.

[0068] The recording control unit 35A records in the recording unit 41A the user voice collected by the user voice acquisition unit 21. When the surroundings of the mobile object are in an area where noise is regulated or in an area where people are densely packed, the recording control unit 35A records the user voice in the recording unit 41A.

[0069] The recording control unit 35A may record the user's voice in the recording unit 41A, and then delete the user's voice after a predetermined time has elapsed.

[0070] When the surrounding conditions are outside of an area where noise is regulated or outside of an area where people are densely populated, the audio output control unit 39A controls to output the user's voice recorded in the recording unit 41A.

[0071] Next, information processing in the conversation support system 10A will be described with reference to Fig. 5. Steps S111 to S114 and S119 are similar to steps S101 to S104 and S107 in the flowchart shown in Fig. 3.

[0072] When it is determined that the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed (Yes in step S114), the control unit 20A records the user voice (step S115). More specifically, the control unit 20A stores the user voice acquired by the user voice acquisition unit 21 in the recording unit 41A by the recording control unit 35A. The control unit 20A proceeds to step S119.

[0073] If the control unit 20A does not determine that the current location of the mobile object is in an area where noise is regulated and is not determined to be an area where people are densely packed (No in step S114), the control unit 20A determines whether or not there is user voice recorded within a predetermined time period (step S116). The control unit 20A determines whether or not there is user voice recorded within a predetermined time period in the recording unit 41A. If the control unit 20A determines that there is user voice recorded within the predetermined time period (Yes in step S116), the control unit 20A proceeds to step S117. If the control unit 20A does not determine that there is user voice recorded within the predetermined time period (No in step S116), the control unit 20A proceeds to step S118.

[0074] When it is determined that there is user voice recorded within the predetermined time (Yes in step S116), the control unit 20A outputs the recorded user voice at a normal volume (step S117). More specifically, the control unit 20A controls the audio output control unit 39A to output the user voice recorded in the recording unit 41A at a normal volume from the front speaker 18 or the rear speaker 19 facing the direction recognized by the other user. The control unit 20A proceeds to step S119.

[0075] If it is not determined that there is user voice recorded within the predetermined time (No in step S116), the control unit 20A outputs the user voice at a normal volume (step S118). More specifically, the control unit 20A controls the audio output control unit 39A to output the user voice at a normal volume from the front speaker 18 or the rear speaker 19 facing the direction recognized by the other user. The control unit 20A proceeds to step S119.

[0076] (effect) As described above, in this embodiment, when the distance between the moving object and the other user is greater than the distance threshold, when the distance between the moving object and the other user is equal to or less than the distance threshold and the surrounding situation of the moving object is an area where noise is regulated or an area where people are densely populated, the user voice is recorded in the recording unit. In this embodiment, when the distance between the moving object and the other user is equal to or less than the distance threshold and the surrounding situation is outside the area where noise is regulated or outside the area where people are densely populated, the recorded user voice is controlled to be output. According to this embodiment, when the distance between the moving object and the other user is greater than the distance threshold or the surrounding situation is outside the area where noise is regulated or the area where people are densely populated, the user voice can be recorded and played back later. According to this embodiment, the conversation can be continued at an appropriate time without interruption.

[0077] [Third embodiment] The conversation support system 10 according to the present embodiment will be described with reference to FIG. 6. FIG. 6 is a flowchart showing the flow of processing of a conversation support device according to a third embodiment. The conversation support system 10 differs from the first embodiment in the processing in the audio output control unit 39. The third embodiment is assumed to be used in a specific area such as a theme park. In the third embodiment, the voice quality is changed to a character voice as an example of changing the audio output form.

[0078] When the surroundings of the mobile object are in an area where noise is regulated or in an area where people are densely packed, the audio output control unit 39 controls the user audio acquired by the user audio acquisition unit 21 to convert the collected user audio into the audio of a character and output the converted audio. When the mobile object is used in a theme park, for example, the audio output control unit 39 converts the collected user audio into the audio of a character related to the theme park and outputs the converted audio. The audio output control unit 39 may change the audio of the character depending on the characteristics of the area where the conversation support system 10 is used, for example.

[0079] Next, information processing in conversation support system 10A will be described with reference to Fig. 6. Steps S121 to S124, S126, and S127 are similar to steps S101 to S104, S106, and S107 in the flowchart shown in Fig. 3.

[0080] When it is determined that the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed (Yes in step S124), the control unit 20 converts the user voice into a character voice and outputs it (step S125). More specifically, the control unit 20 controls the voice output control unit 39 to output the user voice converted into the character voice from the front speaker 18 or the rear speaker 19 facing the direction recognized by the other user. The control unit 20 proceeds to step S127.

[0081] (effect) Areas in theme parks where attractions, shows, etc. are held are crowded with spectators, and there is a risk that the voice of the user of the conversation support system 10 becomes noise for the spectators and ruins the atmosphere of the theme park. As described above, in this embodiment, when the distance between the moving object and other users is equal to or less than the distance threshold and the situation around the moving object is an area where noise is regulated or an area where people are densely populated, the user voice acquisition unit is controlled to convert the collected user voice into the voice of a character and output it.

[0082] Each component of the illustrated conversation support device is a functional concept, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to that shown in the figure, and all or part of the device may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.

[0083] The configuration of the conversation support device is realized, for example, as software, by a program loaded into a memory. In the above embodiment, the functional blocks are described as being realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms by only hardware, only software, or a combination of both.

[0084] The components described above include those that a person skilled in the art can easily imagine and those that are substantially the same. Furthermore, the configurations described above can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the configurations are possible without departing from the scope of the present invention.

[0085] In the above description, the peripheral sound collection section is described as including the front microphone 12 and the rear microphone 13, but is not limited to this.

[0086] In the above description, the audio output unit is described as including the front speaker 18 and the rear speaker 19, but is not limited to this.

[0087] In the third embodiment, when the system is used in a specific area such as a theme park, the process of acquiring the surrounding conditions and determining whether the surrounding conditions are information indicating a noise regulated area or information indicating a crowded area is not essential. In this case, the processes of steps S123, S124, and S126 are not essential. [Explanation of symbols]

[0088] 10 Conversation Support System 11 User Microphone 12 Front microphone (peripheral sound pickup section) 13 Rear microphone (peripheral sound pickup section) 14 Camera unit 15 Sensor unit 16 GNSS receiver 17 Map information storage unit 18 Front speaker (audio output section) 19 Rear speaker (audio output section) 20 Control unit (conversation support device) 21 User voice acquisition unit 22 Surrounding audio acquisition unit 23 Video Acquisition Unit 24 Sensor data acquisition unit 25 Location information acquisition section 26 Map information acquisition section 27 Mobile information acquisition unit 31 Other user recognition unit 32 Surrounding situation acquisition unit 33 Distance determination unit 34 Surrounding situation determination section 39 Audio output control section

Claims

1. A conversation support device for users who are riding on a plurality of moving objects, comprising: a user voice acquisition unit that acquires a user voice from a user voice collection unit that collects the voice of a user riding on the moving object; a surrounding sound acquisition unit that acquires surrounding sounds from a surrounding sound collection unit that collects sounds around the moving object; a voice output control unit that controls output of a user voice from a voice output unit that is capable of outputting a user voice from the moving object toward the surroundings; A different user recognition unit that recognizes different users who are riding on different moving objects and are located in the vicinity of the moving object; A surrounding condition acquisition unit that acquires a surrounding condition of the moving object; a distance determination unit that determines whether or not a distance between the moving object and the other user recognized by the other user recognition unit is equal to or less than a distance threshold; a surrounding condition determination unit that determines whether a current location of the moving object is in an area where noise is regulated or in an area where people are densely packed, based on the surrounding conditions acquired by the surrounding condition acquisition unit, and thus determines whether the current location of the moving object is unsuitable for conversation between users aboard the moving object; Equipped with The audio output control unit controls the user to output a user voice by changing an audio output form based on a determination result of the surrounding situation determination unit. Conversation support device.

2. The surrounding situation determination unit determines whether or not the moving object and the other moving object have a predetermined positional relationship; When the surrounding situation determination unit determines that a predetermined positional relationship exists, the audio output control unit changes a directivity of the audio output unit to output a user voice. The conversation support device according to claim 1 .

3. a recording control unit that records the user voice collected by the user voice acquisition unit in a recording unit; Equipped with the recording control unit records the user's voice in the recording unit when the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed; the voice output control unit controls the recording unit to output the user voice recorded in the recording unit when the current location of the moving object is outside an area where noise is regulated or is outside an area where people are densely populated. The conversation support device according to claim 1 .

4. the voice output control unit controls the user voice collected by the user voice acquisition unit to be converted into a character's voice and output the converted voice when the current location of the moving object is in an area where noise is regulated or in an area where people are densely populated. The conversation support device according to claim 1 .

5. A method for supporting conversation between users who are respectively on board a plurality of moving objects, comprising: a user voice acquisition step of acquiring a user voice from a user voice collection unit that collects the voice of a user riding on the moving object; a surrounding sound acquisition step of acquiring a surrounding sound from a surrounding sound collection unit that collects sounds around the moving object; a voice output control step of controlling an output of a user voice from a voice output unit capable of outputting a user voice from the moving object toward a surrounding area; a user recognition step of recognizing other users who are riding on other moving objects located in the vicinity of the moving object; a surrounding situation acquisition step of acquiring a surrounding situation of the moving object; a distance determination step of determining whether or not a distance between the moving object and the other user recognized in the other user recognition step is equal to or less than a distance threshold; a surrounding situation determination step of determining whether the current location of the moving object is in an area where noise is regulated or in an area where people are densely packed, based on the surrounding situation acquired by the surrounding situation acquisition step, and whether the current location is unsuitable for conversation between users aboard the moving object; Including, The voice output control step controls the user's voice to be output by changing a voice output form based on a result of the determination step of the surrounding situation determination step. Conversation support methods.

Images