Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program which make passengers who ride in different vehicles be able to easily communicate with each other. The present invention comprises: an acquisition unit which acquires a situation of an own vehicle and a situation of another vehicle; a setting unit which sets a position, at which a sound image of the passenger who rides in the other vehicle is localized, in the own vehicle on the basis of the situation of the own vehicle and the situation of the other vehicle; and a sound image localization processing unit which localizes the sound image of the passenger at the position set in the setting unit. The acquisition unit acquires at least a seat position on which the passenger is being seated, and the setting unit sets a vacant seat of the own vehicle as the position at which the sound image of the passenger of the other vehicle is localized. The present technology can be applied to an information processing device which controls telepresence.

Description

[Technical Field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that enable good communication between users in different locations, for example. [Background technology]

[0002] Telepresence systems have been developed to connect distant spaces using video, audio, and other information channels, making it possible to feel as if the locations are connected and the other person is actually present (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-118832 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 proposes judging the attention level of a user from their line of sight and deciding on a seat for each user based on that level. Because the attention level is judged from the line of sight, etc., if the user is not facing each other, it may be impossible to judge the attention level and decide on a seat.

[0005] It is desirable to be able to determine the optimum seat that allows users to communicate well with each other.

[0006] The present technology has been developed in light of such circumstances, and makes it possible to determine the optimum seat that will allow good communication between users. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present technology includes: an acquisition unit that acquires a situation of a host vehicle and a situation of another vehicle; a setting unit that sets a position in the host vehicle where a sound image of an occupant riding in the other vehicle is to be localized based on the situation of the host vehicle and the situation of the other vehicle; and a sound image localization processing unit that localizes a sound image of the occupant at the position set by the setting unit. 、 The setting unit groups the occupants in the host vehicle and the occupants in the other vehicle, and sets the positions so that the sound image positions of the occupants included in the same group are located close to each other. It is an information processing device.

[0008] According to an aspect of the present technology, an information processing method includes: acquiring a situation of a host vehicle and a situation of another vehicle; setting a position in the host vehicle where a sound image of an occupant riding in the other vehicle is to be localized based on the situation of the host vehicle and the situation of the other vehicle; and localizing the sound image of the occupant at the set position. The setting is performed by grouping the occupants in the own vehicle and the occupants in the other vehicle, and setting the positions so that the sound image positions of the occupants included in the same group are located close to each other. It is an information processing method.

[0009] A program according to one aspect of the present technology includes a computer that acquires a situation of a host vehicle and a situation of another vehicle, sets a position in the host vehicle where a sound image of an occupant in the other vehicle is to be localized based on the situation of the host vehicle and the situation of the other vehicle, and localizes the sound image of the occupant at the set position. The setting is performed by grouping the occupants in the own vehicle and the occupants in the other vehicle, and setting the positions so that the sound image positions of the occupants included in the same group are located close to each other. It is a program for executing a process that includes steps.

[0010] In an information processing device, an information processing method, and a program according to one aspect of the present technology, a position in the host vehicle where a sound image of an occupant in another vehicle is localized is set based on a situation of the host vehicle and a situation of the other vehicle, and the sound image of the occupant is localized at the set position; The occupants in the vehicle and the occupants in the other vehicle are grouped, and the positions are set so that the sound image positions of the occupants included in the same group are located close to each other. .

[0011] The information processing device may be an independent device or an internal block constituting one device.

[0012] The program can also be provided by transmitting it via a transmission medium or by recording it on a recording medium. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an example of the configuration of an information processing system. [Figure 2] FIG. 10 is a diagram for explaining seating arrangements. [Figure 3] FIG. 2 is a diagram for explaining the arrangement of speakers. [Figure 4] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device. [Figure 5] FIG. 10 is a diagram illustrating another example of the configuration of the information processing device. [Figure 6] FIG. 10 is a diagram illustrating pre-registration parameters. [Figure 7] FIG. 10 is a diagram for explaining departure registration parameters. [Figure 8] 10 is a flowchart illustrating processing by the information processing device. [Figure 9] 10 is a flowchart illustrating a vehicle matching process. [Figure 10] 10 is a flowchart for explaining a determination process related to condition 1. [Figure 11] FIG. 10 is a diagram illustrating a region to be clipped. [Figure 12] 10 is a flowchart illustrating a determination process related to condition 2. [Figure 13] FIG. 10 is a diagram illustrating matching conditions. [Figure 14] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 15] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 16] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 17] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 18] FIG. 10 is a diagram illustrating a group. [Figure 19] 10 is a flowchart illustrating a static seat arrangement process. [Figure 20] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 21] 10 is a flowchart for explaining seating arrangement processing based on age and role. [Figure 22] 10 is a flowchart for explaining seating arrangement processing that takes relative positions into consideration. [Figure 23] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 24] FIG. 10 is a diagram for explaining condition A. [Figure 25] FIG. 10 is a diagram for explaining seating arrangement processing. [Figure 26] 10 is a flowchart illustrating a dynamic seat arrangement process. [Figure 27] FIG. 10 is a diagram illustrating an example of the configuration of a climax detection unit. [Figure 28] 10 is a flowchart illustrating a rise detection process. [Figure 29] FIG. 2 is a diagram illustrating an example of the configuration of a sound image localization processing unit. [Figure 30] FIG. 2 is a diagram illustrating an example of the configuration of a sound processing unit. [Figure 31] FIG. 10 is a diagram for explaining processing related to sound image localization. [Figure 32] FIG. 2 is a diagram illustrating an example of the configuration of a sound processing unit. [Figure 33] FIG. 10 is a diagram for explaining processing related to sound image localization. [Figure 34] FIG. 10 is a diagram illustrating a second embodiment. [Figure 35] FIG. 10 is a diagram illustrating a third embodiment. [Figure 36] FIG. 10 is a diagram illustrating a third embodiment. [Figure 37] FIG. 10 is a diagram illustrating a fourth embodiment. [Figure 38] FIG. 10 is a diagram illustrating a fourth embodiment. [Figure 39] FIG. 10 is a diagram illustrating a fifth embodiment. [Figure 40] FIG. 13 is a diagram illustrating a sixth embodiment. [Figure 41] FIG. 13 is a diagram illustrating a seventh embodiment. [Figure 42] FIG. 13 is a diagram illustrating a seventh embodiment. [Figure 43] FIG. 13 is a diagram illustrating an eighth embodiment. [Figure 44] FIG. 13 is a diagram illustrating a ninth embodiment. [Figure 45] FIG. 13 is a diagram illustrating a ninth embodiment. [Figure 46] FIG. 22 is a diagram illustrating a tenth embodiment. [Figure 47] FIG. 1 illustrates an example of the configuration of a personal computer. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described.

[0015] <System configuration example> Fig. 1 is a diagram showing the configuration of an embodiment of an information processing system to which the present technology is applied. The information processing system 10 shown in Fig. 1A is configured to include vehicles 11-1 to 11-3 and a network 12. In the information processing system 10 shown in Fig. 1A, the vehicles 11-1 to 11-3 are configured to be able to exchange data with each other via the network 12.

[0016] The information processing system 10 shown in Fig. 1B includes vehicles 11-1 to 11-3, a network 12, and a server 13. In the information processing system 10 shown in Fig. 1B, the vehicles 11-1 to 11-3 are configured to be able to exchange data with each other via the network 12 and the server 13.

[0017] When there is no need to distinguish between the vehicles 11-1 to 11-3, they will be simply referred to as vehicle 11. The same applies to other parts.

[0018] According to the information processing system 10 to which the present technology is applied, it is possible to give the passengers who are riding in multiple vehicles 11 the feeling that they are riding in and driving the same vehicle 11. This will be further explained with reference to FIG. 2.

[0019] As shown in Fig. 2, vehicle 11-1 is equipped with seats 31a-1 to 31h-1, and vehicle 11-2 is equipped with seats 31a-2 to 31h-2, and both vehicles are in a traveling state. At time T1, passenger 32a-1 is riding in seat 31a-1 of vehicle 11-1, and passenger 32b-1 is riding in seat 31b-1. Passenger 32a-2 is riding in seat 31a-2 of vehicle 11-2.

[0020] In the following description, a seat in vehicle 11-1 will be referred to as seat 31-1, and an occupant in vehicle 11-1 will be referred to as occupant 32-1. That is, the symbol "-1" means that it relates to vehicle 11-1. Similarly, a seat in vehicle 11-2 will be referred to as seat 31-2, and an occupant in vehicle 11-2 will be referred to as occupant 32-1. That is, the symbol "-2" means that it relates to vehicle 11-2.

[0021] When it is desired to specify the position of seat 31, for example, it is written as seat 31a, seat 31b, etc. In other words, the alphabet of the symbol is a description that indicates the position of the seat. For example, the description of seat 31a-1 means the seat located at the front right of vehicle 11-1. For example, the description of occupant 32a-1 means the occupant sitting in seat 31a-1 located at the front right of vehicle 11-1.

[0022] At time T2, when vehicles 11-1 and 11-2 are matched and ready to communicate, a virtual occupant 32a-2' is generated in seat 31c-1 of vehicle 11-1, as if occupant 32a-2 in vehicle 11-2 were actually riding there.

[0023] In other words, the sound image of the occupant 32a-2 in the vehicle 11-2 is localized at the seat 31c-1 of the vehicle 11-1. By localizing the sound image, the content of speech uttered by the occupant 32a-2 in the vehicle 11-2 is spoken by the virtual occupant 32a-2' (output from the sound image localized at the seat 31c-1).

[0024] In the following description, for example, generating a virtual occupant 32a-2' in seat 31c-1 means controlling the sound output from the speaker so that the sound image of occupant 32a-2 is localized in seat 31c-1.

[0025] A symbol with a prime added to the occupant 32, such as occupant 32', indicates a virtual occupant 32. For example, occupant 32-2' indicates the occupant 32-2 who is actually riding in the vehicle 11-2 and is presented as a sound image to occupant 32-1 in the vehicle 11-1.

[0026] A state is created in which the vehicle 11-1 is occupied by the occupants 32a-1, 32b-1, and 32a-2'. The occupants 32a-1 and 32b-1 in the vehicle 11-1 can feel as if they are driving together with the occupant 32a-2 in another vehicle 11-2.

[0027] On the other hand, in vehicle 11-2, a virtual occupant 32a-1' is generated in seat 31c-2 of vehicle 11-2 as if the occupant 32a-1 in vehicle 11-1 were riding there. Also, a virtual occupant 32b-1' is generated in seat 31e-2 as if the occupant 32b-1 in vehicle 11-1 were riding there.

[0028] The contents uttered by the occupant 32a-1 in the vehicle 11-1 are spoken by the occupant 32a-1' virtually riding in the vehicle 11-2. The contents uttered by the occupant 32b-1 in the vehicle 11-1 are spoken by the occupant 32b-1' virtually riding in the vehicle 11-2.

[0029] A state is created in which the vehicle 11-2 is occupied by the occupants 32a-1', 32b-1', and 32a-2. The occupant 32a-2 in the vehicle 11-2 can get the feeling that he or she is driving together with the occupants 32a-1 and 32b-1 in another vehicle 11-1.

[0030] In this way, the information processing system 10 allows passengers in multiple vehicles 11 to feel as if they are in the same vehicle 11 and driving.

[0031] In the following description, the vehicle 11-1 and the vehicle 11-2 are taken as examples, but it is possible to process a plurality of vehicles 11 as targets.

[0032] For example, as described above, the virtual occupant 32a-1' is generated by localizing the sound image of the speech of the occupant 32a-1 at the predetermined seat 31. In other words, by creating a sound image of the occupant 32a-1 at the predetermined seat 31, a state is created in which it is as if the occupant 32a-1 is sitting in the predetermined seat 31 and speaking.

[0033] A sound image of a passenger 32a-2 riding in the vehicle 11-2 is generated in a predetermined seat 31 of the vehicle 11-1. The sound image is generated by controlling the sound output from the speakers provided in the vehicle 11-1. The number and arrangement of the speakers provided in the vehicle 11-1 differ depending on the vehicle model, etc. Figure 3 shows an example of speaker arrangement.

[0034] The speaker arrangement shown in Fig. 3A shows the case where so-called seat speakers are installed. Speakers 33a-1 and 33a-2 are arranged in seat 31a. Speakers 33b-1 and 33b-2 are arranged in seat 31b. Speakers 33c-1 and 33c-2 are arranged in seat 31c.

[0035] Speakers 33e-1 and 33e-2 are arranged in seat 31e. Speakers 33f-1 and 33f-2 are arranged in seat 31f. Speakers 33h-1 and 33h-2 are arranged in seat 31h. In this way, two speakers 33 are arranged in each seat. Note that, although an example has been shown in which no speakers are arranged in seats 31d and 31g, speakers 33 may also be arranged in seats 31d and 31g.

[0036] In the speaker arrangement shown in Fig. 3B, speaker 33a-3 is located to the right of seat 31a, and speaker 33b-3 is located to the left of seat 31b. Speaker 33c-3 is located to the right of seat 31c, and speaker 33e-3 is located to the left of seat 31e. Speaker 33f-3 is located to the right of seat 31f, and speaker 33h-3 is located to the left of seat 31h.

[0037] The number and locations of the speakers 33 vary depending on the vehicle 11, but by performing control to localize the sound image according to the number and locations of the speakers 33, it is possible to localize the sound image of the passenger 32 riding in the other vehicle 11 at a specific seat 31 as described above.

[0038] <Configuration of information processing device> The configuration of an information processing device that performs the processing related to the localization of the sound image will be described below. Fig. 4 is a diagram showing the configuration of an information processing device 51 in the case where vehicles 11 communicate directly with each other and perform processing related to the localization of the sound image, as in the information processing system 10 shown in Fig. 1A. The information processing device 51 is mounted on each vehicle 11.

[0039] 4 includes an input unit 61, an input / output processing unit 62, an output unit 63, and an information processing unit 64. The information processing unit 64 includes a seating arrangement processing unit 71, a vehicle matching processing unit 72, a seating arrangement information storage unit 73, a vehicle matching information storage unit 74, a vehicle information storage unit 75, an occupant information storage unit 76, a sensor information processing unit 77, a communication unit 78, and an output information creation unit 79. The seating arrangement processing unit 71 includes a seating arrangement rule storage unit 82. The vehicle matching processing unit 72 includes a vehicle matching rule storage unit 83. The output information creation unit 79 includes a sound image localization processing unit 81.

[0040] When vehicles 11 communicate with each other via a server 13, as in the information processing system 10 shown in B of Fig. 1, some of the processing of the information processing device 51 is performed on the server 13 side. In this case, a configuration such as that shown in Fig. 5 can be used. Fig. 5 is a diagram showing an example of a configuration in which the information processing device 51 is distributed between the vehicle 11 side and the server 13 side.

[0041] The information processing device 51 shown in Fig. 5 is configured such that the seating arrangement processing unit 71, the vehicle matching processing unit 72, the seating arrangement information storage unit 73, and the vehicle matching information storage unit 74 of the information processing device 51 shown in Fig. 4 are provided on the server 13 side as a seating arrangement processing unit 71', a vehicle matching processing unit 72', a seating arrangement information storage unit 73', and a vehicle matching information storage unit 74'. The server 13 also has a communication unit 78' that communicates with a communication unit 78 provided on the vehicle 11 side.

[0042] The input unit 61 is an input device for acquiring information on the status of the occupant, the driving status of the vehicle, and operations from the occupant. For example, the input unit 61 includes an image sensor, a depth sensor, a touch sensor, etc. The image sensor may be a visible light camera, an infrared camera, etc. that can acquire two-dimensional images.

[0043] A depth sensor is a sensor that can acquire three-dimensional information that includes depth, and can use a stereo camera, time-of-flight sensor, or structured light sensor.

[0044] Information about the traveling state of the vehicle 11, such as whether or not the vehicle is being driven autonomously, is also input to the input unit 61. Input information from operations by the user (passenger) is also input to the input unit 61, including input information from operations using a touch display, voice operations, gesture operations using skeletal information, and the like.

[0045] The input / output processing unit 62 receives input information from the input unit 61 and provides a function for displaying graphics to the passengers. It can be a control layer of a general OS (Operating System) that controls the rendering of multi-content, such as windows that display applications, and delivers events such as touches to each content.

[0046] The input / output processing unit 62 supplies the information from the input unit 61 to the information processing unit 64, which is necessary for the information processing unit 64 to perform the matching processing and seat allocation processing described below, such as images for determining the seats in which the occupants are sitting, images for estimating the facial expressions of the occupants, and audio for obtaining information about the conversation.

[0047] The output unit 36 ​​presents the created output information to the occupants. Sound is presented through the speaker 33, headphones, etc. Visual information is presented through a projector, a liquid crystal screen such as a display of a navigation system, etc.

[0048] The seating arrangement processing unit 71 stores seating arrangement rules in the seating arrangement rule storage unit 82 for how to arrange virtual occupants among a plurality of vehicles 11 matched in the matching process performed by the vehicle matching processing unit 72 described later, and performs seating arrangement processing based on the seating arrangement rules.

[0049] The seating arrangement processing unit 71 performs seating arrangement using information regarding the seat arrangement in the vehicle, information regarding the seats in the vehicle in which the occupants are seated, information regarding the ages, conversation contents, facial expressions, etc. This information includes information stored in the vehicle information storage unit 75, information stored in the occupant information storage unit 76, information acquired by the sensor information processing unit 77, etc., and information is acquired from each of these units as necessary to execute seating arrangement processing.

[0050] The vehicle matching processing unit 72 includes a vehicle matching rule storage unit 83, and performs matching processing based on rules for selecting a plurality of vehicles 11 to be matched from a plurality of vehicles stored in the vehicle matching rule storage unit 83. Information about the subject vehicle and information about other vehicles are used in the matching processing. Information stored in the vehicle information storage unit 75 is used as the information about the subject vehicle, and information acquired via the communication unit 78 is used as the information about other vehicles.

[0051] The seating arrangement information accumulation unit 73 holds information about seating arrangements determined by the seating arrangement processing of the seating arrangement processing unit 71. The seating arrangement information accumulated in the seating arrangement information accumulation unit 73 is referenced when the sound image localization processing unit 81 executes processing to localize the sound images of virtual passengers at the positions where the virtual passengers are seated.

[0052] The vehicle matching information storage unit 74 holds information about the vehicles 11 matched by the vehicle matching processing unit 72. The stored information includes, for example, information such as the age and gender of the occupants of the matched vehicles, the number of seats, and information about the seats in which the occupants are seated. The information stored in the vehicle matching information storage unit 74 is referenced when the seat allocation processing unit 71 performs seat allocation processing.

[0053] The vehicle information storage unit 75 stores vehicle information such as the vehicle capacity, vehicle model, current number of occupants, and location of the speaker 33. When the information processing device 51 is provided separately on the vehicle 11 side and the server 13 side as shown in Fig. 5, the vehicle information storage unit 75' of the server 13 can store vehicle information of all vehicles participating in the system.

[0054] The occupant information storage unit 76 stores information such as the age and gender of occupants riding in the vehicle 11. When the server 13 is used, the occupant information storage unit 76′ of the server 13 can store information on all occupants participating in the system, grouped by vehicle 11.

[0055] The sensor information processing unit 77 processes information such as the facial expressions and postures of the occupants, the speed of the vehicle 11, and GPS (Global Positioning System) information, and supplies the information as vehicle information to the vehicle information storage unit 75 and as occupant information to the occupant information storage unit 76. The communication unit 78 communicates with the server 13 and other vehicles 11. The communication unit 78 performs communication based on, for example, TCP / IP (Transmission Control Protocol / Internet Protocol) communication.

[0056] The output information creation unit 79 creates information to be output from the output unit 63 based on sensor information, vehicle matching information, seating arrangement information, etc. The output information creation unit 79 performs processing for localizing a sound image at a predetermined seat 31 by the sound image localization processing unit 81, and performs processing so that the processing result is output by the output unit 63. The sound image localization processing unit 81 executes processing for localizing the sound image of a virtual occupant using the seating arrangement information stored in the seating arrangement information storage unit 73 and the vehicle information (mainly information related to the placement of the speakers 33) stored in the vehicle information storage unit 75.

[0057] <About parameters> When a sound image of an occupant of one vehicle 11 is generated in another vehicle 11, matching of the vehicles 11 for generating the sound image is performed. The parameters that are set before this matching will now be described. Fig. 6 is a diagram showing an example of pre-registered parameters 91 that are set in advance by the user.

[0058] The pre-registration parameters 91 include the following items: "Vehicle type," "Seat," "Speaker information," "Owner information," "License plate," and "Friend's vehicle." The "Vehicle type" item is used to register the vehicle type of the vehicle 11 in which the information processing system 10 will be installed.

[0059] The "seats" item registers information about available seats in the vehicle 11 and the relative positions of the seats, for example, a seat located on the left side of the first row, a seat located in the center of the second row, etc. The information registered in the "seats" item is referenced, for example, when setting (seating arrangement) seats to be assigned to virtual passengers, in other words, when setting positions for localizing sound images.

[0060] The item "Speaker Information" registers information such as the type and location of the speaker 33 in the vehicle 11. The information registered in the item "Speaker Information" is referenced when determining how to present a sound image, etc.

[0061] It should be noted that when the vehicle model is registered, the vehicle model may be searched on the Internet to obtain information about seats and speakers, and then the information may be registered.

[0062] The "owner information" field registers information such as the name, age, sex, and facial information of the owner of the vehicle 11. The information registered in the "owner information" field is referenced, for example, when facial authentication is performed.

[0063] The owner information may also include information about multiple occupants who may ride in the vehicle 11. By registering information about multiple occupants, it becomes possible to accurately and appropriately perform processes such as face recognition and calculation of the smile degree, which will be described later.

[0064] In the item "license plate", information such as the number and registered area of ​​the vehicle 11 is registered. The information registered in the item "license plate" is referenced, for example, when performing vehicle matching.

[0065] In the "Friend Vehicles" field, vehicle accounts of family members, friends, etc. are registered. The information registered in the "Friend Vehicles" field is referenced, for example, when performing vehicle matching.

[0066] The information registered as the pre-registered parameters 91 may be information other than the information described above, and information other than the information described above may be additionally registered.

[0067] 7 is a diagram showing an example of the departure registration parameters 92 that are registered when using the information processing system 10. The departure registration parameters 92 include items such as "destination," "matching settings," "itinerary," and "keywords."

[0068] The "destination" field is where the destination of the drive is registered. Even if the destination is not registered in the "destination" field, it is possible to use the information processing system 10, and vehicle matching is performed, and the vehicle is matched with another vehicle 11, or is set as a search target when matching with another vehicle 11.

[0069] In the "Matching Settings" item, "Public" is set if the user wishes to match with a vehicle 11 owned by a stranger, and "Private" is set if the user wishes to match with a vehicle 11 owned by an acquaintance.

[0070] If "public" is set, when matching for a vehicle 11 is performed, matching is performed for all vehicles using the system.

[0071] When "Private" is set, matching is performed with pre-registered vehicles 11 so that matching can be performed between people who know each other, such as friends or family. Pre-registered vehicles 11 are vehicles registered in the "Friend Vehicles" item of the pre-registration parameters 91. When "Private" is set, the vehicle 11 to be matched may be registered by the user.

[0072] In the "itinerary" field, information such as stopovers and planned meal times and locations relative to the destination is registered. The information registered in the "itinerary" field is referenced, for example, when matching vehicles 11 or during dynamic seat allocation processing (described later). As itinerary information, general times, for example, information such as lunch time at 12 o'clock, may be registered in the "itinerary" field.

[0073] In the "Keywords" field, keywords related to topics you want to talk about while driving, things you are interested in, etc. are registered. The information registered in the "Keywords" field is referenced when matching vehicles 11.

[0074] The information registered as the departure registration parameters 92 may be information other than the information described above, and information other than the information described above may be additionally registered.

[0075] These parameters are referenced to perform processes such as matching the vehicle 11 and presenting a sound image.

[0076] <Processing of information processing device> The processing of the information processing device 51 will be described with reference to the flowchart of Fig. 6. Here, the explanation will be continued taking as an example a case where the vehicles 11 communicate directly with each other in the information processing system 10 shown in Fig. 1A, and taking as an example a case where the information processing device 51 of Fig. 4 is installed in each vehicle 11.

[0077] In the following description, one of the vehicles 11 to be matched is the host vehicle 11-1, and the other is the other vehicle 11-2. The processing of the flowchart shown in Fig. 6 will be described as processing executed by the information processing device 51 mounted on the host vehicle 11-1, but basically the processing of the flowchart shown in Fig. 6 is also executed in the other vehicle 11-2 in the same way.

[0078] In step S11, the seat positions of the occupants of the host vehicle 11-1 are acquired. For example, the host vehicle 11-1 may be provided with an imaging device that captures images of the interior of the host vehicle 11-1, and the number of occupants and the seats in which the occupants are sitting may be identified by analyzing images obtained from the imaging device. The occupant may operate the input unit 61 of the information processing device 51 to input the number of occupants and their seat positions, and the input information may be acquired.

[0079] The seats may be provided with sensors, such as pressure sensors or heat sensors, and the number of occupants and their seat positions may be identified by analyzing information obtained from these sensors.

[0080] The sensor information processing unit 77 (FIG. 4) executes the process in step S11 by acquiring and processing images and operation data from the imaging device and user interface provided as the input unit 61. The acquired information about the occupant is stored in the vehicle information storage unit 75.

[0081] In step S12, the seat positions of the occupants of the other vehicle 11-2 are acquired. As with the host vehicle 11-1, information regarding the number of occupants, seat positions, etc. is acquired in the other vehicle 11-2 and stored in the vehicle information storage unit 75 of the other vehicle 11-2. This stored information is supplied from the other vehicle 11-2 to the host vehicle 11-1, whereby the host vehicle 11-1 acquires the information.

[0082] In step S13, a vehicle matching process is executed. The vehicle matching process executed in step S13 will be described with reference to the flowchart in Fig. 9. The vehicle matching processing unit 72 stores rules for executing the vehicle matching process, which will be described with reference to Fig. 9, in the vehicle matching rule storage unit 83, and executes the vehicle matching process based on these stored rules.

[0083] In step S31, it is determined whether or not there is a vehicle registered in the favorites. For example, by referring to the pre-registration parameters 91 (FIG. 6), it is determined whether or not there is a vehicle registered in the "friend vehicle" field. For example, a system may be provided in which a vehicle that the user wants to be matched with is registered before a drive, and if the vehicle is registered, the registered vehicle may be determined to be a favorite vehicle.

[0084] If it is determined in step S31 that there is a vehicle registered as a favorite, the process proceeds to step S32. In step S32, matching with the favorite vehicle 11 is performed. For example, if another vehicle 11-2 is registered as a favorite vehicle for the subject vehicle 11-1, the subject vehicle 11-1 and the other vehicle 11-2 are matched.

[0085] If multiple vehicles 11 are registered as favorites, multiple vehicles 11 will be matched.

[0086] On the other hand, if it is determined in step S31 that no vehicle has been registered as a favorite, the process proceeds to step S33. In step S33, a rule is set that vehicles 11 that are close to each other are matched.

[0087] In step S34, the matching condition is distance, and it is determined whether the closest vehicle has changed. It is determined whether there is another vehicle 11-2 traveling in a location close to the host vehicle 11-1 as a matching vehicle 11. When matching with the other vehicle 11-2 has already been performed and a sound image is being presented in the host vehicle 11-1, it is determined whether the matched other vehicle 11-2 has been changed to the other vehicle 11-2 traveling in a location close to the host vehicle 11-1.

[0088] In step S34, if it is determined that the matching condition is distance and the closest vehicle has been changed, the process proceeds to step S35. In step S35, vehicles that are close in distance are matched with each other. For example, if the vehicle 11 determined to be closest to the host vehicle 11-1 is the other vehicle 11-2, the host vehicle 11-1 and the other vehicle 11-2 are matched.

[0089] For example, when the host vehicle 11-1 and the other vehicle 11-2 are matched, if the vehicle 11 closest to the host vehicle 11-1 is changed from the other vehicle 11-2 to the other vehicle 11-3, the host vehicle 11-1 and the other vehicle 11-3 are matched.

[0090] It should be noted that, since the distance between vehicles 11 often changes while traveling, when the processing of step S34 is performed, there is a possibility that the matched vehicle 11 will be frequently changed. Therefore, the processing from step S34 onwards may be performed after a predetermined time has elapsed after matching has been performed, or may be performed when a determination is made to change the matched vehicle, as will be described later.

[0091] In this way, by matching vehicles 11 that are close to each other, it is possible to find common topics based on the location of travel and create a situation in which conversation can be promoted.

[0092] On the other hand, if it is determined in step S34 that the matching condition is not distance and / or the closest vehicle has not changed, the process proceeds to step S36, where it is determined whether the speech frequency and / or smile level has decreased.

[0093] Such a determination is made, for example, in the event that the occupants of the matched vehicle 11 are not having a lively conversation with each other, in order to cancel the match and allow the vehicle 11 to be matched with another vehicle 11. The process of determining whether the speech frequency has decreased and the process of determining whether the smile level has decreased will be described later.

[0094] If it is determined in step S34 that the speech frequency has not decreased and the smile level has not decreased, the matching between the matched vehicles 11 is maintained as is.

[0095] On the other hand, if it is determined in step S34 that the speech frequency and / or the smile level has decreased, the process proceeds to step S37, where it is determined whether or not there is a vehicle that satisfies condition 1 and / or condition 2.

[0096] Condition 1 is a condition in which the degree of similarity in the interior design of the vehicle is measured and the degree of similarity is equal to or greater than a predetermined threshold. If condition 1 is met, it is possible that the occupants have similar hobbies and preferences, and for example, it is possible that a conversation about the interior design of the vehicle will lead to other topics. Condition 1 is a condition for matching vehicles 11 in which such occupants are riding.

[0097] Condition 2 is a condition in which the degree of match of an object held by an occupant is measured and the degree of match is equal to or greater than a predetermined threshold. For example, if the object held in the hand is food, it is conceivable that the occupant may get excited talking about the food or the accessories worn on the hand. Condition 2 is a condition for matching vehicles 11 in which such occupants are riding. Note that condition 2 may also include the degree of match of accessories on parts other than the hand, clothing, etc.

[0098] The process of determining whether or not Condition 1 is satisfied will be described with reference to the flowchart of Fig. 10. In step S51, an image captured by a camera is acquired. In step S52, the interior position is clipped.

[0099] In step S53, recognition of objects in the clipped region is performed, and in step S54, the color of the recognized object is recognized.

[0100] Such processing is performed by analyzing an image captured inside the vehicle, for example, as shown in Fig. 11. Fig. 11 shows an example of an image captured of parts such as the windshield, dashboard, and instrument panel inside the vehicle 11. A camera is provided to capture an image of the area around the windshield inside the vehicle 11, and the image captured by the camera is acquired in the processing of step S51.

[0101] The area to be clipped can be, for example, an area 101 near the top of the windshield or an area 102 near the dashboard. The area to be clipped can be registered by the user when the camera is installed, or can be registered by executing a predetermined program when the camera is installed without any user intervention.

[0102] In step S12, the images of area 101 and area 102 are subjected to object recognition and color recognition processing. For example, on the dashboard, i.e., in this case, area 102, a stuffed toy or a car navigation system may be placed. Such objects are called interior, and such interior is detected in the processing of steps S53 and S54.

[0103] In step S53, objects (interior) in the clipped areas 101 and 102 are recognized. In step S54, the colors of the recognized objects are recognized. The types of objects placed there and the colors of the objects are recognized. The objects and colors can be used as factors for inferring the preferences of the occupants of the vehicle 11, and it is thought that if preferences are similar, it will be easier to find common topics and have lively conversations.

[0104] The processing of steps S51 to S54 is performed by both the subject vehicle 11-1 and the other vehicle 11-2 and is stored in each vehicle 11. In step S55, the subject vehicle 11-1 acquires information about the interior of the other vehicle 11-2, compares it with the information about its own interior, and determines the degree of match between the color and the object.

[0105] A threshold value may be set, and if there is a vehicle 11 whose matching rate is equal to or greater than the threshold value, that vehicle 11 may be set as a vehicle 11 that satisfies condition 1. A process of calculating the matching rate may be performed for multiple other vehicles 11, and the vehicle 11 with the highest matching rate may be set as a vehicle 11 that satisfies condition 1.

[0106] The process of determining whether or not Condition 2 is satisfied will be described with reference to the flowchart in Fig. 12. In step S61, an image captured by a camera is acquired. A camera for capturing images of occupants is provided inside the vehicle 11, and an image captured by the camera is acquired.

[0107] In step S62, the acquired image is analyzed to detect the occupant and recognize the skeletal structure of the occupant. Here, since it is desired to detect an object that the occupant is holding in their hand, it is sufficient to recognize the skeletal structure of the hand.

[0108] In step S63, object recognition is performed on the area around the occupant's hand. In step S64, the matching rate of the object being held by the occupant is determined.

[0109] The processing of steps S61 to S63 is performed by both the host vehicle 11-1 and the other vehicle 11-2, and is held in each vehicle 11. In step S64, the host vehicle 11-1 acquires information about the object being held from the other vehicle 11-2, compares it with the information about the object being held that the host vehicle 11-1 holds, and determines the degree of match between the object being held.

[0110] For example, when an occupant is eating something, they are likely holding food or a spoon, etc., so if the object they are holding is detected as food, the matching rate will be high. For example, when an occupant is playing a game console, they are likely holding the game console, so if the object they are holding is detected as a game console, the matching rate will be high.

[0111] A threshold value may be set, and if there is a vehicle 11 whose matching rate is equal to or greater than the threshold value, that vehicle 11 may be set as a vehicle 11 that satisfies condition 2. A process of calculating the matching rate may be performed for multiple other vehicles 11, and the vehicle 11 with the highest matching rate may be set as a vehicle 11 that satisfies condition 2.

[0112] Note that the matching rate may be calculated by detecting an object other than the object being held in the hand or by detecting an object other than the object being held in the hand. For example, the clothing of the occupant may be recognized and the matching rate of the clothing of the occupant may be calculated.

[0113] Returning to the explanation with reference to the flowchart shown in Fig. 9, in step S37, it is determined whether or not there is a vehicle 11 that satisfies condition 1 and / or condition 2, and if it is determined that there is a vehicle 11 that satisfies the condition, the process proceeds to step S38.

[0114] In step S38, matching is performed with the vehicle 11 that is determined to satisfy condition 1 and / or condition 2. For example, if the vehicle 11 that satisfies condition 1 and / or condition 2 is another vehicle 11-2, the host vehicle 11-1 and the other vehicle 11-2 are matched.

[0115] On the other hand, if neither a vehicle 11 satisfying condition 1 nor a vehicle 11 satisfying condition 2 is detected in step S37, the process proceeds to step S39. In step S39, matching based on the distance between the vehicles 11 is performed. Matching based on the distance between the vehicles 11 is performed by changing the matching rule to one in which vehicles that are close in distance are matched. Matching of vehicles 11 based on this rule in which vehicles that are close in distance are matched can be performed in the same way as the process in step S33.

[0116] In this manner, matching of the vehicles 11 is performed. Matching may be performed using a method other than the above-described matching method. Furthermore, matching may be performed taking into consideration factors such as those shown in FIG. 13. Alternatively, when a plurality of other vehicles 11-2 are matched with the host vehicle 11-1 by the above-described method, selection may be performed based on factors such as those shown in FIG. 13 in the process of narrowing down candidates to one or two other vehicles 11-2 from among the plurality of other vehicles 11-2.

[0117] FIG. 13 is a diagram showing another example of a matching method, in which matching conditions and effects are associated with each other. Vehicles 11 may be classified by "itinerary," and, for example, vehicles 11 with the same destination may be matched with each other, or vehicles 11 with the same upcoming schedule may be matched with each other. When matching is performed by itinerary, exchanging information about destinations can stimulate conversation, and schedule management and reminders can be performed.

[0118] Vehicles 11 may be classified using "occupant information." For example, vehicles 11 with similar occupant configurations may be matched. When matching is performed based on occupant information, matching can be performed so that all occupants can talk to people of the same generation, and vehicles 11 containing people who are relatively easy to talk to can be matched together.

[0119] The vehicles 11 may be classified using the "vehicle situation." For example, vehicles 11 that are close to the vehicles 11, such as mountains, the sea, or buildings, may be matched with each other. In this case, for example, the scenery can be a common topic of conversation, allowing the vehicles 11 to get to know each other.

[0120] As the "vehicle status," it is also possible to focus on speed, and match vehicles 11 with matching speeds. In this case, the exchange of information about traffic congestion can be used as a common topic to get to know each other. As the "vehicle information," it is also possible to match vehicles 11 with matching license plate regions, or vehicles 11 with matching vehicle models. In this case, it is also easy to find common topics, and get to know each other.

[0121] The vehicles 11 may be classified using "topics." For example, vehicles 11 may be matched with each other if the keywords in the conversations between the occupants match. In this case, the conversations are about the same topic, so the occupants can smoothly join in the conversation.

[0122] For example, vehicles 11 that have the same keywords set in advance may be matched with each other. In this case, the vehicles 11 can have interesting conversations and become friendly with each other.

[0123] The vehicles 11 may be classified using the "in-vehicle situation." For example, vehicles 11 whose occupants are viewing the same content may be matched with each other. In this case, the occupants can talk about common topics and become friendly with each other.

[0124] When matching vehicles 11 using information such as "itinerary," "occupant information," "vehicle status," "topics," and "in-vehicle status," the information can be obtained by referring to pre-registered parameters 91 (Figure 6) and departure-registered parameters 92 (Figure 7).

[0125] The above information may be acquired by referring to information stored in the vehicle information storage unit 75 and the occupant information storage unit 76 (FIG. 4).

[0126] The matching process based on the conditions shown in Figure 13 may be included as part of the process of the flowchart in Figure 9, or may be performed as a process to narrow down the final matching vehicle 11 from multiple vehicles 11 when more detailed matching is performed after matching has been performed according to the flowchart in Figure 9.

[0127] The matching process based on the conditions shown in Fig. 13 may be performed instead of matching based on vehicle distance when the process reaches step S39. If a matching vehicle 11 is not detected even based on the conditions shown in Fig. 13, the process may be changed to matching based on vehicle distance.

[0128] Once the vehicle 11 has been matched in this way, the process proceeds to step S14 (FIG. 8). In step S14, a static seating arrangement process is executed. The static seating arrangement process will be described with reference to FIGS. 14 to 17. FIGS. 14 to 17 are diagrams for explaining seating arrangement patterns. The explanation with reference to FIGS. 14 to 17 is for the host vehicle 11-1, and although the other vehicle 11-2 is not shown, the same seating arrangement is performed in the other vehicle 11-2 as in the host vehicle 11-1.

[0129] After the vehicles 11 are matched, the initial seating arrangement process between the vehicles 11 involves seating arrangement that reproduces relative positions or seating arrangement that takes age and gender into consideration. When the vehicle 11-1 and the other vehicle 11-2 are matched, a seating arrangement that reproduces the relative positional relationship of the occupants of the other vehicle 11-2 in the vehicle 11-1 is referred to as a seating arrangement that reproduces relative positions. By reproducing the relative positions, it is possible to present the conversation of the occupants of the other vehicle 11-2 without making the occupants of the vehicle 11-1 feel uncomfortable, and it is possible to enhance the presence of the occupants of the other vehicle 11-2.

[0130] 14 is a diagram showing seating arrangement patterns 1 and 2 that reproduce the relative positions. Referring to pattern 1 shown in FIG. 14, occupant 32a-1 is seated in seat 31a-1 of host vehicle 11-1, and occupant 32a-2 is seated in seat 31a-2 of other vehicle 11-2.

[0131] In the case of pattern 1, the occupant 32a-2 of the other vehicle 11-2 is placed in the seat 31b-1 of the host vehicle 11-1. The seats are arranged so that a virtual occupant 32a-2' of the other vehicle 11-2 is sitting next to the occupant 32a-1 in the host vehicle 11-1. In other words, when the other vehicle 11-2 speaks, the seating is arranged so that the sound image of the speaker is localized in the seat 31b-1 next to the occupant 32a-1.

[0132] 14, in the host vehicle 11-1, an occupant 32a-1 is seated in the seat 31a-1, and an occupant 32b-1 is seated in the seat 31b-1. In the other vehicle 11-2, an occupant 32a-2 is seated in the seat 31a-2, and an occupant 32b-2 is seated in the seat 31b-2.

[0133] In the case of pattern 2, the occupants 32a-2 and 32b-2 of the other vehicle 11-2 are respectively placed in seats 31c-1 and 31e-1 of the host vehicle 11-1. The seating arrangement is such that the occupants 32a-2' and 32b-2' are seated in the rear seats of the occupants 32a-1 and 32b-1 in the host vehicle 11-1.

[0134] In the other vehicle 11-2, the occupants 32a-2 and 32b-2 are seated next to each other. In the own vehicle 11-1, the occupants 32a-2' (sound images) and 32b-2' (sound images) corresponding to the occupants 32a-2 and 32b-2 are seated next to each other. In this way, seating is performed while maintaining the relative positions.

[0135] Patterns 1 and 2 are patterns in which, when the total number of occupants in the own vehicle 11-1 and the other vehicle 11-2 is within the number of seats in the own vehicle 11-1, virtual occupants of the other vehicle 11-2 are placed in vacant seats in the own vehicle 11-1 while maintaining the relative positions of the occupants 32 in the other vehicle 11-2.

[0136] 15 is a diagram showing seating arrangement patterns 3 and 4 that reproduce the relative positions. Pattern 3 is a pattern in which seats are arranged while maintaining the relative positional relationship when the total number of occupants in the host vehicle 11-1 and the other vehicle 11-2 is the same as the number of seats in the host vehicle 11-1.

[0137] 15, occupants 32a-1, 32b-1, 32c-1, and 32e-1 are seated in seats 31a-1, 31b-1, 31c-1, and 31e-1 of the host vehicle 11-1, respectively. Occupants 32a-2, 32b-2, 32c-2, and 32e-2 are seated in seats 31a-2, 31b-2, 31c-2, and 31e-2 of the other vehicle 11-2, respectively.

[0138] In the case of pattern 3, occupants 32a-2 and 32b-2 of the other vehicle 11-2 are placed in seats 31c-1 and 31e-1 of the host vehicle 11-1, respectively, but since occupants 32c-1 and 32e-1 are seated in seats 31c-1 and 31e-1, occupants 32a-2' and occupants 32b-2' are placed in positions slightly offset from occupants 32c-1 and 32e-1 so as not to overlap.

[0139] 15, the virtual occupant 32a-2' of occupant 32a-2 is disposed between seats 31c-1 and 31d-1, and the virtual occupant 32b-2' of occupant 32b-2 is disposed between seats 31d-1 and 31e-1. In this case, too, in the other vehicle 11-2, the occupants 32a-2 and 32b-2 are seated adjacent to each other, and in the host vehicle 11-1, the occupants 32a-2' and 32b-2' corresponding to the occupants 32a-2 and 32b-2 are seated adjacent to each other.

[0140] The occupant 32-2 of the other vehicle 11-2 is presented as a sound image to the occupant 32-1 of the host vehicle 11-1. It is possible to localize the sound image at a position where there is no seat. Therefore, as described above, it is also possible to place the sound image of the virtual occupant 32a-2' of the occupant 32a-2 between the seats 31c-1 and 31d-1, which are not seats.

[0141] Occupants 32c-2 and 32e-2 of the other vehicle 11-2 are seated in seats 31f-1 and 31h-1 of the host vehicle 11-1, respectively. In this case, too, occupants 32c-2 and 32h-2 are seated next to each other in the other vehicle 11-2, and occupants 32c-2' and 32h-2' corresponding to occupants 32c-2 and 32h-2 are also seated next to each other in the host vehicle 11-1.

[0142] Pattern 4 shown in Figure 15 is a pattern in which seats are allocated while maintaining the relative positional relationship when the total number of occupants in the vehicle 11-1 and the other vehicle 11-2 is greater than the number of seats in the vehicle 11-1.

[0143] 15, occupants 32a-1, 32b-1, 32c-1, and 32e-1 are seated in seats 31a-1, 31b-1, 31c-1, and 31e-1 of the host vehicle 11-1, respectively. Occupants 32a-2, 32b-2, 32c-2, 32d-2, 32e-2, 32f-2, and 32h-2 are seated in seats 31a-2, 31b-2, 31c-2, 31d-2, 31e-2, 31f-2, and 31h-2 of the other vehicle 11-2, respectively.

[0144] In the case of pattern 4, all three rows of seats in the other vehicle 11-2 are occupied by occupants 32-2. In this state, if pattern 2 or pattern 3 is applied to place the occupants 32-2 in the first row of the other vehicle 11-2 in the second row of the host vehicle 11-1 and place the occupants 32-2 in the second row of the other vehicle 11-2 in the third row of the host vehicle 11-1, and place the occupants 32-2 in the third row of the other vehicle 11-2 while maintaining the relative positions of the occupants 32, the occupants 32-2 will be placed outside the host vehicle 11-1.

[0145] Therefore, in pattern 4, the occupant 32-2 in the first row of the other vehicle 11-2 is placed in the first row of the host vehicle 11-1, the occupant 32-2 in the second row of the other vehicle 11-2 is placed in the second row of the host vehicle 11-1, and the occupant 32-2 in the third row of the other vehicle 11-2 is placed in the third row of the host vehicle 11-1.

[0146] Occupants 32a-2 and 32b-2 seated in the first row of the other vehicle 11-2 are seated in seats 31a-1 and 31b-1 of the host vehicle 11-1, respectively, but seats 31a-1 and 31b-1 are occupied by occupants 32a-1 and 32b-1. In this case, occupants 32a-2' and 32b-2' are seated slightly offset from occupants 32a-1 and 32b-1 so as not to overlap.

[0147] 15, a virtual occupant 32a-2' of occupant 32a-2 is located between seats 31a-1 and 31b-1 and closer to seat 31a-1. A virtual occupant 32b-2' of occupant 32b-2 is located between seats 31a-1 and 31b-1 and closer to seat 31b-1.

[0148] Occupants 32c-2, 32d-2, and 32e-2 seated in the second row of other vehicle 11-2 are respectively placed in seats 31c-1, 31d-1, and 31e-1 in the second row of host vehicle 11-1. Because occupants 32c-1 and 32e-1 are seated in seats 31c-1 and 31e-1, occupants 32c-2' and 32e-2' are placed slightly offset from occupants 32c-1 and 32e-1 so as not to overlap.

[0149] In the example shown in Figure 15, the virtual occupant 32c-2' of occupant 32c-2 is located outside seat 31c-1 (on the right side in the figure), and the virtual occupant 32e-2' of occupant 32e-2 is located outside seat 31e-1 (on the left side in the figure).

[0150] The occupants 32f-2 and 32h-2 seated in the third row of the other vehicle 11-2 are placed in the seats 31f-1 and 31h-1 in the third row of the host vehicle 11-1, respectively.

[0151] In this way, the seats are arranged so that the positional relationship of the occupant 32-2 of the other vehicle 11-2 is maintained also in the virtual occupant 32-2' (sound image) of the own vehicle 11-1.

[0152] 16 shows a case where three vehicles 11-1 to 11-3 are matched. Pattern 5 also shows a case where the number of seats in the host vehicle 11-1 is smaller than the number of seats in the other vehicle 11-2.

[0153] 16, in the host vehicle 11-1, seats 31a-1, 31b-1, 31c-1, and 31e-1 are occupied by occupants 32a-1, 32b-1, 32c-1, and 32e-1, respectively. The seats 31 in the host vehicle 11-1 are located in the first and second rows, for a total of five seats.

[0154] The seats 31 of the other vehicle 11-2 are arranged in rows 1 to 3, for a total of eight seats. Of these eight seats, seats 31a-2, 31b-2, 31c-2, and 31e-2 are occupied by occupants 32a-2, 32b-2, 32c-2, and 32e-2, respectively.

[0155] The seats 31 of the other vehicle 11-3 are arranged in rows 1 to 3, for a total of eight seats. Of these eight seats, seats 31a-3, 31b-3, 31c-3, 31d-3, 31e-3, 31f-3, and 31h-3 are occupied by occupants 32a-3, 32b-3, 32c-3, 32d-3, 32e-3, 32f-3, and 32h-3, respectively.

[0156] Virtual occupants 32a-2' and 32b-2' corresponding to occupants 32a-2 and 32b-2 seated in the first row of the other vehicle 11-2 are positioned between seats 31a-1 and 31b-1 of the host vehicle 11-1, respectively, as in pattern 4 (Figure 15).

[0157] A virtual occupant 32c-2' of occupant 32c-2 seated in the second row of the other vehicle 11-2 is positioned outside seat 31c-1 (on the right side in the figure), and a virtual occupant 32e-2' of occupant 32e-2 is positioned outside seat 31e-1 (on the left side in the figure).

[0158] Occupants 32a-3 and 32b-3 seated in the first row of the other vehicle 11-3 are respectively positioned between seats 31a-1 and 31b-1 of the host vehicle 11-1, but virtual occupants 32a-2' and 32b-2' are already positioned there. In this case, virtual occupants 32a-3' and 32b-3' of occupants 32a-3 and 32b-3 are positioned at positions shifted from these occupants 32a-2' and 32b-2'.

[0159] A virtual occupant 32d-2' of the occupant 32d-2 seated in the second row of the other vehicle 11-3 is placed in the seat 31d-1 because the seat 31d-1 is vacant.

[0160] Occupants 32c-3 and 32e-3 seated in the second row of the other vehicle 11-3 are positioned between seats 31c-1 and 31e-1 of the host vehicle 11-1, respectively, but because occupants 32c-1 and 32e-1 of the host vehicle 11-1 are seated, virtual occupants 32c-2' and 32e-2' are positioned outside occupants 32a-1 and 32b-1, respectively. Virtual occupants 32c-3' and 32e-3' of occupants 32c-3 and 32e-3 are positioned at positions offset from these occupants 32c-1, 32e-1, 32c-2' and 32e-2'.

[0161] In the example shown in FIG. 16, virtual occupants 32c-3' and 32e-3' of occupants 32c-3 and 32e-3 are respectively positioned behind occupants 32c-1 and 32e-1 of the host vehicle 11-1.

[0162] Occupants 32f-3 and 32h-3 seated in the third row of the other vehicle 11-3 are respectively positioned behind seats 31c-1 and 31e-1 of the host vehicle 11-1. Since the host vehicle 11-1 does not have a third row of seats, virtual occupants 32f-3' and 32h-3' of occupants 32f-3 and 32h-3 are positioned behind the second row.

[0163] In the example shown in Figure 16, occupants 32c-3' and 32e-3' are positioned at the rear of the second row, so occupants 32f-3' and 32h-3' are positioned further rearward of occupants 32c-3' and 32e-3'.

[0164] In this way, seats are allocated so that sound images do not overlap. Seats are allocated so that the positional relationship of the occupant 32-2 of the other vehicle 11-2 is maintained in the virtual occupant 32-2' (sound image) of the host vehicle 11-1. In addition, in the example of pattern 5, seats are allocated so that the positional relationship of the occupant 32-3 of the other vehicle 11-3 is also maintained in the virtual occupant 32-3' (sound image) of the host vehicle 11-1.

[0165] In the explanation with reference to Figures 14 to 16, the seating arrangement in the own vehicle 11-1 has been used as an example, but similar processing related to seating arrangement is also performed in other vehicles 11-2 and 11-3, and the positions of virtual occupants (sound images) of other vehicles are set.

[0166] FIG. 17 is a diagram explaining how to assign seats according to age, gender, role, etc. Roles include the role of driving (chauffeur), the role of navigating, etc. Groups are formed based on age, gender, role, etc., and seats are assigned. When people are of similar age, gender, or have the same role (such as driver), they have more common topics to talk about and can stimulate conversation.

[0167] As shown in the upper diagram of Fig. 17, the driver occupant 32a-1 is seated in seat 31a-1 of the host vehicle 11-1, adult occupant 32b-1 is seated in seat 31b-1, and child occupant 32e-1 is seated in seat 31e-1. In Fig. 17, the shaded circles represent child occupants 32.

[0168] The other vehicle 11-2 has a driver occupant 32a-2 sitting in seat 31a-2, a child occupant 32b-2 sitting in seat 31b-1, a child occupant 32c-2 sitting in seat 31c-2, and an adult occupant 32e-2 sitting in seat 31e-2.

[0169] In such a case, the seats are arranged as shown in the lower diagram of FIG. 17. The drivers are divided into groups based on their roles, and the seats are arranged so that they are close to each other. A virtual occupant 32a-2' of the occupant 32a-2 who is the driver of the other vehicle 11-2 is arranged in the vicinity of the occupant 32a-1 who is the driver of the own vehicle 11-1, at a position diagonally to the lower left in the diagram. Similarly, in the other vehicle 11-2, a virtual occupant 32a-1' of the occupant 32a-1 who is the driver of the own vehicle 11-1 is arranged in the vicinity of the occupant 32a-2 who is the driver of the other vehicle 11-2, at a position on the left in the diagram.

[0170] The passengers are divided into groups by age, and seats are allocated so that adults are positioned close to each other and children are positioned close to each other. Since adult occupant 32b-1 is seated in seat 31b-1 of host vehicle 11-1, virtual occupant 32e-2' of adult occupant 32e-2 of other vehicle 11-2 is positioned near occupant 32b-1. In the example shown in the lower diagram of FIG. 17, occupant 32e-2' is positioned diagonally to the lower right of occupant 32b-1.

[0171] Similarly, in the other vehicle 11-2, a virtual occupant 32b-1' of the adult occupant 32b-1 of the host vehicle 11-1 is placed near the adult occupant 32e-2 of the other vehicle 11-2, on the right side of the drawing. Because the seat 31d-2 to the right of the occupant 32e-2 of the other vehicle 11-2 is vacant, the virtual occupant 32b-1' may be placed in the seat 31d-1, but in the example shown in the lower drawing of FIG. 17, the virtual occupant 32b-1' is placed in a position slightly closer to the occupant 32e-2.

[0172] In other words, the virtual occupant 32b-1' is positioned a little distance away from the child of the occupant 32c-2 of the other vehicle 11-2. In this way, the seats are allocated in a way that occupants belonging to the same age group are positioned closer to each other, while occupants belonging to different age groups are not seated too close to each other.

[0173] Because child occupant 32e-1 is seated in seat 31e-1 of host vehicle 11-1, virtual occupants 32b-2' of child occupant 32b-2 and 32c-2' of child occupant 32c-2 of other vehicle 11-2 are positioned near occupant 32b-1. In the example shown in the lower diagram of Figure 17, seats 31c-1 and 31d-1 to the right of occupant 32e-1 are vacant, so occupants 32b-2' and occupants 32c-2' are positioned in seats 31c-1 and 31d-1, respectively.

[0174] If seats 31c-1 and 31d-1 to the right of occupant 32e-1 are vacant, seat 31c-1 or seat 31d-1 is selected to accommodate occupant 32b-2' or occupant 32c-2', so as to reproduce the positional relationship between occupant 32b-2 and occupant 32c-2 in the other vehicle 11-2.

[0175] 17, when occupant 32c-2 is viewed from occupant 32b-2, occupant 32c-2 is seated diagonally below and to the right of occupant 32b-2. This positional relationship corresponds to a state in which occupant 32c-2 is seated to the right of occupant 32b-2 if occupants 32b-2 and 32c-2 were seated side by side.

[0176] Therefore, in this case, the seats (positions of the sound images) of the virtual occupants 32b-2' and 32c-2' in the vehicle 11-1 are set so as to recreate a state in which the occupant 32c-2 is seated to the right of the occupant 32b-2. That is, as shown in Fig. 17, the occupant 32b-2' is seated in the seat 31d-1, and the occupant 32c-2' is seated in the seat 31c-1.

[0177] If an occupant were seated in seat 31c-1, the situation in which occupant 32c-2 is seated to the right of occupant 32b-2 can be reproduced in the vehicle 11-1, for example, by placing virtual occupant 32c-2' in seat 31f-1, which is located diagonally below and to the right of seat 31d-1 in which virtual occupant 32b-2' is seated.

[0178] Similarly, in the other vehicle 11-2, a virtual occupant 32e-1' of the child occupant 32e-1 of the host vehicle 11-1 is positioned near the child occupants 32b-2 and 32c-2 of the other vehicle 11-2. In the example shown in the lower diagram of Figure 17, the virtual occupant 32e-1' is positioned between the occupants 32b-2 and 32c-2. The occupants 32b-2, 32c-2, and the virtual occupant 32e-1' are positioned so as to form a straight line in a diagonal direction.

[0179] The seating pattern described with reference to Fig. 17 is a seating arrangement pattern in which children are seated close together, creating an environment in which the children can easily talk to each other. In the seating pattern described with reference to Fig. 17, the seating arrangement for adults is a pattern in which the relative positions of the adults are taken into consideration, enhancing the presence of the virtual passengers. In this way, the seating pattern can be changed depending on the age of the occupants, such as adults and children.

[0180] In the example shown in Figure 17, seating patterns are explained for when people are divided into two groups, adults and children. However, as shown in Figure 18, it is also possible to divide people into two or more groups and apply a seating pattern appropriate for each group to make seating arrangements.

[0181] Instead of placing sound sources at normal seat positions, virtual sound sources may be placed between actual seat rows, between seats, or at the boundaries (or near the boundaries) between seats, such as for occupant 32a-2', occupant 32e-2', and occupant 32b-1'.

[0182] The groups shown in Figure 18 are six in total: a preschool group aged 0-5, an elementary school group aged 6-12, a junior high / high school group aged 13-18, a university student group aged 19-24, a working adult group aged 25-60, and a senior group aged 60 or older. Seating allocation processing may be performed for these six groups. Note that the groups listed here are just an example, and the age range and number of groups can be changed as appropriate.

[0183] A case where static seating arrangement is performed by switching between two patterns, namely, a seating arrangement pattern (rule) that reproduces relative positional relationships and a seating arrangement pattern (rule) that takes age and other factors into consideration, will be described with reference to the flowchart in Fig. 19. The flowchart in Fig. 19 is a flowchart that explains the details of the static seat arrangement process in step S14 (Fig. 8). The process of the flowchart in Fig. 19 is performed based on the rules stored in the seating rule storage unit 82 (Fig. 4) of the seating arrangement processing unit 71.

[0184] In step S81, the number of passengers in the host vehicle 11-1 is obtained. In step S82, the number of passengers in the host vehicle 11-1 and the number of passengers in the matched other vehicle 11-2 are obtained. If multiple other vehicles 11-2 are matched with the host vehicle 11-1, the number of passengers is obtained from each of the multiple other vehicles 11-2. In step S82, the total number of passengers in the host vehicle 11-1 and the other vehicles 11-2 is obtained.

[0185] In step S83, it is determined whether the total number of occupants is within the capacity of the host vehicle 11-1. If it is determined in step S83 that the total number of occupants is within the capacity of the host vehicle 11-1, the process proceeds to step S84.

[0186] In step S84, the occupants of the other vehicle 11-2 are placed in the vacant seats in order. For example, as shown in FIG. 20, a ranking can be assigned, and the occupants of the other vehicle 11-2 can be placed in the vacant seats 31 in that order. Referring to FIG. 20, the vehicle 11 has eight seats, seats 31a to 31h. Eight seats is the passenger capacity of this vehicle 11.

[0187] The seating order is seat 31a, seat 31b, seat 31c, seat 31d, seat 31e, seat 31f, seat 31g, and seat 31h. Normally, the driver is seated in seat 31a, which is the driver's seat, so seats 31b to 31h remain vacant. For example, if passengers are seated in seats 31a and 31b and seats 31c to 31h are vacant, passengers from the other vehicle 11-2 are seated in order, starting with seat 31c.

[0188] On the other hand, if it is determined in step S83 that the total number of occupants is greater than the seating capacity of the host vehicle 11-1, the process proceeds to step S85. In step S85, the ages of all occupants are measured. The ages of the occupants may be registered in advance by the user, and the registered information may be used. The ages of the occupants may be estimated by analyzing images captured by a camera installed in the vehicle 11 and performing face recognition.

[0189] In step S86, grouping by age is performed. For example, as described with reference to Fig. 18, six age groups are set, and the age information acquired in the process of step S85 is used to determine which group the person belongs to.

[0190] In step S87, it is determined whether there are multiple groups each having two or more members. If it is determined in step S87 that there are groups each having two or more members, the process proceeds to step S88. In step S88, seating arrangement processing based on age and role is executed. This seating arrangement processing is the seating arrangement processing an example of which was described with reference to FIG. 17.

[0191] The seating arrangement process based on age and role executed in step S88 will now be described with reference to the flowchart shown in Figure 21. In step S101, grouping based on age is performed. This grouping has been performed in step S86 (Figure 19), so the results of that process can be used.

[0192] In step S102, it is determined whether or not a group exists in the matched other vehicle 11-2, and if no group exists, a process of consolidating the groups into groups of similar ages is executed. For example, if no group exists in the other vehicle 11-2, in other words, if there are multiple groups each containing only one person, the groups are consolidated so that each group contains two or more people. If a group exists in the other vehicle 11-2, the process of step S102 may be skipped.

[0193] In step S103, one group is selected as the processing target, and it is determined whether the occupants in the group selected as the processing target are seated across multiple rows of seats in the vehicle 11. For example, if the occupants in the group are seated in only one predetermined row of the vehicle 11, the determination in step S103 is NO. For example, if the occupants in the group are seated in the first and second rows of the vehicle 11, the determination in step S103 is YES.

[0194] 17 again, for example, occupants 32b-2 and 32c-2 in the other vehicle 11-2 belong to a group called children, and occupant 32b-2 is seated in row 1, and occupant 32c-2 is seated in row 2. In such a case, in step S103, it is determined that the occupants in the group are spread across multiple rows.

[0195] For example, occupants 32a-1 and 32b-1 in the host vehicle 11-1 belong to a group called adults, and both occupants 32a-1 and 32b-1 are seated in row 1. In this case, in step S103, it is determined that the occupants in the group do not span multiple rows.

[0196] In step S103, if it is determined that the occupants in the group do not span multiple rows, in other words, if it is determined that the occupants in the group are seated in seats 31 in a specific row, processing proceeds to step S104.

[0197] In step S104, the seats are arranged so as to maintain the front-to-back relationship with the row number of the other vehicle 11-2. In this case, since the occupants included in the group are seated in the same row of the other vehicle 11-2, the seating arrangement is such that the positional relationship within the row in which they are seated is also maintained within the own vehicle 11-1.

[0198] When the process of step S104 is completed, the process proceeds to step S105. In step S105, it is determined whether the process has been executed for all groups, and if it is determined that there is a group that has not yet been processed, the process returns to step S103, and the subsequent processes are repeated.

[0199] On the other hand, if it is determined in step S105 that the processing has been executed for all groups, the processing of the flowchart shown in FIG. 21 ends, and the processing returns to the processing of the flowchart shown in FIG.

[0200] On the other hand, if it is determined in step S103 that the occupants in the group are spread out in multiple rows, the process proceeds to step S106. In step S106, it is determined whether the group being processed is a group of university students or older. If it is determined in step S106 that the group being processed is not a group of university students or older, the process proceeds to step S107.

[0201] In step S107, the average age is calculated for each row. In step S108, occupants whose average age is close to the average age of the occupants of the other vehicle 11-2 belonging to the group being processed are seated around the occupants of the host vehicle 11-1 belonging to the group being processed.

[0202] In the example shown in Figure 17, the processing is for a group of children, and for example, occupants 32b-1 and 32c-1 belonging to the group of children in another vehicle 11-2 are each positioned around occupant 32e-1 belonging to the group of children in the host vehicle 11-1.

[0203] When the process of step S108 ends, the process proceeds to step S105. The process from step S105 onwards has already been explained, so the explanation thereof will be omitted.

[0204] On the other hand, if it is determined in step S106 that the group being processed is a group of university students or above, the process proceeds to step S109.

[0205] In step S109, the occupants 32 included in the group being processed are labeled with roles such as driver, navigator, etc. In step S110, the labeling results are used to arrange occupants 32 with the same role close to each other.

[0206] For example, in the example shown in Figure 17, occupant 32a-1 of host vehicle 11-1 and occupant 32a-2 of other vehicle 11-2 are labeled as drivers, so a virtual occupant 32a-2' of occupant 32a-2 of other vehicle 11-2 is placed near occupant 32a-1 of host vehicle 11-1.

[0207] When the process of step S110 ends, the process proceeds to step S105. The process from step S105 onwards has already been explained, so the explanation thereof will be omitted.

[0208] Returning to the explanation with reference to the flowchart in Fig. 19, if it is determined in step S87 that there are no groups including two or more occupants, the process proceeds to step S89. In step S89, an arrangement process taking relative positions into consideration is executed. The seat arrangement process taking relative positions into consideration, which is executed in step S89, will be described with reference to the flowchart in Fig. 22.

[0209] In step S121, the number of seat rows S1 of the vehicle 11-1 is acquired. The number of seat rows is acquired by, for example, referring to information registered in the pre-registered parameters 91 (FIG. 6).

[0210] In step S122, the number of seat rows S2 of the other vehicle 11-2 is acquired. For example, the number of seat rows S2 is acquired by referring to information registered in the pre-registered parameters 91 (FIG. 6) of the other vehicle 11-2 and being supplied from the other vehicle 11-2.

[0211] In step S123, the number of seat rows S2 is divided by the number of seat rows S1 to obtain a quotient a and a remainder b.

[0212] In step S124, it is determined whether there is an empty seat in the row of seats (referred to as row s1) in the subject vehicle 11-1. The initial value of row s1 is 1. The row of seats in the other vehicle 11-2 is referred to as row s2, and the initial value of row s2 is 1. If it is determined in step S124 that there is no empty seat in row s1, the process proceeds to step S125.

[0213] In step S125, seats are allocated between the rows of seats as shown in Fig. 23. Fig. 23 shows a case where there are no vacant seats in rows s1 and (s1+1) of the host vehicle 11-1, and a total of three occupants seated in rows s2 to (s2+a) of the other vehicle 11-2 are allocated in the host vehicle 11-1. In this case, a total of three virtual occupants of the other vehicle 11-2 are allocated between rows s1 and (s1+1) of the host vehicle 11-1.

[0214] Even when virtual passengers are arranged between the rows, the virtual passengers are arranged so that the positional relationship among the passengers in the other vehicle 11-2 is reproduced within the host vehicle 11-1 as well. In the example of FIG. 23, since the three passengers in the other vehicle 11-2 are seated on the left, center, and right when viewed in the lateral direction, the virtual passengers are arranged so as to be on the left, center, and right when viewed in the lateral direction within the host vehicle 11-1 as well.

[0215] When there are a plurality of virtual passengers to be arranged, a virtual seat is set between the rows where the virtual passengers are arranged, and by executing processing equivalent to the processing of steps S126 to S129 described later for the virtual seat, the virtual passengers may be arranged. Steps S126 to S129 are processing for setting the positions of virtual passengers so that the positional relationship of the passengers in the other vehicle 11-2 is reproduced in the host vehicle 11-1.

[0216] When s1 < b, the (s2 + a)th column becomes the (s2 + a + 1)th column.

[0217] In this way, by arranging virtual passengers between the rows, in other words, by localizing the sound image, it is possible to prevent the sound image from concentrating on the front or rear in the vehicle 11 with a small number of seat rows. In particular, when there is a large difference between the number of seat rows S1 of the host vehicle 11-1 and the number of seat rows S2 of the other vehicle 11-2, there is a possibility that the sound image may concentrate on the front or rear, but by executing the processing of steps S124 and S125, it is possible to prevent the sound image from concentrating.

[0218] On the other hand, in step S124, when it is determined that there is an empty seat in the s1th column that is the processing target, the processing proceeds to step S126. In step S126, it is determined whether or not condition A is satisfied. Condition A will be described with reference to FIG. 24.

[0219] Condition A is a condition that the empty seat position in row s1 and the seat position in row s2 are the same. When occupant 32-2 sitting in the middle seat 31-2 in row s2 of other vehicle 11-2 is placed in host vehicle 11-1, the seat in row s1 of host vehicle 11-1 that corresponds to the seat position where occupant 32-2 is sitting is seat 31-1, and it is determined whether seat 31-1 is empty. If the middle seat 31-1 in row s1 of host vehicle 11-1 is empty, it is determined that condition A is met.

[0220] If it is determined in step S126 that condition A is not satisfied, the process proceeds to step S127. In this case, the seat of the host vehicle 11-1 corresponding to the seat occupied by the occupant 32-2 of the other vehicle 11-2 is not vacant. In such a case, a virtual occupant 32-2' corresponding to the occupant 32-2 of the other vehicle 11-2 is placed in an empty seat close to the seat of the host vehicle 11-1 corresponding to the seat occupied by the occupant 32-2 of the other vehicle 11-2.

[0221] On the other hand, if it is determined in step S126 that condition A is satisfied, the process proceeds to step S128. In step S128, a seat is allocated to the vacant seat. In this case, since the vacant seat position in row s1 of the host vehicle 11-1 and the seating position in row s2 of the other vehicle 11-2 are the same, a virtual occupant 32-2' corresponding to the occupant 32-2 of the other vehicle 11-2 is allocated to the vacant seat.

[0222] Once the seat position setting rule for arranging the passengers of the other vehicle 11-2 in the own vehicle 11-1 is set in step S127 or step S128, seat allocation is executed in step S129. This will be described with reference to FIG.

[0223] When arranging the passengers 32c-2 and 32e-2 of the other vehicle 11-2 in the host vehicle 11-1, since the seat in the host vehicle 11-1 corresponding to the seat where the passenger 32c-2 of the other vehicle 11-2 is sitting is an empty seat, in step S126, it is determined as YES, and in step S128, it is determined to arrange in the empty seat. Therefore, as shown in FIG. 25, the passenger 32c-2 of the other vehicle 11-2 is arranged as a virtual passenger 32c-2' in the rightmost empty seat in the host vehicle 11-1.

[0224] The seat in the host vehicle 11-1 corresponding to the seat where the passenger 32e-2 of the other vehicle 11-2 is sitting is occupied by the passenger 32e-1 and is not an empty seat. Therefore, in step S126, it is determined as NO, and in step S127, it is determined to arrange in an empty seat near the passenger 32e-1. Therefore, as shown in FIG. 25, the passenger 32e-2 of the other vehicle 11-2 is arranged as a virtual passenger 32e-2' in the empty seat on the right side of the seat where the passenger 32e-1 in the host vehicle 11-1 is sitting.

[0225] Return to the explanation referring to the flowchart shown in FIG. 22. When the passengers of the other vehicle 11-2 are arranged in step S125 or step S129, the process proceeds to step S130. In step S130, it is determined whether the column s1 of the seat to be processed in the host vehicle 11-1 is smaller than the remainder b.

[0226] If it is determined in step S130 that s1 < b is not satisfied, the process proceeds to step S131. In step S131, the quotient a is added to the column s2 to be processed in the other vehicle 11-2, and a newly processed column s2 is set.

[0227] On the other hand, if it is determined in step S130 that s1 < b is satisfied, the process proceeds to step S132. In step S132, (quotient a + 1) is added to the column s2 to be processed in the other vehicle 11-2, and a newly processed column s2 is set.

[0228] When the new s2 sequence to be processed is set in step S131 or step S132, the process proceeds to step S133. In step S133, 1 is added to the s1 sequence to be processed for the host vehicle 11-1, and the new s1 sequence to be processed is set.

[0229] In step S134, it is determined whether the number of columns S1 of the host vehicle 11-1 is greater than the column s1 to be processed. If it is determined in step S134 that S1>s1 is satisfied, the process returns to step S124, and the subsequent processes are repeated. That is, in this case, it is determined that there remains column s1 to be processed, and the process returns to step S124 to process the unprocessed column s1.

[0230] On the other hand, if it is determined in step S134 that S1>s1 is not satisfied, the processing of the flowchart shown in FIG. 22 is terminated, and the processing returns to the processing of the flowchart shown in FIG.

[0231] Referring to the flowchart shown in FIG. 19, when the seating arrangement process taking the relative positions into consideration is completed in step S89, the process returns to the process of the flowchart shown in FIG.

[0232] By performing seating arrangement as shown in the flowchart in Fig. 19, seating arrangement can be performed with priority given to presenting sound images from empty seats. When a sound image is presented in a position where seating is not possible, for example, between rows in the above example, there is a possibility that the sense of presence will be reduced compared to when a sound image is presented from an empty seat. Therefore, by performing the processing as described above, seating arrangement is performed with priority given to presenting sound images from empty seats.

[0233] Specifically, as described with reference to the flowchart shown in Fig. 19, if the total number of occupants fits within the capacity of the host vehicle 11, the occupants are placed in the vacant seats in order. This allows the occupants of the other vehicle 11-2 to feel as if they are sitting in the seats of the host vehicle 11-1.

[0234] If the total number of crew members exceeds the fixed number, they will be grouped based on the age of everyone. If there are multiple groups with two or more crew members of the same age, it is determined that grouping by age is possible, and seating will be done by age and role. This makes it easier to find common topics to talk about, and crew members who are likely to have lively conversations can be placed in positions where they can easily talk.

[0235] When grouping by age is difficult, seating is performed taking into consideration relative positions, thereby reproducing the positional relationship of the occupants of the other vehicle 11-2 to the own vehicle 11-1.

[0236] Referring to the flowchart shown in FIG. 8, once static seating arrangement has been performed in step S14, sound image localization processing is executed based on the arrangement (step S15).

[0237] In step S15, the passengers start talking to each other. When the passengers start talking to each other, a process for localizing a sound image also starts. The process for localizing a sound image will be described later.

[0238] If the conversation does not go well after the call has started, a determination is made in step S16 as to whether or not to perform dynamic seating changes in order to change the seating arrangement or to perform vehicle 11 matching again.

[0239] If it is determined in step S16 that dynamic seating arrangement should be performed, the process proceeds to step S17. In step S17, dynamic seating arrangement processing is performed. The processing related to the determination in step S16 and the dynamic seating arrangement processing in step S17 will be described with reference to the flowchart in FIG.

[0240] In step S151, it is determined whether or not the occupant 32-1 of the host vehicle 11-1 has responded to the conversation of another group. As described above, the groups are divided based on age, role, etc. In step S151, it is determined whether or not the occupant 32 classified into a predetermined group has responded to the conversation of the occupant 32 classified into another group.

[0241] If it is determined in step S151 that the occupant 32 has responded to the conversation of another group, the process proceeds to step S152. In step S152, the occupant 32 who responded to the conversation is re-seated near the occupant 32 of the group to which the occupant 32 responded. By performing such re-seating, the occupant 32 can be added to a group with whom he or she can have an interesting conversation.

[0242] On the other hand, if it is determined in step S151 that there is no reaction to the conversation of another group, the process proceeds to step S153. In step S153, it is determined whether or not the user has arrived at (comes close to) a point on the itinerary. An example of a point on the itinerary may be a destination for a drive, a place where the user plans to eat, or the like.

[0243] If it is determined in step S153 that the trip has arrived at a point on the itinerary, the process proceeds to step S154. In step S153, seating is rearranged to place drivers close to each other. By rearranging the seats in this way, it becomes easier for people managing the itinerary to check and manage the itinerary.

[0244] On the other hand, if it is determined in step S153 that the point is not a point on the itinerary, the process proceeds to step S155. In step S155, it is determined whether or not anyone has woken up.

[0245] If it is determined in step S155 that someone has woken up, the process proceeds to step S156. In step S156, the person who woke up is reassigned to a group whose average age is closest to the age of the person who woke up. By performing such seating reassignment, the person who woke up can smoothly participate in the conversation.

[0246] On the other hand, if it is determined in step S155 that no one has woken up, the process proceeds to step S157, where it is determined whether or not any one has gone to sleep.

[0247] If it is determined in step S157 that someone has fallen asleep, the process proceeds to step S158. In step S158, seating is rearranged so that the center of gravity of the group is moved away from the sleeping person. By rearranging the seats in this way, the sleeping person can sleep quietly, while those who were talking can continue their conversation.

[0248] In this way, by having seating reassigned when certain conditions are met, such as responding to a conversation in another group or arriving at a point on the itinerary, crew member 32 can naturally join in on conversations that interest them or start necessary conversations.

[0249] The decision as to whether or not to reassign seats may be made based on conditions other than those described above, or conditions other than those described above may be set and a decision may be made based on those conditions.

[0250] For example, it may be possible to determine whether or not there is a group having a conversation that includes the same keyword, and if so, rearrange the seats so that the person is included in that group. In this case, it becomes possible to interact with more people.

[0251] For example, it may be possible to determine whether or not a person has started eating, and if it is determined that the person has started eating, to relocate the person near the person who is eating, in which case there is a common topic about eating and conversation can begin.

[0252] On the other hand, if it is determined in step S157 that no one has fallen asleep, the process proceeds to step S159. In step S159, it is determined whether or not the frequency of speech has decreased. Whether or not the frequency of speech has decreased can be determined by determining whether or not the conversation is getting more lively. Detection of the increase in conversation will be described later, and the process from step S159 onwards will be described first.

[0253] If it is determined in step S159 that the speech frequency has not decreased, the process of the flowchart shown in Fig. 26 ends, and the process returns to the process of the flowchart shown in Fig. 8. If it is determined that the speech frequency has not decreased, in other words, that the excitement level has not decreased, no seating is rearranged so as not to interrupt the lively conversation.

[0254] On the other hand, if it is determined in step S159 that the speech frequency has decreased, the process proceeds to step S160. In step S160, it is determined whether the content has changed. If it is determined in step S160 that the content has changed, the process proceeds to step S161. The content may be, for example, moving images such as movies or dramas, still images such as photographs, music, etc. It may also be determined whether the content of the conversation has changed, including the topic of the conversation.

[0255] In step S161, the group is reclassified into groups that fit the age and gender targeted by the changed content and other groups, and seating is then assigned to the reclassified groups. This type of seating reassignment allows people of the same generation to have fun talking about the content.

[0256] On the other hand, if it is determined in step S160 that the content has not been changed, the process proceeds to step S162. In step S162, a process is executed to increase the volume from the far seat. In step S163, it is determined whether or not there has been a reaction to the conversation of another group within T seconds.

[0257] When the conversation starts to lose momentum, seating is reassigned, but before this, time is provided for crew members 32 to spontaneously find a conversation that they are interested in. When a conversation that interests them is found, seating is reassigned so that they are included in the group that is having that conversation.

[0258] Therefore, in step S162, the volume of the conversation from the distant seat is increased to create a situation in which the content of the conversation can be easily heard by the occupant 32. In step S163, when such a situation continues for a predetermined time T, it is determined whether the occupant 32 has become interested in the conversation of the other group.

[0259] If it is determined in step S163 that the occupant has responded to the conversation of another group, the process proceeds to step S164. In step S164, the occupant is reallocated to the group that responded. By reallocating the seats in this way, the occupant can join in the conversation that interests them.

[0260] On the other hand, if it is determined in step S163 that there is no reaction to the conversations of other groups, the process proceeds to step S165. In step S165, static seating arrangement is performed within the group whose speech frequency has decreased. Since static seating arrangement can be performed in the same manner as the static seating arrangement process in step S14, a description thereof will be omitted here.

[0261] Once the processing related to the dynamic seating arrangement has been executed in this manner, the processing returns to the processing of the flowchart shown in FIG.

[0262] If it is determined in step S16 (FIG. 8) that a dynamic seat change is not to be performed, the process proceeds to step S18. In step S18, the speaker (occupant 32-2) speaking in the other vehicle 11-2 is identified, and in step S19, the seat 31-1 in the host vehicle 11-1 to which the identified speaker is assigned is identified. In step S20, speech is output from the identified seat 31-1.

[0263] The processing of steps S18 to S20 is an example of processing when presenting a sound image. For example, as shown in A of Fig. 3, if a seat speaker is provided for each seat, a situation can be created in which a virtual occupant is seated in seat 31 by outputting sound from speaker 33 provided in seat 31.

[0264] The sound image localization processing in step S15 may be executed as the processing of steps S18 to S20. Furthermore, if the processing of steps S18 to S20 is the sound image localization processing, the sound image localization processing in step S15 may also be the same as the processing of steps S18 to S20.

[0265] In step S21, it is determined whether or not to change the matched vehicle 11. The determination of whether or not to change the matched vehicle 11 can be made using the same conditions as those for determining whether or not to perform dynamic seat change in step S16. For example, in the dynamic seat change processing in step S16 (flowchart in FIG. 26), determinations are made in steps S151, S153, S155, and S157, and if two or more determination results are YES, it may be determined that re-matching of the vehicle 11 is to be performed.

[0266] For example, the degree of excitement may be detected numerically, and when it is determined that the degree of excitement has fallen below a predetermined threshold, it may be determined that re-matching of the vehicle 11 is to be performed. For example, when a predetermined time has elapsed since the matching was performed, re-matching of the vehicle 11 may be performed so that the vehicle 11 can enjoy conversation with new people.

[0267] If it is determined in step S21 that the matched vehicle 11 is to be changed, the process returns to step S12, and the subsequent processes are repeated.

[0268] On the other hand, if it is determined in step S21 that the matched vehicle 11 will not be changed, the processing of the flowchart shown in FIG. 8 is ended.

[0269] <About detecting bulges> The process of determining whether the speech frequency has decreased in step S159 (FIG. 26) will be described. A decrease in speech frequency means that the excitement has decreased. Therefore, an information processing device for detecting excitement will be described with reference to FIG.

[0270] When detecting excitement, the input unit 61 of the information processing device 51 (FIG. 4) is configured to include a camera 201 and a microphone 202, and the information processing unit 64 is configured to include an excitement detection unit 211.

[0271] The excitement detection unit 211 includes a face detection dictionary storage unit 221, an expression dictionary storage unit 222, a face detection unit 223, an expression digitization unit 224, a voice recognition dictionary storage unit 225, a volume detection unit 226, a voice-to-text unit 227, and a detection information storage unit 228.

[0272] The processing of the excitement detection unit 211 shown in Fig. 27 will be described with reference to the flowchart of Fig. 28. In step S201, an image is captured by the camera 201 of the input unit 61, and the captured image is acquired. The camera 201 is provided at a position where it can capture images of the interior of the vehicle 11 and the occupants.

[0273] The image acquired in step S201 is supplied to the face detection unit 223. In step S202, the face detection unit 223 detects the face of an occupant by analyzing the supplied image. Information required for detection is stored in the face detection dictionary storage unit 221. The face detection dictionary storage unit 221 stores, for example, a dictionary for face detection obtained by machine learning. If an image for face authentication is registered in the pre-registration parameters 91 (FIG. 6), that information may be used to perform face detection.

[0274] In step S203, the facial expression quantification unit 224 detects the facial expression of the face detected by the face detection unit 223 and quantifies the detected facial expression. A smile is detected as the facial expression, and a numerical value between 0 and 100% is calculated as the smile degree. In step S205, the calculated smile degree is stored in the detected information accumulation unit 228.

[0275] In step S206, the microphone 202 of the input unit 61 acquires audio data of the voice of the occupant 32. The acquired audio data is supplied to the volume detection unit 226.

[0276] In step S207, the volume detection unit 226 analyzes the audio data and calculates the volume. This volume calculation is performed for a predetermined time period T seconds, and the average volume for T seconds is calculated (step S208).

[0277] The voice data is also supplied to the voice-to-text unit 227. In step S209, the voice-to-text unit 227 refers to a dictionary that is stored in the voice recognition dictionary storage unit 225 and is used for voice recognition, analyzes the voice data, and converts the contents of the occupant's speech into text.

[0278] In step S210, the speech-to-text conversion unit 227 detects keywords from the speech data that has been converted into text.

[0279] In step S211, the detected keywords and information related to the volume are stored in the detected information storage unit 228. The keywords stored in the detected information storage unit 228 can be used in the above-mentioned vehicle matching process and in the process of grouping the occupants 32.

[0280] In step S212, it is determined whether the smile level is low and the average volume is low. If it is determined in step S212 that the smile level is low and the average volume is low, the process proceeds to step S213. In step S213, a determination result is made that the excitement is low.

[0281] On the other hand, if it is determined in step S212 that the smile level is not low and / or the average volume is not low, the process proceeds to step S214, where it is determined that the conversation is lively.

[0282] In this way, the facial expressions, volume of the voices, etc. of the passengers are observed, and it is determined from the observation results whether the passengers are in an excited state. The determination result is used in the determination process of step S159 (FIG. 26), as described above.

[0283] <Sound image localization processing> The processing relating to the localization of a sound image executed in step S15 (FIG. 8) and steps S18 to S19 (FIG. 8) will be described below. Fig. 29 is a diagram showing an example of the configuration of the sound image localization processing unit 81 (FIG. 4).

[0284] The sound image localization processing unit 81 includes a sound object information acquisition unit 301, a sound control unit 302, and a sound processing unit 303. The sound object here refers to a sound emitted by the occupant 32.

[0285] The sound object information acquisition unit 301 acquires information regarding the sound object acquired via the input / output processing unit 62. The information regarding the sound object is the sound data of the sound object and the position data of the sound object. The sound data of the sound object acquired by the sound object information acquisition unit 301 is supplied to the sound processing unit 303, and the position data of the sound object is supplied to the sound control unit 302.

[0286] The position data of the sound object supplied to the sound control unit 302 is data on the position (seat position) of the occupant 32-2' of the other vehicle 11-2 virtually seated within the host vehicle 11-1. Data regarding the voice emitted by the occupant 32-2' is supplied from the sound object information acquisition unit 301 to the sound processing unit 303 as the sound data of the sound object. Note that the sound data includes, in addition to the voice of the occupant, sounds generated in response to the actions of the occupant such as, for example, the operation sound of a smartphone and the sound of opening a can.

[0287] The sound control unit 302 determines the direction of the sound at the position of the occupant 32-1 within the vehicle 11-1 who presents the virtual voice of the occupant 32-2'. The direction of the sound is a direction based on the forward direction of the vehicle 11. The position data (relative position data and direction with respect to the occupant 32-1) of the sound object by the sound control unit 302 is supplied to the sound processing unit 303.

[0288] Based on the position data of the sound object from the sound control unit 302, the sound processing unit 303 generates the data of the sound output from the speaker 33 provided in the host vehicle 11-1 and outputs it to the speaker 33.

[0289] <Overview of VPT> In order to realize the output of the sound of the sound image (sound object) of the occupant 32-2 of the other vehicle 11-2 as described above using the speaker 33 provided in the host vehicle 11-1, virtual surround technology can be used. In this technology, VPT (a registered trademark of Sony Corporation) can be used as such virtual surround technology. When headphones are used, for example, VPT (Virtual Phones Technology) can localize sound outside the user's head rather than inside the head, creating a sound field that sounds as if the sound is being naturally reproduced from speakers placed in front or behind the user.

[0290] Specifically, VPT measures the head-related transfer function (HRTF) as the transfer characteristic from the sound source to both ears, and convolves the measured HRTF into the input signal to create a sound field similar to that of listening to 5.1ch or 7.1ch through speakers, even when played through headphones. The following describes the sound processing of this technology using VPT. However, VPT is assumed to be applied to 7ch surround (without using a 0.1ch subwoofer).

[0291] Note that the following description will be given by taking an example in which sound is output not from headphones but from speakers 33 provided in the vehicle 11. Since the arrangement and number of speakers 33 vary depending on the vehicle 11, as explained with reference to Fig. 3, the following description will explain the processing related to the localization of sound images by taking the arrangement and number of speakers 33 to which a general virtual surround technology is applied as an example.

[0292] <First Audio Path> Fig. 30 is a diagram showing an example of the internal configuration of the sound processing unit 303. As shown in Fig. 30, the sound processing unit 303 is provided with sound object processing units 321-1 to 321-n (n is an integer equal to or greater than 1), which perform sound processing on sound objects included in one or more sound objects. Specifically, the sound object processing unit 321-1 is made up of a VPT filter calculation processing unit 331-1 and a VPT processing unit 332-1.

[0293] Of the sound objects 1 to n, object direction information of the sound object 1 is input to the VPT filter calculation processing unit 331-1. The object direction information indicates the direction of the sound object when the occupant 32-1 of the host vehicle 11-1 is used as a reference.

[0294] The VPT filter calculation processor 331-1 calculates a VPT filter value based on the object direction information of the sound object 1 and supplies it to the VPT processor 332-1. For example, when using a 7-channel surround VPT, this VPT filter value is calculated to provide sound information to multiple channels out of the 7 channels by performing phantom processing or the like so that the sound object is positioned as close as possible to all of its positions. However, the VPT filter value may also reflect the coordinates of the display area A as necessary.

[0295] The VPT processing unit 332-1 updates the VPT filter value when it is supplied with a VPT filter value from the VPT filter calculation processing unit 331-1. For example, since object direction information is calculated 30 times per second, the VPT filter value is calculated and updated according to that interval. The VPT processing unit 332-1 also receives an audio stream of a sound object from the sound object information acquisition unit 301. The VPT processing unit 332-1 convolves an HRTF into the audio stream as an input signal in accordance with the VPT filter value. The audio stream after VPT processing is supplied to the mix unit 322. For example, when using 7-channel surround VPT, a 2-channel output can be obtained by multiplying the input signal by 7-channel HRTF coefficients and convolving them.

[0296] The sound object processing units 321-2 to 321-n are configured in the same manner as the sound object processing unit 321-1. That is, in the sound object processing units 321-2 to 321-n, VPT processing is performed according to the VPT filter value for each audio stream of a sound object included in each sound object. As a result, the audio streams after VPT processing of sound objects 2 to n are supplied to the mix unit 322. The mix unit 322 performs mixing processing on the audio streams 1 to n after VPT processing, and outputs them to the speaker 33. For example, the audio stream is made up of waveforms in wav file format, and the waveform of the audio signal after VPT processing is played back by the speaker 33.

[0297] <Sound processing at intermediate positions> FIG. 31 is a diagram for explaining sound processing at an intermediate position in the first audio path.

[0298] Specifically, when the object direction information indicates position Pa, the VPT filter calculation processor 331-1 determines to use the HRTF for SL-ch(a) of 7ch surround and updates the VPT filter value. However, silence is assigned to the channels corresponding to positions Pb to Pg. The VPT processor 332-1 convolves the HRTF for the left ear of SL-ch(a) with the audio signal according to the VPT filter value. Similarly, the HRTF for the right ear of SL-ch(a) is separately convolved with the audio signal. In this example of sound processing for an intermediate position, the sound object has only one direction and there is only one audio path, so the mix unit 322 outputs the audio signal after VPT processing by the VPT processor 332-1 to the speaker 33 of the vehicle 11. As a result, for example, the waveform of the audio signal after VPT processing is reproduced by the speaker 33 of the vehicle 11.

[0299] Next, when position Px, which is the intermediate position between positions Pa and Pg, is indicated, the VPT filter calculation processor 331-1 determines to use the 7ch surround HRTFs for SL-ch(a) and SL-ch(g), and updates the VPT filter values. However, silence is assigned to the channels corresponding to positions Pb to Pf. The VPT processor 332-1 convolves the left ear HRTF for SL-ch(a) into the waveform in wav file format input as the audio signal, according to the VPT filter values. Similarly, the right ear HRTF for SL-ch(a) is separately convolved into the audio signal. Furthermore, the VPT processor 332-1 convolves the left ear HRTF for SL-ch(g) into the waveform in wav file format input as the audio signal, according to the VPT filter values. Similarly, the right ear HRTF for SL-ch(g) is separately convolved into the audio signal.

[0300] However, the gains of both SL-ch(a) and SL-ch(g) are set to the same value because position Px is midway between positions Pa and Pg. For example, by multiplying the value of SL-ch(a) by 0.7 and the value of SL-ch(g) by 0.7, the output obtained by combining them can be made equivalent to the single output at position Pa. As a result, the speaker 33 of the vehicle 11 plays back the waveform of the audio signal after VPT processing using the HRTFs of SL-ch(a) and SL-ch(g), and the sound can be heard from position Px, which is midway between positions Pa and Pg.

[0301] Here, the example has been explained using the case where speakers 33 are arranged as shown in FIG. 31 and 7-channel surround is used, but the method explained here can also be applied to cases where speakers 33 are arranged for each seat 31 as shown in FIG. 3A, or where speakers 33 are arranged in the doors of the vehicle 11 as shown in FIG. 3B, thereby making it possible to localize the sound image of the occupants at any position.

[0302] <Second Audio Path> FIG. 32 is a diagram illustrating the second audio path of the sound object.

[0303] As shown in FIG. 32, the sound processing unit 303 includes a corresponding channel control unit 351, an HRTF processing unit 352, and a 2-channel conversion unit 353.

[0304] Information on sound objects 1 to n (n is an integer equal to or greater than 1) is input to the corresponding channel control unit 351. When information on multiple sound objects is input, the corresponding channel control unit 351 generates and arranges mixed sounds on predetermined channels in accordance with the information on each sound object. The corresponding channel control unit 351 also determines the HRTF to be used and supplies it to the HRTF processing unit 352.

[0305] The HRTF processing unit 352 convolves the HRTF into the waveform of the audio signal in accordance with the HRTF supplied from the corresponding channel control unit 351. The 2ch conversion unit 353 converts the audio signal from the HRTF processing unit 352 into 2ch in accordance with the number of final output channels, and outputs it to the speaker 33 of the vehicle 11.

[0306] Specifically, this will be described with reference to FIG. 33. The corresponding channel control unit 351 determines to use the HRTF for position Pa, i.e., SL-ch(a) of 7-channel surround, in accordance with the object direction information, and assigns silence to the channels corresponding to other positions Pb to Pg. However, in this example, since there is only one sound object (only one direction), no further sound objects are added. Next, the HRTF processing unit 352 convolves the HRTF for the left ear of SL-ch(a) with the waveform in wav file format input as the audio signal. Similarly, the HRTF for the right ear of SL-ch(a) is separately convolved with the audio signal. Then, the 2-channel conversion unit 353 converts the 7-channel audio signal into 2-channel audio in accordance with the number of final output channels, and outputs the converted audio signal to the speaker 33 of the vehicle 11. As a result, the waveform of the audio signal after VPT processing is reproduced by the speaker 33 of the vehicle 11.

[0307] Next, if the object direction information indicates position Pg, the corresponding channel control unit 351 determines to use the HRTF of SBL-ch(g) of 7ch surround. Subsequently, the HRTF processing unit 352 convolves the HRTF for the left ear of SBL-ch(g) with the waveform in wav file format input as the audio signal. Similarly, the HRTF for the right ear of SBL-ch(g) is separately convolved with the audio signal. Then, the 2ch conversion unit 353 converts the 7ch audio signal into 2ch, so that the waveform of the audio signal after VPT processing is reproduced by the speaker 33 of the vehicle 11.

[0308] Using this method, the sound image corresponding to the occupant 32-2 of the other vehicle 11-2 can be localized at a predetermined position in the own vehicle 11-1.

[0309] 3A, when a speaker 33 is arranged at each seat 31, for example, when the sound image of a virtually arranged passenger is localized at seat 31e, the two channels to be processed by the two-channel conversion unit 353 are speakers 33e-1 and 33e-2. Also, when a sound image is localized between seat 31b and seat 31e, the two channels to be processed by the two-channel conversion unit 353 are speakers 33e-1 and 33e-2.

[0310] For example, when a sound image of a virtually placed passenger is localized at seat 31d, the two channels to be processed by the two-channel conversion unit 353 are speakers 33c-2 and 33e-2. Similarly, when a sound image is localized between seats 31d and 31e, the two channels to be processed by the two-channel conversion unit 353 are speakers 33c-1 and 33e-2.

[0311] The speaker 33 that emits sound is selected according to the position of the seat, between rows of seats, between seats, etc. where the sound image is to be localized, and the sound image is localized at the position set by the method described above.

[0312] It should be noted that sound processing can be used to change the volume of sound depending on the position. Sound from a nearby seat is louder, and sound from a distant seat is quieter. This sound processing is just one example, and other sound processing can be applied as long as it allows the occupant 32-1 to preferably hear the sound from the sound image.

[0313] In the above-mentioned VPT 7-channel surround, a configuration has been described in which sound objects are localized horizontally, but by adopting an upward VPT configuration, for example by placing front high speakers (L(FH),R(FH)) directly above the front speakers (L,R), it is possible to adopt a configuration in which sound objects are localized not only horizontally but also vertically. Furthermore, by using the above-mentioned multi-channel configuration of 7 or more channels and adopting a VPT configuration for them, it is possible to configure sound objects in a more three-dimensional manner.

[0314] Here, an example using seat speakers 33 (2 channels) provided in each seat 31 (case A in FIG. 3) has been given. However, if there are no seat speakers 33, for example, as shown in case B in FIG. 3, the speakers 33 of the vehicle 11 may be used to target a specific occupant and perform the above-described processing depending on the number and location of the speakers. In this case, the specific occupant is usually the occupant in the driver's seat. However, it may also be an occupant in another seat, such as the passenger seat, other than the occupant in the driver's seat. However, the seat of the target occupant / target occupant must be set in advance.

[0315] <Other embodiments> Other embodiments to which the present technology is applied will now be described.

[0316] 34 is a diagram for explaining another embodiment (second embodiment) to which the present technology is applied. In the above-described embodiment (first embodiment), an example has been described in which an occupant 32-2 of another vehicle 11-2 rides in the host vehicle 11-1 as a virtual occupant 32-2'. The person riding in the host vehicle 11-1 as the virtual occupant 32' may be someone other than the occupant 32-2 of the other vehicle 11-2.

[0317] For example, a person 401 in a remote location may be allowed to ride in the vehicle 11-1 as a virtual passenger, in other words, the sound image of the person 401 in the remote location may be localized at a predetermined position in the vehicle 11-1. The remote location may be, for example, a home, a hospital, an office, or the like.

[0318] For example, by virtually having a person 401 in a hospital ride in the vehicle 11-1 and allowing them to converse with the occupant 32 of the vehicle 11-1, the occupant 32 and the person 401 can each feel as if they are driving together.

[0319] Person 401 can also act as a tour conductor and provide sightseeing guidance.

[0320] As shown in A of Figure 34, when occupants 32a-1, 32b-1, 32c-1, and 32e-1 are seated in seats 31a-1, 31b-1, 31c-1, and 31e-1 of the host vehicle 11-1, respectively, a person 401 in a remote location, for example, is virtually seated in the vacant seat 31d-1.

[0321] By localizing the sound image of virtual person 401' in seat 31d-1, passengers 32a-1, 32b-1, 32c-1, and 32e-1 can be given the sensation that person 401 is sitting in seat 31d-1 and speaking. When passengers 32a-1, 32b-1, 32c-1, and 32e-1 speak, the content of their speech can be supplied to person 401 as well, giving the sensation that passengers 32a-1, 32b-1, 32c-1, and 32e-1 and person 401 are driving together.

[0322] 34B, it is also possible to virtually have people 401 riding in a plurality of vehicles 11. Occupants 32a-1, 32b-1, 32c-1, and 32e-1 are seated in seats 31a-1, 31b-1, 31c-1, and 31e-1 of the host vehicle 11-1, respectively. Occupants 32a-2, 32c-2, and 32e-2 are seated in seats 31a-2, 31c-2, and 31e-2 of the other vehicle 11-2, respectively.

[0323] In the example shown in B of Fig. 34, a virtual person 401' is placed at the front of the vehicle interior in the host vehicle 11-1 and the other vehicle 11-2. The position of the virtual person 401' may be an empty seat in the vehicle interior, or may be a position where there is no seat.

[0324] For example, as shown in B of Fig. 34, when multiple vehicles 11 are traveling to the same destination, a virtual person 401 who will be providing sightseeing tours is placed in each of the multiple vehicles 11. A virtual person 401' is also placed at the front of the vehicle. By doing this, it is possible to create the feeling that one is actually traveling while being guided by a tour conductor 401.

[0325] In this way, the virtual passenger may be a person who is not riding in the vehicle 11, or the same person may be placed in multiple vehicles 11, and the placement position may be any position within the vehicle.

[0326] <Third embodiment> The third embodiment will be described with reference to Fig. 35 and Fig. 36. The third embodiment is an example of a case where matched vehicles 11 enjoy the same event. The case where the same event is karaoke will be described as an example.

[0327] The vehicle 11-1 and the other vehicle 11-2 are matched, and a playlist shared between the vehicle 11-1 and the other vehicle 11-2 is prepared. The shared playlist is configured so that the occupants of the vehicle 11-1 and the other vehicle 11-2 can add songs that they want to sing. Songs may be added to the playlist using the smartphone 451 of the occupant 32, or may be added using a car navigation system (not shown).

[0328] Where in the vehicle the sound image should be localized when enjoying karaoke together will be described with reference to Fig. 36. The locations of the occupants 32 and the virtual occupants 32' in the subject vehicle 11-1 and the other vehicle 11-2 shown in Fig. 36 are the same as those described as pattern 3 in Fig. 15.

[0329] The example shown in A of Fig. 36 is an example in which the singing voice is heard from the seat 31 corresponding to the singing occupant 32. For example, when an occupant 32e-1 of the other vehicle 11-2 is singing, control is performed so that the singing voice is heard from a virtual occupant 32e-1' seated in the seat 31h-1 of the host vehicle 11-1. In other words, control is performed so that the singing voice is heard from the position set to localize the sound image.

[0330] In the example shown in B of Fig. 36, the sound image of the singing occupant 32 is localized in the front of the vehicle interior. For example, when an occupant 32e-1 of the other vehicle 11-2 is singing, a virtual occupant 32e-1' is placed in the front of the vehicle interior of the subject vehicle 11-1, and control is performed so that the singing voice can be heard from that sound image. The front of the vehicle interior may be on the dashboard, and control may be performed so that the sound image is localized on the dashboard.

[0331] In the example shown in FIG. 36B, the occupant 32c-1 of the host vehicle 11-1 is also singing along. In such a case, a virtual occupant 32c-1' of the occupant 32c-1 may be placed on the dashboard inside the host vehicle 11-1, and the singing voice of the occupant 32c-1 may be heard from a sound image placed on the dashboard. In this case, the sound images localized on the dashboard are the sound images of the occupants 32c-1 and 32e-2, and the singing voices of these two people are output from the front of the vehicle. This gives the occupants 32 who are listening a stronger sense of singing together.

[0332] In this way, the sound image of the occupant 32-1 in the host vehicle 11-1 can be generated within the host vehicle 11-1.

[0333] As shown in B of Figure 36, by positioning the sound image of the singing passenger 32 at the front of the vehicle, the singer can be made to stand out and the listener can feel more like they are singing along.

[0334] In this way, the own vehicle 11-1 and the other vehicle 11-2 are matched, and the occupants 32 respectively riding in the own vehicle 11-1 and the other vehicle 11-2 can enjoy the same event together.

[0335] <Fourth embodiment> The fourth embodiment will be described with reference to Figures 37 and 38. As the fourth embodiment, a case where congestion information is shared will be described as an example.

[0336] 37 shows an example of a map showing the locations of the vehicles 11-1 and 11-2. The vehicle 11-2 is traveling on the route to the destination of the vehicle 11-1.

[0337] When matching vehicles 11, vehicles 11 stuck in the same traffic jam are matched using traffic jam information, the speed of the vehicles 11, information from the GPS (Global Positioning System), etc. The occupants 32 of the matched vehicles 11 can communicate with each other to obtain information about the traffic jam.

[0338] For example, a conversation as shown in Fig. 38 can be held, and information about traffic congestion can be obtained. Referring to Fig. 38, at time t1, the occupant 32-1 of the subject vehicle 11-1 asks the occupant 32-2 of the other vehicle 11-2 a question such as "Do you know why there is a traffic jam?" In response to this question, at time t2, the occupant 32-2 of the other vehicle 11-2 replies, "It looks like there has been an accident."

[0339] At time t3, the occupant 32-1 of the vehicle 11-1 asks the occupant 32-2 of the other vehicle 11-2, "Are you not making any progress at all?" In response to this question, the occupant 32-2 of the other vehicle 11-2 replies at time t4, "No, we're just stuck at a standstill."

[0340] The vehicle 11-1 and the other vehicle 11-2 that are stuck in the same traffic jam are matched, and the occupants are allowed to talk to each other, so that such conversations can take place. Therefore, the occupant 32-1 of the vehicle 11-1 can obtain more detailed information that he or she wants to know.

[0341] <Fifth embodiment> The fifth embodiment will be described with reference to Fig. 39. As the fifth embodiment, a case where information about food is shared will be described as an example.

[0342] When matching vehicles 11, vehicles 11 in which conversations within the vehicle indicate that someone wants to eat something are matched with vehicles 11 in which passengers are actually eating that food or have recently eaten it.

[0343] For example, as shown in the left diagram of Figure 39, suppose that a passenger 32-1 in vehicle 11-1 says, "I want to go get some ramen for lunch." From this conversation, the keyword "ramen" is extracted, and another vehicle 11-2 that matches this keyword is searched for. At this time, ramen shops near the location where vehicle 11-1 is traveling are also used as keywords, and by performing matching, it is possible to present information about shops that are easy for vehicle 11-1 to go to.

[0344] As a result of the matching, a vehicle 11-2 in which a person has eaten "ramen" or is eating "ramen" in the vehicle is searched for and matched. Note that instead of a vehicle, a person may be matched as in the second embodiment. Information about food can be obtained by the occupants 32 of the matched vehicles 11 talking to each other.

[0345] For example, a conversation like the one shown in Figure 39 can be held, and information about food can be obtained. Referring to Figure 39, at time t1, the occupant 32-1 of the subject vehicle 11-1 asks the occupant 32-2 of the other vehicle 11-2 a question such as "How long did you wait in line?" In response to this question, at time t2, the occupant 32-2 of the other vehicle 11-2 replies, "It was quite crowded, but the turnover was fast."

[0346] At time t3, the occupant 32-1 of the vehicle 11-1 asks the occupant 32-2 of the other vehicle 11-2, "Can children eat this too?" In response to this question, the occupant 32-2 of the other vehicle 11-2 replies, "We have a kids' menu," at time t4.

[0347] The vehicle 11-1 is matched with another vehicle 11-2 whose occupant is presumed to have the information desired by the occupant of the vehicle 11-1, and the occupants are placed in a state where they can converse with each other, so that such a conversation can take place. Thus, the occupant 32-1 of the vehicle 11-1 can obtain the information he or she wants to know in more detail.

[0348] Sixth Embodiment The sixth embodiment will be described with reference to Fig. 40. The sixth embodiment will be described taking as an example a case where previously acquired information is presented.

[0349] When matching vehicles 11, instead of matching with vehicles 11 that are traveling at the time of matching, it is also possible to match with vehicles 11 that have traveled in the same place in the past. When matching with vehicles 11 that have traveled in the same place in the past is performed, an information processing system 10 including a server 13 is applied as shown in B of FIG. 1, and the server 13 is configured to accumulate information on past vehicles 11.

[0350] Referring to the left diagram of Figure 40, another vehicle 11-2 that has previously traveled in the area where the host vehicle 11-1 is currently traveling is searched for by referring to the logs accumulated in the server 13. If the other vehicle 11-2 is found as a result of this search, it is matched with the host vehicle 11-1. After matching, the content of the conversation between the other vehicle 11-2 and the host vehicle 11-1 is provided to the host vehicle 11-1.

[0351] For example, as shown in the right diagram of Fig. 40, if a conversation such as "That mall under construction is apparently going to open next year," "The cherry blossoms are beautiful this time of year," and "The festival is coming up soon" is taking place in the other vehicle 11-2, this conversation is uttered from the sound image of the occupant 32-2 of the other vehicle 11-2 generated in the subject vehicle 11-1. The voice data of such conversation is extracted from the log stored in the server 13.

[0352] The positions where the occupants of the other vehicle 11-2 were seated may also be recorded in a log on the server 13, and the sound image may be localized within the vehicle 11-1 based on the log, or may be localized at the front of the vehicle interior as in the second embodiment.

[0353] For example, when the vehicle 11-1 is traveling in the area where the vehicle has moved after moving, the vehicle 11-1 is matched with a previous vehicle 11 and information is presented, thereby enabling the vehicle 11-1 to acquire information about the area.

[0354] When matching with past vehicles 11 is performed, it may be possible to set how far back in time the vehicles 11 are to be matched from the present time. For example, it is possible to match with vehicles 11 that have been traveling within the past week, or further, to match with vehicles 11 that have been traveling during the same time period. By doing so, for example, if it is lunchtime, there is a high possibility that a conversation about potential lunch locations will be presented, and potential lunch location locations can be obtained.

[0355] In this way, matching with past vehicles 11 may be performed and information may be presented.

[0356] Seventh Embodiment The seventh embodiment will be described with reference to Figures 41 and 42. The first to sixth embodiments have been described by taking as an example a case where a sound image is localized and audio information is presented from that sound image, but it is also possible to have an odor presented. The seventh embodiment will be described by taking as an example a case where an odor is also presented.

[0357] An example of smell presentation shown in Figure 41 will be described. In another vehicle 11-2, an occupant 32b-2 is eating something. The smell of the food that the occupant 32b-2 is eating is presented from the vicinity of the seat 31e-1 assigned to the occupant 32b-2 in the host vehicle 11-1. This enhances the presence of the virtually placed occupant 32b-2.

[0358] In this way, when an odor is also presented, a device for presenting the odor is also installed inside the vehicle. The presented odor does not have to be the same as the odor of the food eaten by the occupant 32b-2. If it is difficult to accurately detect the odor of the food eaten by the occupant 32b-2 and reproduce it in the host vehicle 11-1, a similar odor may be presented.

[0359] An example of the presentation of an odor shown in FIG. 42 will be described. When the other vehicle 11-2 approaches the seaside, the odor of the ocean (smell of the sea) is presented in the host vehicle 11-1. The olfactory information allows the occupants of the host vehicle 11-1 to get the feeling that they are traveling on the seaside together with the occupants of the other vehicle 11-2. This makes it possible to enhance the sense of sharing the entire space.

[0360] <Eighth embodiment> An eighth embodiment will be described with reference to Fig. 43. The first to sixth embodiments have been described by taking as an example a case where a sound image is localized and audio information is presented from the sound image, but it is also possible to have a video (image) presented as well. The eighth embodiment will be described by taking as an example a case where a video is also presented.

[0361] When audio is presented from the seat, an image of the occupant 32 of the other vehicle 11-2 is presented as an image reflected in the window or rearview mirror of the vehicle 11. In the example shown in A of Fig. 43, an image of the occupant 32-2' of the other vehicle 11-2 seated in the rear seat is presented in the rearview mirror 501 of the host vehicle 11-1.

[0362] The image to be presented may be an image of the occupant 32-2' actually captured inside the other vehicle 11-2, and the captured image may be presented in the own vehicle 11-1, or it may be an image of the occupant 32-2' that is provided by default.

[0363] As shown in FIG. 43B, an image of an occupant 32-2' of another vehicle 11-2 seated in the rear seat may be displayed on the rear seat window of the own vehicle 11-1.

[0364] As shown in C of Fig. 43, a facial image representing an occupant 32-2 of another vehicle 11-2 who is positioned in the host vehicle 11-1 may be displayed at the bottom of the windshield 502 of the host vehicle 11-1. In the example shown in C of Fig. 43, the faces of occupants 32a-2', 32b-2', 32c-2', and 32e-2' who are positioned as virtual occupants in the host vehicle 11-1 are displayed.

[0365] As shown in D of FIG. 43, an image captured from the ceiling of another vehicle 11-2 may be displayed on the ceiling of the own vehicle 11-1.

[0366] When configured to present an image, a device for projecting the image is provided in an appropriate position inside the vehicle. For example, when projecting an image on the ceiling, a display may be provided over the entire ceiling, or a projector may be provided in a position that allows the image to be projected onto the ceiling.

[0367] In this way, by presenting not only sound from a localized sound image but also video, it becomes possible to present not only auditory information but also visual information, thereby further enhancing the sense of presence of the occupant 32-2 of the other vehicle 11-2.

[0368] <Ninth embodiment> The ninth embodiment will be described with reference to Figures 44 and 45. Like the eighth embodiment, the ninth embodiment is an embodiment in which visual information such as video and images is presented to share visual information.

[0369] In the seventh embodiment, an example has been described in which an image of a virtually positioned occupant 32 is presented as an image. However, the presented image may be an image other than an image of a virtually positioned occupant 32.

[0370] For example, a scene seen from one vehicle 11 may be presented as an image on the other vehicle 11. For example, an image of the scene seen through the windshield of the other vehicle 11-2 is projected onto the ceiling of the host vehicle 11-1. When multiple vehicles 11 are matched, the driving scene of one of the multiple vehicles 11 is displayed on the other vehicle 11.

[0371] A driving scene of a virtual location may be displayed on all vehicles 11. In the example shown in Fig. 44, a driving scene of the same virtual location is displayed on the ceiling of the subject vehicle 11-1 and the ceiling of the other vehicle 11-2.

[0372] In this way, by presenting a common landscape, it gives the feeling that the participants are traveling in the same place, enhancing their mood. Also, by viewing the same video, they can find a common topic to talk about, encouraging communication.

[0373] When a scene seen from one vehicle 11 is to be presented as an image to another vehicle 11, a selection is made as to which vehicle 11 the scene to be presented from. For example, a vehicle 11 traveling on a route with no traffic jams or a vehicle 11 traveling on a route with many tourist attractions can be preferentially selected.

[0374] If all matched vehicles are congested and there are no tourist attractions along the entire route, a driving scene of a virtual location may be selected.

[0375] Instead of the above-described scenery, an image provided by a passenger 32 riding in the vehicle 11 may be shared. When an image is shared, a display may be displayed indicating that the image has been inserted from the direction of the seat where the passenger 32 who inserted the image is sitting.

[0376] As shown in A of Figure 45, when an image is inserted from an occupant 32, for example, sitting in the front seat diagonally to the left of an occupant 32 watching a video on a display 531 installed in front of the seat 31, the image is displayed as if it were inserted from the upper left diagonal side of the display 531 (a display with an effect applied), as shown in B of Figure 45.

[0377] The passenger 32 seated in the front seat may be the passenger 32 who is actually seated in the vehicle 11, or may be a virtual passenger 32' whose sound image is localized. This also applies hereinafter.

[0378] When an image is inserted from an occupant 32 sitting in the seat in front of seat 31, the image is displayed as if it were inserted from above on display 531, and when an image is inserted from an occupant 32 sitting in the front seat diagonally to the right, the image is displayed as if it were inserted from diagonally above and to the right on display 531.

[0379] When an image is inserted from an occupant 32 seated to the left of the seat 31, the image is displayed as if it were inserted from the left side of the display 531, and when an image is inserted from an occupant 32 seated to the right, the image is displayed as if it were inserted from the right side of the display 531.

[0380] When an image is inserted from an occupant 32 seated diagonally rearward to the left of the seat 31, the image is displayed as if it were inserted from the diagonally lower left of the display 531, and when an image is inserted from an occupant 32 seated diagonally rearward to the right, the image is displayed as if it were inserted from the diagonally lower right of the display 531. When an image is inserted from an occupant 32 seated at the rear of the seat 31, the image is displayed as if it were inserted from below the display 531.

[0381] Similarly, as shown in C of Figure 45, when an image is inserted for an occupant 32 who is looking at a display 532 (image projected from a projector) mounted on the ceiling, an effect is applied to indicate that the image has been inserted, and the image is displayed from the direction of the occupant 32 who has inserted the image.

[0382] In the case of a display 532 installed on the ceiling, the display is turned upside down, so that when an image is inserted from a passenger 32 seated in the front of the seat 31, for example, the image is displayed as if it were inserted from the bottom of the display 532. When an image is inserted from a passenger 32 seated in the back of the seat 31, for example, the image is displayed as if it were inserted from the top of the display 532.

[0383] In this way, the image is displayed taking into consideration the relative positional relationship between the occupant 32 who inserted the image and the occupant 32 who views the inserted image.

[0384] By displaying the image in this way, the direction from which the voice of the occupant who inserted the image is heard matches the direction from which the data appears, making it possible to share data more naturally.

[0385] <Tenth embodiment> The tenth embodiment will be described with reference to Fig. 46. The tenth embodiment is an embodiment in which, when vehicles are heading to the same destination via different routes, the travel routes are adjusted so that the arrival times are the same.

[0386] For example, when the subject vehicle 11-1 and the other vehicle 11-2 depart from different locations, head to the same destination, and meet at the destination, one of the subject vehicle 11-1 and the other vehicle 11-2 may arrive at the destination first. Even if the vehicles 11 are matched as described above and the driver is driving in a state where the driver feels as if he or she is driving the same vehicle 11, the moment one of the vehicles 11 arrives at the destination, the driving will end.

[0387] Therefore, in order to ensure that the vehicles arrive at the destination at the same time as much as possible, the planned travel route of the vehicle 11 that is likely to arrive at the destination earlier is changed and the vehicle is guided to a longer route, thereby lengthening the time until arrival. This will be described with reference to Fig. 46.

[0388] If the time it will take for the subject vehicle 11-1 to reach the destination is estimated to be 40 minutes and for the other vehicle 11-2 it is estimated to be 20 minutes, then unless the route is changed, the other vehicle 11-2 will arrive at the destination about 20 minutes before the subject vehicle 11-1. In this situation, the planned route of the other vehicle 11-2 is changed, and a route to the destination that is a longer route but takes about 40 minutes is searched for. The search results are presented to the driver of the other vehicle 11-2 via a car navigation system or the like of the other vehicle 11-2.

[0389] In this way, by adjusting the route, the driving time between the matched vehicles 11 can be prevented from being shortened, making the drive more enjoyable.

[0390] The first to tenth embodiments can also be implemented in combination.

[0391] According to this technology, comfortable communication can be achieved between passengers in vehicles 11 that are separated from each other.

[0392] By grouping the passengers in vehicle 11 according to their age, gender, etc., it becomes easier to find common topics and start a conversation smoothly.

[0393] The group can be changed depending on the state and interests of the passengers, and changing the group can also stimulate conversation.

[0394] By matching vehicles 11 so that occupants who are likely to have common topics of conversation can be matched, occupants can enjoy conversation even if they are meeting for the first time.

[0395] By matching vehicles 11 based on speed, what the occupants are holding, what is inside the vehicle, etc., occupants can exchange information they desire, such as traffic congestion information and reviews, with each other.

[0396] It will also be possible to share images and smells between vehicles 11, enhancing the sense of presence of each other's occupants.

[0397] <Regarding recording media> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, for example, that can execute various functions by installing various programs.

[0398] 47 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes using a program. In the computer, a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004. An input / output interface 1005 is further connected to the bus 1004. An input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.

[0399] The input unit 1006 includes a keyboard, a mouse, a microphone, etc. The output unit 1007 includes a display, a speaker, etc. The storage unit 1008 includes a hard disk, a non-volatile memory, etc. The communication unit 1009 includes a network interface, etc. The drive 1010 drives removable media 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0400] In a computer configured as described above, the CPU 1001 performs the above-described series of processes by, for example, loading a program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it.

[0401] The program executed by the computer (CPU 1001) can be provided by being recorded on removable media 1011 such as package media, for example. The program can also be provided via wired or wireless transmission media such as a local area network, the Internet, or digital satellite broadcasting.

[0402] In a computer, a program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable medium 1011 into the drive 1010. The program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Alternatively, the program can be installed in the ROM 1002 or the storage unit 1008 in advance.

[0403] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0404] In this specification, a system refers to an entire device made up of multiple devices.

[0405] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0406] It should be noted that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the present technology.

[0407] The present technology can also be configured as follows. (1) an acquisition unit that acquires the status of the host vehicle and the status of other vehicles; a setting unit that sets a position in the host vehicle where a sound image of an occupant in the other vehicle is to be localized based on a situation of the host vehicle and a situation of the other vehicle; a sound image localization processing unit that localizes a sound image of the occupant at the position set by the setting unit; An information processing device comprising: (2) The acquisition unit acquires at least the seat position where the occupant is seated, The setting unit sets an empty seat of the host vehicle as a position where a sound image of an occupant of the other vehicle is to be localized. The information processing device according to (1) above. (3) the setting unit groups the occupants of the host vehicle and the occupants of the other vehicle based on any one of age, role, and gender; The positions are set so that the occupants belonging to the same group are located close to each other. The information processing device according to (1) or (2). (4) The setting unit sets the position of the sound image to a position where the relative positional relationship between the occupants of the other vehicle is reproduced in the own vehicle. The information processing device according to any one of (1) to (3). (5) the setting unit determines whether or not to reset the set position; The determination is made using the status of the occupants of the vehicle. The information processing device according to any one of (1) to (4). (6) The setting unit measures the degree of excitement of conversation between occupants in the vehicle, and resets the position when the degree of excitement decreases. The information processing device according to any one of (1) to (5). (7) The setting unit resets the position when an occupant included in a predetermined group reacts to a conversation between occupants included in another group. The information processing device according to (3) above. (8) a matching unit that matches the host vehicle with the other vehicle; The matching unit matches vehicles that satisfy predetermined conditions. The information processing device according to any one of (1) to (7). (9) The matching unit matches a vehicle registered as a matching vehicle with the vehicle, and if there is no registered vehicle, matches a vehicle that is close in distance. The information processing device according to (8). (10) The matching unit matches vehicles that satisfy any one of the following conditions: a vehicle with a similar itinerary, a vehicle with a similar crew member configuration, a vehicle with a similar vehicle situation, a vehicle with a similar conversation content between the crew, and a vehicle with a similar in-car situation. The information processing device according to (8) or (9). (11) The matching unit matches a vehicle with a high degree of match with an object held by an occupant. The information processing device according to any one of (8) to (10). (12) The matching unit matches vehicles traveling on a route along which the host vehicle is scheduled to travel. The information processing device according to any one of (8) to (11). (13) The matching unit detects keywords included in conversations of occupants of the vehicle, and matches the keywords with vehicles in which occupants who are estimated to have topics related to the keywords are riding. The information processing device according to any one of (8) to (12). (14) The setting unit sets a position for localizing a sound image of a person in a remote location. The information processing device according to any one of (1) to (13). (15) The setting unit sets a sound image of the singing passenger at a position in front of the vehicle. The information processing device according to any one of (1) to (14). (16) An image of the occupant of the other vehicle, which is presented to the occupant in the own vehicle as the sound image, is displayed at a predetermined position in the own vehicle. The information processing device according to any one of (1) to (15). (17) A view of the location where the other vehicle is traveling is displayed at a predetermined position within the vehicle. The information processing device according to any one of (1) to (16). (18) The conversations emitted from the sound images of the passengers are conversations extracted from logs stored on a server. The information processing device according to any one of (1) to (17). (19) The information processing device Obtain the status of your vehicle and the status of other vehicles, setting a position in the host vehicle where a sound image of a passenger in the other vehicle is to be localized based on a situation of the host vehicle and a situation of the other vehicle; The sound image of the passenger is localized at the set position. Information processing methods. (20) On the computer, Obtain the status of your vehicle and the status of other vehicles, setting a position in the host vehicle where a sound image of a passenger in the other vehicle is to be localized based on a situation of the host vehicle and a situation of the other vehicle; The sound image of the passenger is localized at the set position. A program that executes a process that includes steps. [Explanation of symbols]

[0408] 10 Information processing system, 11 Vehicle, 12 Network, 13 Server, 31 Seat, 32 Occupant, 33 Speaker, 36 Output unit, 51 Information processing device, 54 Step, 55 Step, 61 Input unit, 62 Input / output processing unit, 63 Output unit, 64 Information processing unit, 71 Seat allocation processing unit, 72 Vehicle matching processing unit, 73 Seat allocation information storage unit, 74 Vehicle matching information storage unit, 75 Vehicle information storage unit, 76 Occupant information storage unit, 77 Sensor information processing unit, 78 Communication unit, 79 Output information creation unit, 81 Sound image localization processing unit, 91 Pre-registered parameters, 92 Departure registration parameters, 101 Area, 102 Area, 201 Camera, 202 Microphone, 211 Excitement detection unit, 221 Face detection dictionary storage unit, 222 facial expression dictionary storage unit, 223 face detection unit, 224 facial expression quantification unit, 225 speech recognition dictionary storage unit, 226 volume detection unit, 227 speech-to-text unit, 228 detection information storage unit, 301 sound object information acquisition unit, 302 sound control unit, 303 sound processing unit, 321 sound object processing unit, 322 mix unit, 331: VPT filter calculation unit, 332: VPT processing unit, 351 corresponding channel control unit, 352 HRTF processing unit, 353 2ch unit, 401 person, 451 smartphone, 501 rearview mirror, 502 windshield, 531, 532 display

Claims

1. an acquisition unit that acquires the status of the host vehicle and the status of other vehicles; a setting unit that sets a position in the host vehicle where a sound image of an occupant in the other vehicle is to be localized based on a situation of the host vehicle and a situation of the other vehicle; a sound image localization processing unit that localizes a sound image of the occupant at the position set by the setting unit; Equipped with The setting unit groups the occupants in the host vehicle and the occupants in the other vehicle, and sets the positions so that the sound image positions of the occupants included in the same group are located close to each other. Information processing device.

2. The acquisition unit acquires at least the seat position where the occupant is seated, The setting unit sets an empty seat of the host vehicle as a position where a sound image of an occupant of the other vehicle is to be localized. The information processing device according to claim 1 .

3. The setting unit groups the occupants of the vehicle and the occupants of the other vehicle based on any one of age, role, and sex. The information processing device according to claim 1 .

4. When a first occupant and a second occupant are included in the same group, the setting unit sets the position so that the sound image position of the second occupant relative to the sound image position of the first occupant included in the same group is located closer to the sound image position of the first occupant than the sound image of an occupant included in another group. The information processing device according to claim 1 .

5. the setting unit determines whether or not to reset the set position; The determination is made using the status of the occupants of the vehicle. The information processing device according to claim 1 .

6. The setting unit measures the degree of excitement of conversation between occupants in the vehicle, and resets the position when the degree of excitement decreases. The information processing device according to claim 1 .

7. The setting unit resets the position when an occupant included in a predetermined group reacts to a conversation between occupants included in another group. The information processing device according to claim 3 .

8. a matching unit that matches the host vehicle with the other vehicle; The matching unit matches vehicles that satisfy predetermined conditions. The information processing device according to claim 1 .

9. The matching unit matches a vehicle registered as a matching vehicle with the vehicle, and if there is no registered vehicle, matches a vehicle that is close in distance. The information processing device according to claim 8 .

10. The matching unit matches vehicles that satisfy any one of the following conditions: vehicles with similar itineraries, vehicles with similar crew member configurations, vehicles with similar vehicle conditions, vehicles with similar conversations between crew members, and vehicles with similar interior conditions. The information processing device according to claim 8 .

11. The matching unit matches a vehicle with a high degree of match with an object held by an occupant. The information processing device according to claim 8 .

12. The matching unit matches vehicles traveling on a route along which the host vehicle is scheduled to travel. The information processing device according to claim 8 .

13. The matching unit detects keywords included in conversations of occupants of the vehicle, and matches the keywords with vehicles in which occupants who are estimated to have topics related to the keywords are riding. The information processing device according to claim 8 .

14. The setting unit sets a position for localizing a sound image of a person in a remote location. The information processing device according to claim 1 .

15. The setting unit sets a sound image of the singing passenger at a position in front of the vehicle. The information processing device according to claim 1 .

16. An image of the occupant of the other vehicle, which is presented to the occupant in the own vehicle as the sound image, is displayed at a predetermined position in the own vehicle. The information processing device according to claim 1 .

17. A view of the location where the other vehicle is traveling is displayed at a predetermined position within the vehicle. The information processing device according to claim 1 .

18. The conversations emitted from the sound images of the passengers are conversations extracted from logs stored on a server. The information processing device according to claim 1 .

19. The information processing device Obtain the status of your vehicle and the status of other vehicles, setting a position in the host vehicle where a sound image of a passenger in the other vehicle is to be localized based on a situation of the host vehicle and a situation of the other vehicle; The sound image of the occupant is localized at the set position, The setting is performed by grouping the occupants in the own vehicle and the occupants in the other vehicle, and setting the positions so that the sound image positions of the occupants included in the same group are located close to each other. Information processing methods.

20. On the computer, Obtain the status of your vehicle and the status of other vehicles, setting a position in the host vehicle where a sound image of a passenger in the other vehicle is to be localized based on a situation of the host vehicle and a situation of the other vehicle; The sound image of the occupant is localized at the set position, The setting is performed by grouping the occupants in the own vehicle and the occupants in the other vehicle, and setting the positions so that the sound image positions of the occupants included in the same group are located close to each other. A program that executes a process that includes steps.

Images