Information providing program and information providing method

Abstract

To provide an information providing program and an information providing method for providing more appropriate information to a user.SOLUTION: An information providing program causes a computer to: acquire user information including emotion information of a user of a vehicle, location information of the vehicle, and to-be-provided information including information on a recommendation spot to be provided to the user; determine a search area of the recommendation spot based on the location information of the vehicle and the emotion information; search for the recommendation spot based on the determined search area; and provide, to the user, information on the recommendation spot extracted by the search.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information providing program and an information providing method. [Background technology]

[0002] Conventionally, there are known technologies that generate stop-off point information based on at least one of driving environment information and vehicle occupant information, transmit the generated stop-off point information to a terminal device traveling with the vehicle, or recommend facilities frequented by locals as popular facilities to tourists based on the driving information (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-134193 [Patent Document 2] International Publication No. 2019 / 130752 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technologies have had the problem that, depending on the situation, if the vehicle user's condition is not fully understood, appropriate information may not be provided to the user.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide an information providing program and an information providing method that can provide more appropriate information to a user. [Means for solving the problem]

[0006] The information providing program and the information providing method according to the present invention employ the following configuration. (1): An information provision program according to one embodiment of the present invention is an information provision program that causes a computer to acquire user information including emotional information of a vehicle user, location information of the vehicle, and available information including information about recommended locations that can be provided to the user, determine a search area for the recommended locations based on the location information of the vehicle and the emotional information, search for the recommended locations based on the determined search area, and provide information about the recommended locations extracted by the search to the user.

[0007] (2): In the above aspect (1), when the degree of the user's emotion based on the user's emotion information is equal to or greater than a threshold, the recommended location is searched for in the search area based on the vehicle's location information, excluding the area opposite to the vehicle's direction of travel.

[0008] (3) In the above aspect (1), the search conditions for the recommended locations include preference information of the user.

[0009] (4) In the above aspect (3), if the number of recommended locations extracted by the search is less than a predetermined number, the recommended locations are searched again without including the user's preference information in the search conditions.

[0010] (5): In the above aspect (3), the emotional information includes information regarding the user's level of impatience, and when the user's level of impatience is equal to or greater than a threshold, the recommended locations are searched for without including the user's preference information in the search conditions.

[0011] (6): In the above aspect (1), the emotional information includes the user's level of impatience, and the level of impatience is estimated from at least one of the user's facial expression, the user's tone of voice, the user's speaking speed, and the words uttered by the user.

[0012] (7): In the above aspect (1), the emotional information includes the user's level of impatience, and the level of impatience is estimated based on information obtained from at least one of a camera and a microphone mounted on the vehicle and a camera and a microphone mounted on a terminal device carried by the user.

[0013] (8): An information provision method according to one aspect of the present invention is an information provision method in which a computer acquires user information including emotional information of a vehicle user, location information of the vehicle, and available information including information about recommended locations that can be provided to the user, determines a search area for the recommended locations based on the location information of the vehicle and the emotional information, searches for the recommended locations based on the determined search area, and provides the user with information about the recommended locations extracted by the search. [Effects of the Invention]

[0014] According to the above aspects (1) to (8), more appropriate information can be provided to the user. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a configuration diagram showing an example of an information providing system 1 according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of the contents of user information 332. [Figure 3] FIG. 10 is a diagram showing an example of the contents of vehicle information 334. [Figure 4] FIG. 10 is a diagram showing an example of the contents of a recommended spot DB 336. [Figure 5] FIG. 10 is a sequence diagram showing the flow of processing in the information providing system according to the first providing pattern. [Figure 6] 10 is a flowchart illustrating an example of a satisfaction degree estimation process and a re-search process. [Figure 7] FIG. 10 is a sequence diagram showing the flow of processing in the information providing system according to the second providing pattern. [Figure 8] 10 is a flowchart showing a first embodiment of a search area adjustment process. [Figure 9] 10 is a flowchart showing a second embodiment of the search area adjustment process. [Figure 10] FIG. 10 is a sequence diagram showing the flow of processing in the information providing system according to the third providing pattern. [Figure 11] 10 is a flowchart showing an example of a search process in a third provision pattern. [Figure 12] 10 is a flowchart illustrating an example of a recommended spot search process based on free time. [Figure 13] FIG. 10 is a configuration diagram of an in-vehicle device 100A according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of an information providing program and an information providing method of the present invention will be described with reference to the drawings. Note that the following description will mainly focus on a system or device to which the information providing program is applied.

[0017] [First embodiment] [System Configuration] FIG. 1 is a configuration diagram showing an example of an information provision system 1 according to a first embodiment. For example, the information provision system 1 suggests recommended points such as nearby stores, facilities, and tourist attractions to a user traveling in a vehicle, depending on the user's condition according to the vehicle's status. The recommended points may be, for example, points that the user stops at on the way to the destination by vehicle, or may be points that become the user's new destination. Examples of stores include restaurants and convenience stores, and examples of facilities include, but are not limited to, theme parks, shopping malls, public restrooms, and parks.

[0018] The information providing system 1 shown in FIG. 1 includes, for example, an in-vehicle device 100, a terminal device 200, and a server device 300. In the first embodiment, the server device 300 is an example of an "information providing device." The in-vehicle device 100, the terminal device 200, and the server device 300 can communicate with each other via a network NW. The network NW includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a public line, a telephone line, a provider device, a dedicated line, a wireless base station, and the like. For convenience of explanation, only one in-vehicle device 100 and one terminal device are shown in FIG. 1, but the information providing system 1 may include multiple in-vehicle devices 100 and multiple terminal devices, or may not include the terminal device 200.

[0019] The in-vehicle device 100 is mounted, for example, on a vehicle used by a user, and includes a telematics control unit (TCU) that communicates with other devices, systems, etc. to exchange information with the other devices and systems. The vehicle may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its driving source may be, for example, an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or power discharged from a secondary battery or a fuel cell. The in-vehicle device 100 transmits information obtained from the vehicle and information received from the terminal device 200 to the server device 300, and outputs information received from the terminal device 200 or the server device 300 to provide the information to the user.

[0020] The terminal device 200 is, for example, a portable terminal such as a smartphone or tablet terminal carried by a vehicle user, and may be used inside or outside the vehicle. A mechanism (for example, a grip member) to which the terminal device 200 can be detachably attached is provided inside the vehicle, and the terminal device 200 can be installed in a position where the user can view images and the like displayed on the terminal device 200 while driving the vehicle. The terminal device 200 transmits information transmitted from the in-vehicle device 100 and information acquired by the terminal device 200 to the server device 300, outputs information received from the server device 300 to provide it to the user, and transmits it to the in-vehicle device 100.

[0021] The server device 300 may be, for example, a server device or a PC (Personal Computer), or may be a cloud server configured by cloud computing including one or more information processing devices. The server device 300 acquires information such as location information and user information transmitted from the in-vehicle device 100 and the terminal device 200, searches for recommended points according to the user's status based on the acquired information, and provides the user with information about the searched recommended points. The server device 300 may also provide a route to the recommended point determined by the user's instruction. The functional configurations of the in-vehicle device 100, the terminal device 200, and the server device 300 will be specifically described below.

[0022] [In-vehicle device 100] The in-vehicle device 100 includes, for example, a vehicle-side communication unit 110, a vehicle-side short-range communication unit 120, a camera 130, a microphone 140, an output unit 150, a vehicle sensor 160, a navigation function unit 170, a vehicle-side control unit 180, and a vehicle-side storage unit 190. The navigation function unit 170 and the vehicle-side control unit 180 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD, CD-ROM, or memory card, and installed by inserting the storage medium into a drive device. Camera 130 is an example of an "imaging unit," and microphone 140 is an example of an "audio input unit."

[0023] The vehicle-side storage unit 190 may be realized by the various storage devices described above, or a solid-state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), or a random-access memory (RAM). The vehicle-side storage unit 190 stores, for example, map information 192, programs, and various other information. The map information 192 includes, for example, information representing road shapes using links indicating roads and nodes connected by the links. The map information 192 may also include roads associated with location information (latitude and longitude), road curvatures and gradients, the number of lanes, road widths, traffic rules (e.g., speed limits), and point-of-interest (POI) information. The POI information may include information such as overviews of tourist spots, facilities, and stores. The map information 192 may be updated as needed by the vehicle-side communication unit 110 communicating with an external device via the network NW.

[0024] The vehicle-side communication unit 110 is, for example, a wireless communication module for connecting to a network NW and communicating with the server device 300 via the network NW. The vehicle-side communication unit 110 communicates with, for example, a server-side communication unit 310 of the server device 300. The vehicle-side communication unit 110 may also communicate with a terminal-side communication unit 210 of the terminal device 200, etc. The vehicle-side communication unit 110 performs wireless communication based on Wi-Fi (registered trademark), DSRC (Dedicated Short Range Communications), Bluetooth (registered trademark), or other communication standards.

[0025] The vehicle-side short-range communication unit 120 is a wireless communication module for performing short-range wireless communication with another device (e.g., terminal device 200) that is located in a short distance (within a predetermined distance). The vehicle-side short-range communication unit 120 performs wireless communication based on Wi-Fi, Bluetooth, or other communication standards. Information transmitted by the vehicle-side communication unit 110 and the vehicle-side short-range communication unit 120 may include identification information for identifying the vehicle (e.g., a vehicle ID), identification information for identifying the user (e.g., a user ID), time information, etc.

[0026] The camera 130 is a digital camera using a solid-state imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 130 is attached to any location on the vehicle. When capturing an image of the area in front of the vehicle, the camera 130 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 130 periodically and repeatedly captures images of the area around the vehicle. The camera 130 may also be attached to any location on the vehicle in a position and orientation that allows it to capture an image of the head of an occupant (user) in the vehicle cabin from the front (in an orientation that captures the face). In this case, the camera 130 is attached to, for example, the top of a display device provided in the center of the vehicle's instrument panel, and periodically and repeatedly captures images of the occupant. The camera 130 may be a stereo camera. The microphone 140, for example, captures the voices of the occupants in the vehicle cabin.

[0027] The output unit 150 includes, for example, a display unit 152 and a speaker 154. The speaker 154 is an example of an "audio output unit." The display unit 152 is, for example, a liquid crystal display disposed in the center of the instrument panel, but may be a display other than a liquid crystal display. The display unit 152 may be a meter display provided in a portion of the instrument panel facing the driver (user), a center display provided in the center of the instrument panel, a HUD (Head Up Display), or the like. The display unit 152 displays an image in response to control by the vehicle-side control unit 180. The display unit 152 may also display information from the navigation function unit 170. The display unit 152 may also be a touch panel that also functions as an input unit that accepts user input operations. The speaker 154 outputs, for example, voice or alarm corresponding to an image displayed on the display unit 152, or other sound information, in response to control by the vehicle-side control unit 180.

[0028] The vehicle sensor 160 includes a vehicle speed sensor that detects the speed of the vehicle, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the direction of the vehicle. The vehicle sensor 160 may also include a position sensor that acquires the position of the vehicle. The position sensor is, for example, a sensor that acquires position information (e.g., latitude and longitude) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver of the navigation function unit 170. The vehicle sensor 160 may also include a perspiration sensor that measures the amount of perspiration from the hands of the vehicle user (driver). The perspiration sensor is, for example, installed on a driving operator such as a steering wheel through which the user steers the vehicle.

[0029] The navigation function unit 170 includes, for example, a GNSS receiver, and identifies the vehicle position (vehicle location) based on signals received from GNSS satellites. The vehicle location may be identified or supplemented by an INS (Inertial Navigation System) that uses outputs from vehicle sensors 160 such as a vehicle speed sensor and an acceleration sensor. The navigation function unit 170 may output the identified vehicle location to the location acquisition unit 181.

[0030] Furthermore, the navigation function unit 170 acquires destination information input by the user via the microphone 140, the display unit 152, or the like, and generates a route from the vehicle position to the destination by using the acquired destination and vehicle position and referring to map information 192 previously stored in the vehicle-side storage unit 190. The navigation function unit 170 then causes the output unit 150 to output the route as an image or sound, and provides route guidance that guides the vehicle toward the destination through the user's driving operation. Furthermore, when one recommended point is determined from one or more recommended points provided by the server device 300, the navigation function unit 170 may generate a route to the recommended point by referring to the map information 192, and output the generated route as an image or sound to the output unit 150 to provide route guidance to the user.

[0031] The vehicle-side control unit 180 controls the overall components of the in-vehicle device 100. The vehicle-side control unit 180 includes, for example, a position acquisition unit 181, a communication control unit 182, and a display control unit 183. The position acquisition unit 181 acquires the vehicle position from the vehicle sensor 160 or the navigation function unit 170. The position acquisition unit 181 may also acquire, as the vehicle position, position information (position information of the terminal device 200) from the terminal device 200 communicating via the vehicle-side short-range communication unit 120.

[0032] The communication control unit 182 controls, for example, communication (transmission and reception) with the communication destination in the vehicle-side communication unit 110 or the vehicle-side short-range communication unit 120. For example, the communication control unit 182 transmits various pieces of information, such as location information acquired by the location acquisition unit 181, image information (camera image) captured by the camera 130, and sound information acquired by the microphone 140, from the vehicle-side communication unit 110 to the server device 300. The sound information may include, for example, request information (search request) for a recommended location by a user (occupant). The communication control unit 182 may also transmit information on speed and acceleration detected by the vehicle sensor 160 to the server device 300. The communication control unit 182 may also transmit the various pieces of information described above to the terminal device 200 via the vehicle-side short-range communication unit 120, and cause the terminal device 200 to transmit the information to the server device 300 via the network NW. The communication control unit 182 also performs control according to the information received by the vehicle-side communication unit 110 or the vehicle-side short-range communication unit 120.

[0033] The display control unit 183 also causes the output unit 150 to output recommended location information including one or more recommended locations acquired from the server device 300. The display control unit 183 also causes the output unit 150 to output various information transmitted from the terminal device 200. The display control unit 183 may also cause the display unit 152 to display information input by the user. The display control unit 183 may also generate sound information and output it to the speaker 154 in addition to (or instead of) displaying an image.

[0034] [Terminal device 200] The terminal device 200 includes, for example, a terminal-side communication unit 210, a terminal-side short-range communication unit 220, a camera 230, a microphone 240, an output unit 250, a terminal-side control unit 260, an application execution unit 270, and a terminal-side storage unit 280. The terminal-side control unit 260 and the application execution unit 270 are realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, GPU, or SOC, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD, CD-ROM, or memory card, and installed by inserting the storage medium into a drive device.

[0035] The terminal-side storage unit 280 may be realized by the various storage devices described above, or an SSD, an EEPROM, a ROM, a RAM, or the like. The terminal-side storage unit 280 stores, for example, an information provision application 282, programs, and various other information. The information provision application 282 will be described later.

[0036] The terminal-side communication unit 210 is, for example, a wireless communication module for connecting to a network NW and communicating with the server device 300 via the network NW. The terminal-side communication unit 210 communicates with the server-side communication unit 310 of the server device 300. The terminal-side communication unit 210 may also communicate with the vehicle-side short-range communication unit 120 of the in-vehicle device 100, etc. The terminal-side communication unit 210 performs wireless communication based on a cellular network, Wi-Fi, Bluetooth, or other communication standard.

[0037] The terminal-side short-range communication unit 220 is a wireless communication module for performing short-range wireless communication with another device (e.g., the in-vehicle device 100) that is located in a short distance. The terminal-side short-range communication unit 220 performs wireless communication based on Wi-Fi, Bluetooth, or other communication standards. Information transmitted by the terminal-side communication unit 210 and the terminal-side short-range communication unit 220 may include identification information (e.g., a terminal ID) for identifying the terminal device 200, a user ID, time information, etc.

[0038] The camera 230 is, for example, a digital camera using a solid-state imaging element such as a CCD or CMOS. For example, when the terminal device 200 is mounted in a position and orientation that allows the camera 230 to capture an image of the head of an occupant (user) in the vehicle cabin from the front (in an orientation that captures the face), the camera 230 periodically and repeatedly captures images of the occupant. The microphone 240, for example, acquires the voice of the occupant in the vehicle cabin.

[0039] The output unit 250 includes, for example, a display unit 252 and a speaker 254. The display unit 252 is, for example, a liquid crystal display, but may be a display other than a liquid crystal display. The display unit 252 displays an image in accordance with the control of the information provision application 282 executed by the terminal-side control unit 260 and the application execution unit 270. The display unit 252 may also be a touch panel that also functions as an input unit that accepts user input operations. The speaker 254 outputs sounds, alarms, etc. corresponding to images displayed on the display unit 252, or other sound information, in accordance with the control of the information provision application 282 executed by the terminal-side control unit 260 and the application execution unit 270.

[0040] The terminal-side control unit 260 controls the overall components of the terminal device 200. The terminal-side control unit 260 includes, for example, a position acquisition unit 262. The position acquisition unit 262 acquires position information (e.g., latitude and longitude) of the terminal device 200 from, for example, a GPS device. The position acquisition unit 262 may also acquire, as the position information of the terminal device 200, the vehicle position acquired from the in-vehicle device 100 via the terminal-side short-range communication unit 220.

[0041] The application execution unit 270 is realized by executing an information provision application 282 stored in the terminal-side storage unit 280. The information provision application 282 is, for example, downloaded from an external device via the network NW and installed in the terminal device 200. The information provision application 282 is an application program that acquires recommended location information including one or more recommended locations for the user from the server device 300, and controls the output unit 250 to output the acquired information, or transmits the information to the in-vehicle device 100 to output it from the in-vehicle device 100.

[0042] For example, the information provision application 282 acquires information input by the user via the touch panel of the display unit 252 or the microphone 240, and transmits the acquired information, location information, etc. to the server device 300 via the terminal-side communication unit 210. The information provision application 282 also acquires recommended spot information from the server device 300, generates image information and audio information corresponding to the acquired information, and outputs the image information and audio information to the output unit 250. The information provision application 282 also transmits information acquired from the in-vehicle device 100 via the terminal-side short-range communication unit 220 to the server device 300 via the terminal-side communication unit 210. The information provision application 282 may also transmit information acquired from the server device 300 via the terminal-side communication unit 210 to the in-vehicle device 100 via the terminal-side short-range communication unit 220, or may transmit images and audio generated by the information provision application 282 to the in-vehicle device 100.

[0043] [Server device 300] The server device 300 includes, for example, a server-side communication unit 310, a server-side control unit 320, and a server-side storage unit 330. The server-side control unit 320 is realized, for example, by a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as an LSI, ASIC, FPGA, GPU, or SOC, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device having a non-transitory storage medium), or may be stored in a removable storage medium (non-transitory storage medium) such as a DVD, CD-ROM, or memory card, and installed by inserting the storage medium into a drive device.

[0044] The server-side storage unit 330 may be realized by the various storage devices mentioned above, or an SSD, an EEPROM, a ROM, or a RAM, etc. The server-side storage unit 330 stores, for example, user information 332, vehicle information 334, a recommended spot DB (Database) 336, map information 338, programs, and various other information. At least one of the information stored in the server-side storage unit 330 may be stored in an external device (for example, a DB server) that can communicate with the server device 300. The recommended spot DB 336 and the map information 338 are examples of "providable information" that includes information about recommended spots that can be provided to the user.

[0045] 2 is a diagram showing an example of the contents of the user information 332. The user information 332 is information in which, for example, a user ID is associated with personal information, terminal information, vehicle information, basic state information, past state information, and schedule information. The personal information includes, for example, the user's age, sex, address, place of employment, and preference information (e.g., hobbies, favorite cuisine, favorite genre, etc.). The terminal information includes, for example, a terminal ID and address information for the server device 300 to communicate with the terminal device 200. The vehicle information includes, for example, a vehicle ID and address information for the server device 300 to communicate with the in-vehicle device 100.

[0046] The basic state information is, for example, information about the user's state when they are calm (normal) (in other words, when their mood is stable). The basic state information includes, for example, basic facial expression information and basic personality information. The basic facial expression is, for example, information about the user's facial expression when they are calm. The basic personality information is information about the user's personality (for example, strong emotions, easily gets impatient, etc.). The past state information is information indicating the user's state in the past. The past state information includes, for example, information about the user's satisfaction when information was provided in the past, information about the user's level of impatience depending on the vehicle's past locations, and information about the user's regional understanding. The past state information may also include information about recommended points determined by the user in the past and information about changes in the user's state (emotions) before (or after) arriving at the recommended points. The schedule information includes information about the user's current and future plans (for example, travel, business trip, etc.) and free time. Each piece of information included in the user information 332 is acquired, for example, by the user performing a registration process in advance or answering inquiries such as a questionnaire. Furthermore, various pieces of information in the user information 332 may be updated depending on the results of an information provision process, which will be described later.

[0047] FIG. 3 is a diagram showing an example of the contents of the vehicle information 334. In the vehicle information 334, a vehicle ID and a user ID are associated with past driving history information of the vehicle. The driving history information includes the date and time of past driving, the route, and points (e.g., shops, facilities, tourist spots) that have been used (stopped at or visited) in the past. The driving history information is acquired, for example, from the in-vehicle device 100 or the terminal device 200. A single vehicle may be occupied by multiple users, and each vehicle may be driven by a different driver. Therefore, such information may be stored in the vehicle information 334 in a distinguishable manner. The driving history information may also include information such as a history of visits made by the user using a train or the like.

[0048] FIG. 4 is a diagram showing an example of the contents of the recommended point DB 336. The recommended point DB 336 associates, for example, identification information (e.g., a recommended point ID) for identifying a recommended point with summary information, location information, available time, usage waiting time information, rating information, and usage fee information of the recommended point. The summary information includes, for example, information about the genre of the recommended point (e.g., tourist spot, restaurant, public restroom, etc.) and the content (description) of the recommended point. The available time includes information about available times such as the opening hours of the recommended point. The usage waiting time information is, for example, information about the waiting time from arrival at a store until using the service (e.g., until eating or drinking at the store). The rating information is, for example, information about other users' ratings of the recommended point (e.g., word-of-mouth information). The usage fee information is information about the fee for using the recommended point. The usage waiting time information, rating information, and usage fee information may include a plurality of pieces of information, or may be the average, maximum, or minimum value of the plurality of pieces of information. The summary information and location information may be obtained, for example, from map information 338, and the available time, usage waiting time information, rating information, and usage fee information may be obtained, for example, by an external device via a network NW. The information in the recommended spot DB 336 may be updated appropriately at a predetermined timing. Note that the recommended spot DB 336 is only required to include at least one of the above-mentioned items consisting of the summary information, location information, available time information, usage waiting time information, rating information by other users, and usage fee information, and may also include other items.

[0049] The map information 338 stores, for example, information similar to that of the map information 192. The map information 338 may also store map information that covers a wider range and has higher resolution than the map information 192. The map information 338 may be acquired from an external device, and may be updated as needed by the server-side communication unit 310 communicating with the external device via the network NW.

[0050] 1, the server-side communication unit 310 is, for example, a wireless communication module for connecting to a network NW and communicating with the in-vehicle device 100 and the terminal device 200 via the network NW. For example, the server-side communication unit 310 communicates with the vehicle-side communication unit 110 of the in-vehicle device 100 and the terminal-side communication unit 210 of the terminal device 200. The server-side communication unit 310 performs wireless communication based on Wi-Fi, DSRC, Bluetooth, or other communication standards.

[0051] The server-side control unit 320 includes, for example, an acquisition unit 321, an estimation unit 322, a derivation unit 323, a determination unit 324, a search unit 325, and a provision unit 326. The acquisition unit 321 acquires user information about the user of the vehicle, image and audio information, and vehicle location information from the in-vehicle device 100 via the network NW. The acquisition unit 321 may also acquire location information of the in-vehicle device 100 via the terminal device 200, or acquire image information and audio information of the user captured by the terminal device 200. The acquisition unit 321 may also acquire vehicle information such as the vehicle speed and acceleration detected by the vehicle sensor 160 from the in-vehicle device 100 or the terminal device 200, or acquire request information for a recommended location. The acquisition unit 321 may also acquire user registration information from the in-vehicle device 100 or the terminal device 200 and register the information in the user information 332, or may acquire change information to update the user information 332. In this case, the acquisition unit 321 may provide inquiry information such as a questionnaire to the in-vehicle device 100 or the terminal device 200, and generate the user information 332 based on the response results. The acquisition unit 321 also acquires a driving history based on the vehicle's position information over time, and registers the acquired driving history in the vehicle information 334 in association with the vehicle ID and the user ID.

[0052] The estimation unit 322 estimates the state of the user from the information acquired by the acquisition unit 321. The state includes, for example, emotional information. The emotional information includes, for example, emotions related to "joy, anger, sadness, and happiness," as well as "satisfied," "impatient," and "surprised." The emotional information may also include an index value indicating the magnitude of the emotion (for example, emotional level) and information related to changes in the emotional level. For example, the estimation unit 322 estimates the emotional information based on information (image information and audio information of the user) acquired from at least one of the camera 130 and microphone 140 mounted on the vehicle, and the camera 230 and microphone mounted on the terminal device.

[0053] For example, the estimation unit 322 performs image analysis processing on the image captured by the camera 130 (or camera 230) using well-known techniques (such as binarization processing, contour extraction processing, image enhancement processing, feature extraction processing, and pattern matching processing) to recognize the user's facial expression and estimate the user's emotion based on the recognized expression. For example, the estimation unit 322 detects local feature points, such as the start of eyebrows, the tip of the nose, and both ends of the mouth, from the user's facial image included in the camera image. These feature point detection may use various algorithms, such as active shape models (ASM), active appearance models (AAM), constrained local models (CLM), and convolutional neural networks (CNN). The estimation unit 322 also recognizes changes in the user's facial expression using changes in the arrangement of the feature points and the distances and angles between the points.

[0054] The estimation unit 322 then estimates emotional information from the recognized facial expression. For example, if the user's facial expression is a smile, the estimation unit 322 estimates that the user's emotion is "joy" or "fun." Furthermore, if the user's facial expression is angry, the estimation unit 322 estimates that the user's emotion is "anger," and if the user's facial expression is crying, the estimation unit 322 estimates that the user's emotion is "sad." Furthermore, for example, if the user's facial expression is a smile or a satisfied face, the estimation unit 322 estimates that the user's emotion is "satisfaction," if the user's facial expression is restless, the estimation unit 322 estimates that the user's emotion is "impatience," and if the user's facial expression is surprised, the estimation unit 322 estimates that the user's emotion is "surprise."

[0055] Furthermore, instead of (or in addition to) image information (camera images), the estimation unit 322 performs voice analysis processing on voice information using a well-known method to acquire the user's tone of voice, speaking speed, and content of the voice (words uttered by the user), and estimates the user's emotion from the acquired content. As described above, the estimation unit 322 estimates emotion information based on at least one of the user's facial expression, the user's tone of voice, the user's speaking speed, and the content of the user's voice (words). Furthermore, the estimation unit 322 may inquire about the user's emotion via the in-vehicle device 100 or the terminal device 200 and estimate emotion information from the response.

[0056] The estimation unit 322 also estimates the magnitude of the estimated emotion (emotional level) based on, for example, the amount of change in the user's facial expression over a predetermined period of time. The emotion level includes, for example, "satisfaction level" and "impatience level." Regarding "impatience level," a state of not being impatient may be estimated as "comfort level."

[0057] For example, the estimation unit 322 may increase the emotion level as the amount of change in facial expression increases. In this case, the estimation unit 322 may refer to the basic state information of the user information 332 stored in the server-side storage unit 330 and adjust the emotion level based on the user's basic facial expression information and basic personality information included in the basic state information. For example, when a user whose basic facial expression (a neutral expression) is angry smiles, the amount of change in facial expression increases, so the estimation unit 322 increases the emotion level of "joy" compared to when the user changes from a neutral expression to a smile. Furthermore, when a user whose basic facial expression is smiling smiles more, the amount of change in facial expression is small, so the estimation unit 322 decreases the emotion level of "joy." Furthermore, when a user whose basic personality information indicates a "personality that gets angry easily" smiles, the estimation unit 322 increases the emotion level compared to when a user whose basic personality information indicates a "personality that laughs easily" smiles. This allows for a more appropriate emotion level to be estimated according to each user's basic facial expression and personality.

[0058] The estimation unit 322 may also estimate the amount of change in the emotion level over a predetermined time period. For example, the estimation unit 322 may compare the emotion before and after providing information to the user, and estimate whether the emotion (satisfaction level or impatience level) has increased (or decreased) after the information is provided, and the amount of change. The estimation unit 322 may estimate the user's emotion and emotion level from images and audio using deep learning such as CNN or RNN (Recurrent Neural Network).

[0059] Furthermore, the estimation unit 322 may estimate the user's understanding level for each region or the understanding level for the region where the user is currently located based on the user information 332, the vehicle information 334, the current vehicle position, and the like.

[0060] The information estimated by the estimation unit 322 is registered in the user information 332. For example, when the estimation unit 322 estimates emotion information (including the emotion level) after providing information such as recommended spots to the user, the estimation unit 322 may associate the recommended spots with the emotion information and store the associated information in the past state information of the user information 332. Furthermore, when the estimation unit 322 acquires time information or weather information when the recommended spot information is provided, the estimation unit 322 may also associate the information with the emotion level (satisfaction level) and store (update) the information in the past state information.

[0061] The derivation unit 323 derives the degree of deviation between the past state of the user and the current state of the user. For example, the derivation unit 323 derives the degree of deviation between emotional information (e.g., satisfaction level) of the user before providing the user with information such as recommended locations and emotional information (e.g., satisfaction level) of the user after the information is provided. In this case, the derivation unit 323 increases the degree of deviation, for example, as the difference (deviation) in satisfaction level increases. Furthermore, the derivation unit 323 may derive whether the satisfaction level has increased or decreased before and after the information is provided.

[0062] The determination unit 324 determines whether the proposal effectiveness of the recommended spot information is high based on the degree of deviation derived by the derivation unit 323. For example, the determination unit 324 determines that the proposal effectiveness of the recommended spot information is high when the degree of deviation is equal to or greater than a threshold. Furthermore, the determination unit 324 may determine that the proposal effectiveness of the recommended spot information is high, for example, when the satisfaction level increases before and after the information is provided. Also, the determination unit 324 may determine that the proposal effectiveness of the recommended spot information is low, for example, when the satisfaction level decreases before and after the information is provided. Also, the determination unit 324 may determine a search area when searching for recommended spots for the vehicle based on the vehicle's location information and emotion information.

[0063] The search unit 325 searches for recommended spots according to the state of the user, by referring to the recommended spot DB 336 and map information 338, based on information such as user information and location information of the in-vehicle device 100 and the terminal device 200. For example, the search unit 325 may search when there is a search request for recommended spots from the user, may search when the user is in a predetermined state, may search when the vehicle is located at a specific position, or may search at a predetermined interval.

[0064] For example, the search unit 325 searches for recommended spots from available information (recommended spot DB 336, map information 338) using search conditions corresponding to the content of the search request or the user's state (emotion), with an area within a predetermined distance from the vehicle's location as the search area. A search request may consist of a string of characters, such as "Looking for a restaurant that serves Chinese food," "I want to go to the restroom," "I want to buy a drink," or "I want to go to a nearby park." The search unit 325 extracts keywords (words, etc.) that serve as search conditions from the string of characters, and based on the extracted keywords, extracts recommended spots that have a high degree of similarity (above a threshold) by referring to summary information, rating information, etc., of the recommended spot DB 336. Furthermore, if the user requests "I want to go shopping," the search unit 325 may acquire preference information included in the personal information of the user information, and search for recommended spots where shopping can be done by referring to the recommended spot DB 336 using the information included in the preference information as keywords.

[0065] The search unit 325 may also adjust the search area depending on the user's emotional state. The search unit 325 may also weight items included in the available information according to predetermined conditions, and control the extraction of different recommended locations. The search unit 325 may also acquire weather information corresponding to the vehicle's location, and search for tourist spots from the available information if the weather is good, or search for restaurants from the available information if it is close to lunchtime. The search unit 325 may also perform a search based on the number of recommended locations extracted by the search or the user's schedule, or based on the user's level of regional understanding.

[0066] The search unit 325 may also refer to past state information included in the user information 332, extract recommended points where the emotional level (for example, satisfaction) is equal to or greater than a threshold under conditions similar to the weather information corresponding to the current time and vehicle location, and perform a re-search based on the genre and content of the extracted recommended points by referring to the recommended point DB 336. This allows for a more accurate search when searching for recommended points by referring to past recommended points extracted under conditions closer to the conditions (time and weather) at the time of the current search.

[0067] The providing unit 326 transmits information (recommended location information) about one or more recommended locations extracted by the search unit 325 to the in-vehicle device 100 or the terminal device 200 to provide (recommend) the user. The recommended location information includes, for each recommended location, at least one of summary information, location information, available time information, waiting time information, rating information by other users, and usage fee information. In addition to the recommended location information, the providing unit 326 may also provide information about search results (the number of extracted results) and information about search conditions. The provided information is at least one of image information and audio information, and this information may be generated by the providing unit 326. In addition, when the providing unit 326 acquires information about a recommended location determined by the user from the one or more pieces of recommended location information provided, the providing unit 326 may generate a route from the current location of the in-vehicle device 100 (or the terminal device 200) to the recommended location by referring to map information, and provide guidance information including the generated route information to the in-vehicle device 100 (or the terminal device 200).

[0068] [Information provided] Next, examples of information provision in the information provision system 1 of the first embodiment will be specifically described by dividing them into several patterns.

[0069] [First offering pattern] Fig. 5 is a sequence diagram showing the flow of processing in the information provision system according to the first provision pattern. The example of Fig. 5 shows an example of the provision processing between the in-vehicle device 100 and the server device 300, but the terminal device 200 may be used instead of the in-vehicle device 100, or both the in-vehicle device 100 and the terminal device 200 may be used. In the example of Fig. 5, it is assumed that information (user information) about a user who uses a vehicle equipped with the in-vehicle device 100 has already been registered in the server device 300. The same applies to the second and third provision patterns described below.

[0070] 5, the in-vehicle device 100 acquires vehicle position information (step S100). Next, the in-vehicle device 100 acquires image information captured by the camera 130 and voice information of the user input through the microphone 140 (steps S102 and S104). Next, the vehicle-side communication unit 110 of the in-vehicle device 100 transmits the position information, image information, and voice information to the server device 300 (step S106). Note that the in-vehicle device 100 may transmit vehicle information detected by the vehicle sensor 160 to the server device 300, or may acquire surrounding weather information and transmit it to the server device 300.

[0071] The estimation unit 322 of the server device 300 estimates the basic state of the user based on the image information and the audio information among the position information, image information, audio information, etc. from the in-vehicle device 100 acquired by the acquisition unit 321 (step S108). Next, the server device 300 stores the estimated basic state and various information received from the in-vehicle device 100 in the user information 332 and the vehicle information 334 of the server-side storage unit 330 (step S110). The processes of steps S100 to S110 described above may be repeatedly executed at predetermined intervals or other timings until the processes of step S112 and subsequent steps are executed.

[0072] Next, when the in-vehicle device 100 receives a request for a recommended location (search request) from the user (step S112), the in-vehicle device 100 transmits the received request to the server device 300 (step S114). The search unit 325 of the server device 300 searches for recommended locations by referring to the map information 338 based on the request for recommended locations received from the in-vehicle device 100, the vehicle's location information, and the user information (step S116). Note that in the process of step S116, recommended locations may be searched for when the vehicle is traveling through a predetermined location or at predetermined intervals, without receiving a request for recommended locations from the in-vehicle device 100. Next, the providing unit 326 of the server device 300 generates provision information (e.g., image information or audio information) for providing recommended location information including one or more recommended locations extracted by the search (step S118). Next, the providing unit 326 transmits the generated provision information to the in-vehicle device 100 (step S120).

[0073] Next, the in-vehicle device 100 receives the recommended spot information transmitted from the server device 300, outputs the received recommended spot information to the output unit 150 and provides it to the user (step S122), acquires image information of the user after the information has been provided by the camera 130 (step S124), and acquires collected audio information input by the microphone 140 (step S126). Next, the in-vehicle device 100 transmits the acquired image information and audio information to the server device 300 (step S128).

[0074] Next, the estimation unit 322 of the server device 300 estimates emotional information (e.g., satisfaction) based on the image information and voice information of the user after the information is provided (step S130), and estimates the satisfaction level (emotion level) according to a change in the estimated emotional information, "satisfaction" (step S132). Next, the search unit 325 of the server device 300 searches again for recommended locations based on the estimated satisfaction level (step S134). The processing of steps S132 and S134 will be described in detail later. Next, the providing unit 326 of the server device 300 generates information to be provided for providing recommended location information based on the re-search result (step S136), and transmits the generated information to be provided to the in-vehicle device 100 (step S138).

[0075] The in-vehicle device 100 receives the recommended location information transmitted from the server device 300, and causes the output unit 150 to output the received recommended location information and provide it to the user (step S140). The processes of steps S122 to S140 described above may be repeated until the satisfaction level in the process of step S132 becomes greater than a threshold value.

[0076] Next, the in-vehicle device 100 acquires information that a specific recommended point has been determined from one or more recommended points in response to a user instruction (step S142), transmits information about the determined recommended point to the server device 300 (step S144), generates a route from the current vehicle position to the recommended point and outputs it from the output unit 150, thereby providing route guidance to the determined recommended point (step S146). Note that in the example of FIG. 5, the route is generated by the in-vehicle device 100. However, instead, route guidance to the recommended point may be performed by acquiring a route generated by the providing unit 326 of the server device 300 and providing it to the user. Furthermore, the in-vehicle device 100 may acquire image information and audio information including the user during route guidance and transmit the contents of the information to the server device 300. This allows the server device 300 to estimate the user's emotions while heading toward or arriving at the recommended point.

[0077] Next, the server device 300 stores information about the recommended points determined by the user and acquired from the in-vehicle device 100 in the user information 332 (step S148). The in-vehicle device 100 also transmits the vehicle's position (travel history) at the time of route guidance to the server device 300 (step S150). The server device 300 associates the received travel history with the user information and stores it in the server-side storage unit 330 (step S152). This ends the processing of this sequence.

[0078] [Satisfaction estimation process, re-search process] Next, the satisfaction level estimation process in step S132 and the re-search process in step S134 described above will be described. FIG. 6 is a flowchart showing an example of the satisfaction level estimation process and the re-search process. In the example of FIG. 6, the estimation unit 322 acquires the past user satisfaction level (step S132A). The past user satisfaction level may be, for example, the satisfaction level before information about recommended locations is provided, or may be the user satisfaction level when information about recommended locations was previously provided (for example, the previous time information was provided). Next, the estimation unit 322 acquires the user satisfaction level after recommended locations are provided (step S132B).

[0079] Here, the acquisition of the satisfaction level will be described. First, the estimation unit 322 estimates that the user is in a satisfied state from the above-mentioned facial expression and voice. For example, if the user's expression after providing information is a smile or a satisfied face, the estimation unit 322 estimates that the user's emotion is "satisfied." Furthermore, the estimation unit 322 analyzes the user's voice and estimates that the user's emotion is "satisfied" if the user's words are "good," "happy," or the like. The estimation unit 322 may also estimate the user's emotion of "satisfied" based on the tone of voice or speaking speed. Next, the estimation unit 322 may estimate the satisfaction level based on, for example, the amount of change in facial expression or voice over a predetermined period of time in the "satisfied" state, or may acquire basic state information of the user from the user information 332 and estimate the satisfaction level based on the magnitude of the difference between the acquired basic state and the user's current state when satisfied.

[0080] Next, the derivation unit 323 derives the degree of discrepancy between the past user satisfaction level and the user satisfaction level after the provision (step S132C). Next, the determination unit 324 determines whether the degree of discrepancy is equal to or less than a threshold (step S132D). If it is determined that the degree of discrepancy is not equal to or less than the threshold, the determination unit 324 determines whether the satisfaction level after the provision has decreased compared to the past satisfaction level (step S132E).

[0081] If it is determined in the process of step S132D that the deviation is equal to or less than the threshold, or if it is determined in the process of step S132E that the satisfaction level after the provision has decreased compared to the previous satisfaction level, the search unit 325 searches for recommended locations again (step S132F). In this case, the search unit 325 may search using conditions different from the previous search conditions. For example, if the previous search conditions did not include user preference information as a condition, the search unit 325 performs a search by adding preference information included in the personal information of the user information 332 to the search conditions. Furthermore, if the previous search conditions searched for recommended locations within a first predetermined distance from the vehicle location, the search unit 325 may search for recommended locations within a range of a second predetermined distance that is greater than the first predetermined distance. Furthermore, the search unit 325 may refer to rating information in the recommended location DB 336 and perform a search so as to preferentially extract recommended locations that have been highly rated by other users. The search unit 325 may also refer to past state information included in the user information 332, extract recommended locations with satisfaction levels equal to or higher than a threshold under conditions similar to the weather information corresponding to the current time and vehicle location, and perform a re-search based on the genre and content of the extracted recommended locations by referring to the recommended location DB 336. By re-searching under different conditions in this way, different recommended locations can be provided to the user.

[0082] Furthermore, if it is determined in the process of step S132E that the satisfaction level after provision has not decreased compared to the previous satisfaction level, the process of this flowchart ends. Note that the re-search may be executed repeatedly until the satisfaction level reaches a threshold or more, or may be executed a predetermined number of times. According to the process described above, if there is little change in the user's satisfaction level before and after the provision of recommended points or if the satisfaction level has decreased, a re-search is performed to provide recommended points, thereby making it possible to provide more appropriate recommended point information according to the user's emotions.

[0083] According to the first provision pattern described above, it is possible to estimate the user's level of satisfaction with the recommended point based on the user's emotional state before and after the recommended point information is provided. Therefore, it is possible to grasp whether the user is satisfied with the proposed recommended point, and this can be utilized the next time a search or information is provided. This allows more appropriate information to be provided to the user, contributing to improving user satisfaction with information provision.

[0084] [Second provision pattern] Next, the second provision pattern will be described. In the second provision pattern, the level of impatience is extracted as an example of the user's emotional information, and the search area when searching for recommended spots is adjusted based on the extracted level of impatience. Fig. 7 is a sequence diagram showing the processing flow of the information provision system according to the second provision pattern. The processing of steps S200 to S206 shown in Fig. 7 is the same as the processing of steps S100 to S106 of the first provision pattern shown in Fig. 5 above, and therefore will not be described here.

[0085] The server device 300 receives and stores the information transmitted from the in-vehicle device 100 (step S208), and the estimation unit 322 estimates the user's current level of impatience based on the received information and user information (step S208). Next, the search unit 325 of the server device 300 adjusts a search area for the vehicle position based on the estimated level of impatience (step S212), and searches for recommended locations corresponding to the level of impatience within the adjusted search area (step S214). Note that in the second provision pattern, similar to the first provision pattern, recommended locations may be searched for including search conditions according to the search request when a request for recommended locations (search request) is received from the user. Next, the provision unit 326 of the server device 300 generates provision information for providing recommended location information including one or more recommended locations extracted by the search (step S216), and transmits the generated provision information to the in-vehicle device 100 (step S218).

[0086] Next, the in-vehicle device 100 receives the recommended point information transmitted from the server device 300 and provides the received information to the user by outputting it from the output unit 150 (step S230). Next, the in-vehicle device 100 acquires information that a specific recommended point has been determined from one or more recommended points in response to a user instruction (step S222), and transmits information indicating the determination to the server device 300 (step S224). The in-vehicle device 100 also generates a route from the current vehicle position to the determined recommended point and outputs the route from the output unit 150, thereby providing route guidance to the determined recommended point (step S226). Note that the route to the recommended point may be generated by the server device 300, received, and provided to the user, instead of being generated by the in-vehicle device 100. Furthermore, during route guidance, images including the user may be captured and audio recorded, and the content of the captured images and audio may be transmitted to the server device 300, allowing the server device 300 to determine the user's emotions while heading toward or arriving at the recommended point.

[0087] The server device 300 stores information about the recommended points determined by the user in the server-side storage unit 330 (step S228). Furthermore, the in-vehicle device 100 transmits driving history information during route guidance to the server device 300 (step S230). The server device 300 associates the driving history, recommended points, and user information with each other and stores them in the server-side storage unit 330 (step S230). This ends the processing of this sequence.

[0088] [Estimation of Impatience] Here, the estimation of the level of impatience will be described. The level of impatience is estimated from at least one of the user's facial expression, the user's tone of voice, the user's speaking speed, and the user's words. For example, the estimation unit 322 estimates the user's emotion as "impatient" when the user's facial expression is unsettled. The estimation unit 322 may also estimate the user's emotion as "impatient" when the user's facial expression is tense or the user is fidgeting, when the user's speaking speed is above a threshold, or when the user's words include words related to "impatient." The estimation unit 322 may also refer to schedule information included in the user information 332 and estimate the user's emotion as "impatient" based on the distance to the next scheduled location and the remaining time until the schedule. The estimation unit 322 may also increase the level of impatience as time passes since estimating the "impatient" state (emotion). The estimation unit 322 may also estimate the level of impatience based on the difference between the basic state information and the current facial expression, or based on the amount of change over a predetermined period of time.

[0089] Furthermore, instead of (or in addition to) the above-described estimation method, the estimation unit 322 may estimate the user's impatience and the level of impatience based on the acceleration and acceleration included in the vehicle information acquired from the in-vehicle device 100. In this case, the estimation unit 322 estimates that the user is impatient if the number of sudden accelerations or sudden decelerations within a predetermined period of time is equal to or greater than a predetermined number, and further estimates the level of impatience depending on the number of times within the predetermined period of time. This allows the user's emotions to be accurately estimated based on the user's driving operation information. Furthermore, the estimation unit 322 may estimate the user's impatience and the level of impatience based on, for example, the amount of sweat of the user (driver) measured by a sweat sensor of the vehicle sensor 160. For example, the estimation unit 322 estimates that the user is impatient if the amount of sweat is equal to or greater than a predetermined amount, and increases the level of impatience as the amount of sweat increases. Note that because the amount of sweat varies from person to person, the estimation unit 322 may adjust the level of impatience for each user based on statistical results (history information) for each user. This allows the user's emotions to be estimated with high accuracy based on changes in the user's physical condition (sweating state), etc. The estimation unit 322 may perform estimation by combining two or more of the above-mentioned various estimation methods. Information on the degree of impatience estimated by the estimation unit 322 is registered in the user information 332, for example.

[0090] [Adjust search area based on impatience] Next, the search area adjustment process performed in step S212 will be described in detail. FIG. 8 is a flowchart showing a first example of the search area adjustment process. In the example shown in FIG. 8, the search unit 325 determines whether the user's impatience level estimated in step S210 is equal to or greater than a threshold (step S212A). If it is determined that the user's impatience level is equal to or greater than the threshold, the search unit 325 acquires the vehicle's traveling direction based on changes in the vehicle's position information over time, and searches for recommended points in a search area that excludes the direction opposite to the acquired vehicle's traveling direction (reverse direction) (step S212B). That is, in the process of step S212B, the search for recommended points is limited to search areas in the vehicle's traveling direction among search areas within a predetermined distance based on the vehicle's position information. This reduces the driving burden on a user who is impatient and heading towards a recommended point. This contributes to user safety and improves user satisfaction with the information provided.

[0091] If it is determined in the process of step S212A that the degree of impatience is not equal to or greater than the threshold, the search unit 325 searches for recommended points around the vehicle (within a predetermined distance from the vehicle position), including in the direction of travel and in the opposite direction to the direction of travel (step S212C), which then ends the process of this flowchart.

[0092] FIG. 9 is a flowchart showing a second example of the search area adjustment process. In the example of FIG. 9, the search unit 325 determines whether the user's level of impatience estimated in the process of step S210 is equal to or greater than a threshold value (step S212A), as in the first example shown in FIG. 8. If it is determined that the level of impatience is equal to or greater than the threshold value, the search unit 325 searches for recommended locations without including pre-registered user preference information in the search conditions (step S212D). This makes it possible to provide more recommended locations with fewer search conditions when the user is impatient. Therefore, the user can make a satisfactory selection from a certain number of recommended location candidates. This improves the user's satisfaction with the information provided.

[0093] Furthermore, if it is determined in the process of step S212A that the degree of impatience is not equal to or greater than the threshold, the search unit 325 searches for recommended locations by including the user's preference information in the search conditions (step S212E). Including the user's preference information in the search conditions means, for example, including the user's preference information in the search conditions so that restaurants corresponding to the user's preference information (e.g., Chinese food) are searched for, or weighting the preference information more heavily so that recommended locations corresponding to the preference information are more likely to be extracted during a search. If the user is not in a hurry, searching for recommended locations including the preference information can provide more appropriate recommended locations that match the user's preferences. This ends the process of this flowchart.

[0094] In the second provision pattern, the search unit 325 may adjust the search conditions depending on the number of recommended locations extracted by the search and perform a re-search. For example, if the number of recommended locations extracted by the search is less than a predetermined number, the search conditions are adjusted so that at least a predetermined number of recommended locations are extracted. In this case, if the user's preference information was included in the previous search conditions, the search unit 325 performs a re-search without including the preference information in the search conditions. Furthermore, if the previous search conditions did not limit the search area to only the vehicle's traveling direction, the search unit 325 may perform a re-search by extending the search area in the traveling direction, or may perform a re-search in a search area that allows the vehicle to make a predetermined number of right and left turns in addition to the traveling direction. This makes it possible to provide the user with at least a certain number of recommended locations.

[0095] Furthermore, if the number of recommended locations extracted by the search is greater than the upper limit, the search unit 325 may adjust the search conditions so that the number of recommended locations is equal to or less than the upper limit. This prevents the user from being overwhelmed with the selection of recommended locations due to the number of recommended locations being too large.

[0096] According to the second provision pattern described above, the search area for recommended points is adjusted based on the user's emotional information (level of impatience) to search for recommended points, thereby providing more appropriate information in accordance with the user's emotional information. This reduces the driving burden on the user when heading to a recommended point in a hurry, and improves the user's satisfaction with the information provided.

[0097] In the second provision pattern described above, the emotion information may be the user's level of composure instead of the level of impatience, or other emotion information may be used. In the case of the level of composure, the emotion is the opposite of the level of impatience, so the determination using the threshold in the above-described search area adjustment process also becomes the opposite condition (whether the level of composure is less than the threshold).

[0098] [Third provision pattern] Next, a third provision pattern will be described. The third provision pattern estimates the user's understanding of a region (predetermined area) based on the user's past driving information, and provides recommended point information based on the user's understanding of the current location (region). Fig. 10 is a sequence diagram showing the processing flow of the information provision system according to the third provision pattern.

[0099] In the example of FIG. 10 , the in-vehicle device 100 acquires vehicle location information (step S300) and transmits the acquired location information to the server device 300 (step S302). The in-vehicle device 100 may also transmit vehicle information such as the vehicle's speed and acceleration, and weather information, along with the vehicle location information. The server device 300 receives the information transmitted from the in-vehicle device 100 and stores the received location information in the server-side storage unit 330 (step S304). Next, the estimation unit 322 of the server device 300 estimates the user's understanding of the area based on the driving history derived from the location information (step S306). Next, the server device 300 searches for recommended locations based on the understanding level (step S308), generates information to be provided for providing recommended location information including one or more recommended locations extracted by the search (step S310), and transmits the generated information to the in-vehicle device 100 (step S312). In the third provision pattern, similar to the first provision pattern, recommended locations may be searched for including search conditions according to the search request when a request for recommended locations (search request) is received from the user.

[0100] Next, the in-vehicle device 100 receives recommended point information from the server device 300 and provides the received recommended point information to the user by outputting it from the output unit 150 (step S314). Next, when the in-vehicle device 100 receives information that a recommended point has been determined from one or more recommended points in response to a user instruction (step S316), it transmits information about the determined recommended point to the server device 300 (step S318), generates a route from the current vehicle position to the recommended point, and outputs the route from the output unit 150, thereby providing route guidance to the determined recommended point (step S320). Note that instead of generating the route to the recommended point by the in-vehicle device 100, the route may be generated by the server device 300, received, and provided to the user. Furthermore, even during route guidance, images including the user may be captured and audio recorded, and the content of the captured images and audio may be transmitted to the server device 300, allowing the server device 300 to determine the user's emotions while heading toward or arriving at the recommended point.

[0101] The server device 300 stores information about the recommended points determined by the user in the server-side storage unit 330 (step S322). The in-vehicle device 100 also transmits driving history information during route guidance to the server device 300 (step S324). The server device 300 associates the driving history, recommended points, and user information with each other and stores them in the server-side storage unit 330 (step S326). This ends the processing of this sequence.

[0102] [Estimation of understanding] Here, the estimation of the level of understanding will be described. For example, the estimation unit 322 may refer to personal information included in the user information 332 stored in the server-side storage unit 330 and estimate that the closer an area is to the address or workplace included in the personal information, the higher the level of understanding of the user in that area. Alternatively, the estimation unit 322 may refer to vehicle information 334 stored in the server-side storage unit 330 and estimate the level of understanding of the area corresponding to the location information based on the location information of the driving history included in the vehicle information 334. In this case, the estimation unit 322 may determine that the greater the number of times the user has driven through the same area in a predetermined period of time, the higher the level of understanding of the user in that area. Alternatively, the estimation unit 322 may increase the level of understanding of the user in an area with a newer driving history. Alternatively, the estimation unit 322 may increase the level of understanding of the user in an area based on the greater number of times the user has actually visited recommended points within the same area. Alternatively, the estimation unit 322 may inquire about the level of understanding of the user by area and estimate the level of understanding by area based on the response. Information on the level of understanding estimated by the estimation unit 322 is registered in the user information 332, for example.

[0103] [Searching for recommended locations based on comprehension level] Next, the search process performed in step S308 described above will be described in detail. FIG. 11 is a flowchart showing an example of the search process in the third provision pattern. In the example of FIG. 11, the search unit 325 determines whether the level of understanding of the area based on the user's driving history is below a threshold (step S308A). The determination of whether the level of understanding is below the threshold is made, for example, based on the vehicle's driving information and vehicle location information, by determining whether the vehicle has driven through the same area (or a location within a predetermined distance) a predetermined number of times or more within a predetermined time period. This makes it possible to determine whether the user is a local based on the vehicle's driving information, thereby reducing the burden of input, etc., on the user.

[0104] If it is determined in the process of step S308A that the level of understanding of the area is less than the threshold, the search unit 325 searches for recommended locations based on the free time in the user's schedule (step S308B). For example, when searching for a restaurant as a recommended location, the search unit 325 searches for restaurants whose total time, including the vehicle travel time (including the travel time from the current location to the restaurant and the travel time from the restaurant to the next scheduled destination), waiting time, time for eating and drinking, etc., is shorter than the free time in the user's schedule. In this way, in the third provision pattern, if the level of understanding is less than the threshold, it is determined that the user is not a local user, and optimal information (recommendation of recommended locations) can be provided to a user who is in an unfamiliar area due to a set schedule condition such as a business trip.

[0105] Furthermore, if it is determined in the processing of step S308A that the level of familiarity with the area is not below the threshold, the search unit 325 searches for recommended points without including the user's schedule in the search criteria (step S308C). This ends the processing of this flowchart. For example, if the level of familiarity with the area is equal to or greater than the threshold, it can be determined that the user is a local user (familiar with nearby stores), and it is presumed that the user has know-how such as travel time to a store (including road congestion) and waiting time at the store. Therefore, by searching for and providing information on recommended points regardless of the schedule, the number of recommended points provided can be increased, and the user can decide on a store they want to go to from a wider range of store information.

[0106] Next, a specific description will be given of the recommended spot search process based on the user's free time, which corresponds to the process in step S308B. FIG. 12 is a flowchart showing an example of the recommended spot search process based on free time. In the example in FIG. 12, the search unit 325 determines whether the free time is less than a first predetermined time (step S308B-1). If it is determined that the free time is less than the first predetermined time, the search unit 325 weights the waiting time of the recommended spot more heavily than other items and searches (step S308B-2). Weighting the waiting time more heavily means, for example, preferentially extracting recommended spots with shorter waiting times through the search.

[0107] Furthermore, if it is determined in the process of step S308B-1 that the free time is not less than the first predetermined time, the search unit 325 determines whether the free time is equal to or greater than a second predetermined time (step S308B-3). If it is determined that the free time is equal to or greater than the second predetermined time, the search unit 325 weights the ratings of other users more heavily than the waiting time for use and searches for recommended locations (step S308B-4). This makes it possible to provide optimal information even when a user has a fixed schedule, such as a business trip, and has little free time until the next appointment.

[0108] Furthermore, if it is determined in the process of step S308B-3 that the step free time is not equal to or longer than the second predetermined time, the search unit 325 searches for recommended locations without setting weights (step S308B-5). This allows optimal information to be provided even when there is a lot of free time until the next appointment. This ends the process of this flowchart.

[0109] According to the third provision pattern described above, it is possible to estimate whether the user is a user from outside the area (whether the area is a regular route) based on the user's understanding of the area based on the vehicle's driving history. It is also possible to determine, for example, whether the user is lost in an unfamiliar place or a familiar place. If a user from outside the area is looking for a detour in an unfamiliar place, the system can suggest recommended locations by referring to the schedule. Furthermore, if a user is on a business trip or other unfamiliar location and wants to eat while prioritizing work, it is possible to provide restaurant information for an optimal schedule by weighting restaurants with short waiting times.

[0110] [Variations] Each of the first to third provision patterns described above may be processed in combination with at least a part of the other provision patterns. For example, when the user's impatience level is below a threshold as shown in the second provision pattern, recommended locations may be searched again until the user's satisfaction level reaches or exceeds the threshold as shown in the first provision pattern. In addition, in the first and second provision patterns, the search conditions may be varied depending on the user's understanding of the driving location as shown in the third provision pattern. In addition, in the first and third provision patterns, the search conditions may be adjusted depending on the user's emotional information as shown in the second provision pattern.

[0111] In the first embodiment, the in-vehicle device 100 or the server device 300 may select one of the first to third provision patterns according to the user's state (what state) depending on the vehicle situation, and provide information according to the selected provision pattern. In the first embodiment, one of the first to third provision patterns selected by a user's instruction may be executed.

[0112] Furthermore, in the first embodiment, if any of the processes of the first to third provision patterns is not performed, the server device 300 may not have a function corresponding to that process. Furthermore, in the first embodiment, if there are multiple occupants in the vehicle, the state (emotion) of each occupant may be estimated, and the search conditions may be adjusted based on the average value of the estimated states to extract recommended points, or recommended points may be extracted using search conditions that prioritize the driver.

[0113] [Second embodiment] In the information provision system 1 in the first embodiment described above, at least a part of the processing executed by the server device 300 may be executed by the in-vehicle device 100 or the terminal device 200. For example, by providing the functions of the server device 300 in the in-vehicle device 100, the in-vehicle device 100 does not need to communicate with the server device 300, and the in-vehicle device 100 can provide recommended spot information to the user. The above content will be described below as a second embodiment.

[0114] Fig. 13 is a configuration diagram of an in-vehicle device 100A in the second embodiment. In the example of Fig. 13, the same reference numerals are used to designate configurations and processes having the same or similar functions as those in the first embodiment, and redundant descriptions of those configurations and processes may be omitted. In the second embodiment, the in-vehicle device 100A is an example of an "information providing device."

[0115] 13 includes, for example, a vehicle-side communication unit 110, a vehicle-side short-range communication unit 120, a camera 130, a microphone 140, an output unit 150, a vehicle sensor 160, a navigation function unit 170, a vehicle-side control unit 180A, and a vehicle-side storage unit 190A. The vehicle-side device 100A of the second embodiment differs from the vehicle-side device 100 of the first embodiment in that it includes a vehicle-side control unit 180A and a vehicle-side storage unit 190A instead of the vehicle-side control unit 180 and the vehicle-side storage unit 190. The vehicle-side control unit 180A includes a position acquisition unit 181, a communication control unit 182, a display control unit 183, an acquisition unit 184, an estimation unit 185, a derivation unit 186, a determination unit 187, a search unit 188, and a provision unit 189. The acquisition unit 184, the estimation unit 185, the derivation unit 186, the judgment unit 187, the search unit 188, and the provision unit 189 perform the same functions as the acquisition unit 321, the estimation unit 322, the derivation unit 323, the judgment unit 324, the search unit 325, and the provision unit 326 of the server device 300 described above, but do not perform the parts related to communication between the server device 300 and the in-vehicle device 100.

[0116] The vehicle-side storage unit 190A also stores map information 192, user information 194, vehicle information 196, a recommended spot DB 198, programs, and various other information. The user information 194, vehicle information 196, and recommended spot DB 198 store items similar to those of the user information 332, vehicle information 334, and recommended spot DB 336 stored in the server-side storage unit 330 described above, but the user information 332 stores information only about users who ride in the vehicle in which the in-vehicle device 100A is installed, and the vehicle information 334 stores vehicle information only about the vehicle. The recommended spot DB 198 may be information similar to that of the recommended spot DB 336 obtained from an external device via the network NW.

[0117] As described above, in the second embodiment, the in-vehicle device 100A is provided with the functional configuration of the server device 300, so that the recommended spot information can be provided according to the first to third provision patterns described above without going through the server device 300. Also, in the second embodiment, communication with the terminal device 200 may be performed to acquire various types of information from the terminal device.

[0118] According to the second embodiment described above, in addition to achieving the same effects as the first embodiment, the in-vehicle device 100A can provide more appropriate information according to the user's situation without communicating with the server device 300.

[0119] [Third embodiment] Furthermore, by providing the functions of the server device 300 in the terminal device 200, the in-vehicle device 100 and the terminal device 200 do not need to communicate with the server device 300, and the terminal device 200 can provide recommended location information to the user. The above content will be described below as a third embodiment.

[0120] In the third embodiment, the terminal device 200 executes the information provision application 282, thereby enabling the terminal device 200 to execute the functions of the acquisition unit 321, the estimation unit 322, the derivation unit 323, the determination unit 324, the search unit 325, and the provision unit 326 of the server device 300. In addition to the information provision application 282 and the like, the terminal-side storage unit 280 stores map information 192, user information 194, vehicle information 196, and a recommended point DB 198.

[0121] In the third embodiment, the terminal device 200 communicates with the in-vehicle device 100 (or 100A) to acquire vehicle position information.

[0122] According to the third embodiment described above, in addition to achieving the same effects as the first embodiment, it is possible to provide more appropriate information according to the user's situation at the terminal device 200 without communicating with the server device 300.

[0123] Note that each of the first to third embodiments described above may be combined with a part of the other embodiments. For example, the function of the estimation unit 322 may be executed by the in-vehicle device 100 or the terminal device 200, and the function of the search unit 325 may be executed by the server device 300. Furthermore, when the server device 300 is providing information, if communication with the server device 300 becomes impossible due to a communication failure or the like, or if the vehicle is traveling at a speed higher than a predetermined speed or in a specific road environment such as a tunnel, and the communication conditions are likely to deteriorate, the in-vehicle device 100 or the terminal device 200 may switch to processing related to information provision. This allows for uninterrupted information provision according to the situation and more appropriate information provision. Furthermore, some or all of the functions of the information provision processing described in the first to third embodiments may be realized by AI (Artificial Intelligence) technology.

[0124] According to the embodiment described above, the information provision program causes a computer to acquire user information including emotional information of the vehicle user, location information of the vehicle, and available information including information about recommended points that can be provided to the user, determine a search area for the recommended points based on the location information of the vehicle and the emotional information, search for the recommended points based on the determined search area, and provide the user with information about the recommended points extracted by the search, thereby making it possible to provide more appropriate information to the user.

[0125] Specifically, according to the embodiment, for example, by adjusting the search area for recommended locations based on the user's emotional information (e.g., level of impatience, level of ease), it is possible to provide information on optimal recommended locations tailored to the user's emotional information. Providing optimal recommended locations tailored to the user's level of impatience can prevent the user from driving in a rushed state, contributing to safe driving. Furthermore, according to the embodiment, for example, when the user is rushing, it is possible to prevent the user's attention from being distracted by being presented with recommended locations that are far away or a large number of restaurant candidates. Therefore, it is possible to improve the user's satisfaction with the information provided.

[0126] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: acquiring user information including emotion information of a vehicle user, location information of the vehicle, and available information including information on recommended locations that can be provided to the user; determining a search area for the recommended location based on the vehicle location information and the emotion information; Searching for the recommended location based on the determined search area; providing the user with information about the recommended locations extracted by the search; Information providing device.

[0127] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0128] 1...information provision system, 100, 100A...in-vehicle device, 110...vehicle-side communication unit, 120...vehicle-side short-range communication unit, 130, 230...camera, 140, 240...microphone, 150, 250...output unit, 160...vehicle sensor, 170...navigation function unit, 180, 180A...vehicle-side control unit, 184, 321...acquisition unit, 185, 322...estimation unit, 186, 323 ...derivation unit, 187, 324...determination unit, 188, 325...search unit, 189, 326...providing unit, 190, 190A...vehicle-side storage unit, 200...terminal device, 210...terminal-side communication unit, 220...terminal-side short-range communication unit, 260...terminal-side control unit, 270...application execution unit, 280...terminal-side storage unit, 310...server-side communication unit, 320...server-side control unit, 330...server-side storage unit

Claims

1. On the computer, acquiring user information including emotion information of a user of a vehicle, location information of the vehicle, and provideable information including information on recommended locations that can be provided to the user; determining a search area for the recommended points based on the vehicle position information and the emotion information; Searching for the recommended points based on the determined search area; providing the user with information about the recommended points extracted by the search; Informational program.

2. when the degree of emotion of the user based on the emotion information of the user is equal to or greater than a threshold, the recommended point is searched for in the search area based on the position information of the vehicle, excluding an area in the opposite direction to the traveling direction of the vehicle. The information providing program according to claim 1 .

3. The search conditions for the recommended locations include preference information of the user. The information providing program according to claim 1 .

4. If the number of the recommended locations extracted by the search is less than a predetermined number, the recommended locations are searched again without including the user's preference information in the search conditions. The information providing program according to claim 3.

5. the emotion information includes information regarding the user's level of impatience, When the degree of impatience of the user is equal to or greater than a threshold, the recommended location is searched for without including preference information of the user in search conditions. The information providing program according to claim 3.

6. the emotion information includes a degree of impatience of the user, The degree of impatience is estimated from at least one of a facial expression of the user, a tone of voice of the user, a speaking speed of the user, and words uttered by the user. The information providing program according to claim 1 .

7. the emotion information includes a degree of impatience of the user, The degree of impatience is estimated based on information acquired from at least one of a camera and a microphone mounted on the vehicle, and a camera and a microphone mounted on a terminal device carried by the user. The information providing program according to claim 1 .

8. The computer acquiring user information including emotion information of a vehicle user, location information of the vehicle, and available information including information on recommended locations that can be provided to the user; determining a search area for the recommended location based on the vehicle location information and the emotion information; Searching for the recommended location based on the determined search area; providing the user with information about the recommended locations extracted by the search; Information provision method.

Images