Fragrance providing system and fragrance providing method

Abstract

To provide an "aroma providing system and aroma providing method" which can provide an aroma which meets user's intention on a system for providing the aroma to the user boarding a vehicle.SOLUTION: An aroma providing system includes: a user state detection section 1 for detecting a user state including at least one of voice and a visual line of a user boarding a vehicle; an interest object specification section 2 for specifying an object which the user shows interest in on the basis of the detected user state; and an aroma generation section 3 for generating the aroma related to the specified object by an aroma providing mechanism 21. By providing the aroma related to the object of interest, which is estimated from the user state, the probability that the aroma to be provided is matched with user's interest is made to become high. Thus, the aroma which meets user's intention much more can be provided.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an aroma providing system and an aroma providing method, and is particularly suitable for use in an aroma providing system and an aroma providing method that provide an aroma to a user in a vehicle. [Background technology]

[0002] Conventionally, there is known a system that provides a scent inside the vehicle cabin using a scent providing mechanism mounted on the vehicle, which detects the current location of the vehicle and provides a scent corresponding to the current location (see, for example, Patent Document 1). In the system described in Patent Document 1, when a destination or passing point appears on the display screen of the navigation device, a location-scent correspondence table is referenced, and a scent corresponding to the scent number set in correspondence with the location genre indicating the destination or passing point is released from the scent releasing device.

[0003] There is also a known system that recognizes a user's conversation and provides a scent related to the conversation (see, for example, Patent Document 2). In the system described in Patent Document 2, a service robot that provides a guidance service to a user according to a guidance menu responds to a voice uttered by the user, selects from a plurality of scent sources a scent source that corresponds to a keyword included in the utterance, and drives an olfactory display to generate the desired scent and present it to the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2001-349739 A [Patent Document 2] JP 2019-005883 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the systems of Patent Documents 1 and 2 simply provide a scent that corresponds to the current location or a keyword included in the utterance, and have the problem that a scent that is contrary to the user's intention may be provided.

[0006] The present invention has been made to solve such problems, and aims to provide a system that provides scents to users in vehicles that can provide scents that are more in line with the user's wishes. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention detects a user state including at least one of the voice and gaze of a user in a vehicle, identifies an object in which the user is interested based on the detected user state, and generates a fragrance associated with the identified object by an aroma providing mechanism. Effect of the Invention

[0008] According to the present invention configured as described above, a scent related to an object of interest inferred from the user's state is provided, thereby increasing the likelihood that the scent provided will match the user's interests, and making it possible to provide a scent that is more in line with the user's wishes. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of a functional configuration of an aroma providing system according to an embodiment of the present invention; [Diagram 2] FIG. 4 is a diagram illustrating an example of related information stored in a related information storage unit. [Diagram 3] 4 is a block diagram showing an example of a detailed functional configuration of a user state detection unit and an interest subject identification unit according to the first embodiment. FIG. [Figure 4] 13 is a block diagram showing an example of a detailed functional configuration of a user state detection unit and an interest subject identification unit according to the second embodiment. FIG. [Diagram 5]FIG. 13 is a block diagram showing an example of detailed functional configurations of a user state detection unit and an interest subject identification unit according to the third embodiment. [Figure 6] FIG. 13 is a block diagram showing an example of a detailed functional configuration of a user state detection unit and an interest subject identification unit according to the fourth embodiment. [Figure 7] FIG. 13 is a diagram showing the shooting range of a camera used in the fourth embodiment. [Figure 8] FIG. 13 is a block diagram showing an example of a detailed functional configuration of a user state detection unit and an interest subject identification unit according to the fifth embodiment. [Figure 9] 4 is a flowchart showing an example of the operation of the aroma providing system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An embodiment of the present invention will now be described with reference to the drawings. Fig. 1 is a block diagram showing an example of the functional configuration of an aroma providing system according to this embodiment. The aroma providing system of this embodiment is a system that provides an aroma to a user in a vehicle by an aroma providing mechanism 21 provided in an aroma providing device 20 mounted on the vehicle, and includes an in-vehicle device 10 and an aroma providing device 20 mounted on the vehicle. The in-vehicle device 10 includes, as its functional configuration, a user state detection unit 1, an interest target identification unit 2, and an aroma generation unit 3. The in-vehicle device 10 also includes, as a storage medium, a related information storage unit 4.

[0011] Each of the above-mentioned functional blocks 1 to 3 can be configured by any of hardware, DSP (Digital Signal Processor), and software. For example, when configured by software, each of the above-mentioned functional blocks 1 to 3 is actually configured with a CPU, RAM, ROM, etc. of a computer, and is realized by the operation of a program stored in a recording medium such as the RAM, ROM, hard disk, or semiconductor memory.

[0012] The user state detection unit 1 detects a user state including at least one of the user's voice and line of sight. With regard to voice, the user state detection unit 1 detects the user's speech. With regard to line of sight, the user state detection unit 1 detects the line of sight that the user is directing toward the outside of the vehicle. The user state detection unit 1 will be described in detail later.

[0013] The interest subject identifying unit 2 identifies an object in which the user is interested (hereinafter, may be referred to as an interest subject) based on the user state detected by the user state detecting unit 1. That is, the interest subject identifying unit 2 identifies an object in which the user is interested based on at least one of the user's speech voice and the line of sight toward the outside of the vehicle detected by the user state detecting unit 1. The interest subject identifying unit 2 will also be described in detail later.

[0014] The aroma generating unit 3 generates an aroma associated with the object of interest identified by the object of interest identifying unit 2 by means of the aroma providing mechanism 21 of the aroma providing device 20. The aroma providing device 20 is a so-called diffuser, and is fixedly or temporarily installed at a predetermined position inside the vehicle. For example, the aroma providing device 20 is configured to be able to set a plurality of containers for storing spray liquid, and is equipped with an aroma providing mechanism 21 that selectively turns the spray liquid stored in one of the set plurality of containers into a mist and sprays it as a mist. Note that spraying a mist based on the spray liquid corresponds to "providing an aroma to the vehicle user" in the claims.

[0015] The method by which fragrance providing device 20 sprays the mist may be any method, such as a method of generating and spraying mist by vibrating a liquid with ultrasonic waves, a method of generating and spraying mist using a mist nozzle with fine spray holes, a method of introducing air into a housing to suck up the spray liquid by the Venturi effect, and spraying the mist, etc. Furthermore, fragrance providing device 20 does not need to be a stationary device, and may be a portable mobile device that is brought into a car for use.

[0016] The fragrance generating unit 3 selects a fragrance associated with the object of interest identified by the object of interest identifying unit 2 based on the related information stored in the related information storage unit 4, selects a container for storing a spray liquid corresponding to the selected fragrance, and drives the fragrance providing mechanism 21. FIG. 2 is a diagram showing an example of related information stored in the related information storage unit 4. FIG. 2(a) shows an example of first related information indicating the correspondence between various objects and the categories to which they belong. FIG. 2(b) shows an example of second related information indicating the correspondence between the categories and types of fragrances.

[0017] The aroma generating unit 3 identifies a category to which the object of interest identified by the object of interest identifying unit 2 belongs, based on the first related information shown in FIG. 2(a). For example, if the object of interest identified by the object of interest identifying unit 2 is "cherry blossoms", the aroma generating unit 3 identifies the category "flowers" to which the cherry blossoms belong. Next, the aroma generating unit 3 identifies the type of aroma corresponding to the category identified based on the first related information, based on the second related information shown in FIG. 2(b). For example, if the identified category is "flowers", the aroma generating unit 3 identifies a floral aroma corresponding to the "flowers" category. Then, the aroma generating unit 3 selects a container that stores a spray liquid corresponding to a floral aroma and drives the aroma providing mechanism 21.

[0018] Note that the related information shown in FIG. 2 is an example and is not limited to this. For example, in FIG. 2(a), multiple categories may be set by further subdividing "flower", "ocean", etc. Similarly, in FIG. 2(b), multiple scent types may be set by further subdividing "forest", "ocean", etc. Also, while FIG. 2(b) shows an example in which categories and scent types correspond 1:1, the information may show an n:1 (n≧2) correspondence, as in FIG. 2(a).

[0019] The user state detection unit 1 and the interest subject identification unit 2 will be described in detail below. (First embodiment) Fig. 3 is a block diagram showing a detailed functional configuration example of the user state detection unit 1 and the interest subject identification unit 2 according to the first embodiment. As shown in Fig. 3, the user state detection unit 1 according to the first embodiment includes a speech voice detection unit 102 that processes a voice collected by a microphone 101. The interest subject identification unit 2 according to the first embodiment includes a keyword extraction unit 201, an interest estimation unit 202, an object identification unit 203, and an emotion keyword storage unit 204.

[0020] The microphone 101 is installed at a predetermined position inside the vehicle and collects voices inside the vehicle. The speech detection unit 102 detects the user's speech from the voice collected by the microphone 101. For example, the speech detection unit 102 performs A / D conversion on the voice collected by the microphone 101, removes environmental sounds such as road noise and audio sounds by a known method, and detects the voice of the speech section corresponding to the contents of the user's speech as the speech.

[0021] The keyword extraction unit 201 of the interest subject identification unit 2 recognizes the user's speech detected by the speech detection unit 102, and extracts keywords by natural language analysis of the speech recognition result. The keyword extraction unit 201 has a speech recognition dictionary, and performs speech recognition processing by comparing the speech data of the speech detected by the speech detection unit 102 with the speech recognition dictionary, and then extracts keywords by performing natural language analysis processing using the resulting text data.

[0022] Here, the keyword extraction unit 201 performs a known morpheme analysis to decompose text data corresponding to the user's speech into a plurality of morphemes, and extracts each of the plurality of morphemes as a keyword. A morpheme of a specific part of speech may be extracted as a keyword. The specific part of speech may be, for example, a noun, a verb, an adjective, or an adverb.

[0023] The interest estimation unit 202 estimates the user's interest in things, based on keywords related to things extracted by the keyword extraction unit 201 and keywords related to emotions extracted by the keyword extraction unit 201. For example, the interest estimation unit 202 recognizes noun keywords, among the keywords extracted by the keyword extraction unit 201, as keywords related to things. Furthermore, the interest estimation unit 202 recognizes keywords, among the keywords extracted by the keyword extraction unit 201, that are registered in advance in the emotion keyword storage unit 204 as keywords related to emotions.

[0024] Keywords that are empirically known or expected to be uttered when a person shows interest in an object are registered in advance in the emotion keyword storage unit 204. For example, words such as "Look," "Look," "What is this," "Amazing," "Wow," "Awful," "Beautiful," and "Feels good" are registered as emotion keywords in the emotion keyword storage unit 204. When an emotion keyword is included in the multiple keywords extracted by the keyword extraction unit 201, the interest estimation unit 202 estimates that the person is interested in an object included as a noun keyword in the multiple keywords extracted by the keyword extraction unit 201.

[0025] For example, the interest estimation unit 202 estimates that the user is interested in cherry blossoms based on the keyword of the thing "cherry blossoms" extracted by the keyword extraction unit 201 from the utterance content "Look, the cherry blossoms are blooming" and the emotion keyword "Look." Also, the interest estimation unit 202 estimates that the user is interested in flowers based on the keyword of the thing "flowers" extracted by the keyword extraction unit 201 from the utterance content "Some kind of beautiful flower" and the emotion keyword "Some kind of flower."

[0026] Here, the interest estimation unit 202 may perform a perfect match determination between the keyword extracted by the keyword extraction unit 201 and the emotion keyword registered in the emotion keyword storage unit 204, or may perform a similarity determination. The similarity determination can be performed using, for example, a thesaurus. That is, when the keyword extracted by the keyword extraction unit 201 is registered in the thesaurus as a synonym for the keyword registered in the emotion keyword storage unit 204, the extracted keyword is recognized as an emotion keyword.

[0027] When the interest estimation unit 202 estimates that the user is interested in an object, the object identification unit 203 identifies the estimated object as an object in which the user is interested. Then, the object identification unit 203 supplies word data indicating the identified object of interest to the scent generation unit 3.

[0028] In this way, according to the first embodiment, the user's object of interest can be inferred from the user's speech, and a scent corresponding to the object of interest can be provided to the user. If the speech does not include an emotion keyword, the aroma providing mechanism 21 is not driven and no scent is provided to the user. Therefore, a scent contrary to the user's intention is not provided based on casual speech, and only when it is estimated that the user is interested in an object, a scent related to that object can be provided to the user. Second embodiment 4 is a block diagram showing a detailed functional configuration example of the user state detection unit 1 and the interest subject identification unit 2A according to the second embodiment. In FIG. 4, the same reference numerals as those in FIG. 3 have the same functions, so that the duplicated explanation will be omitted here. As shown in FIG. 4, in the second embodiment, the interest subject identification unit 2A is used instead of the interest subject identification unit 2. The interest subject identification unit 2A according to the second embodiment includes a keyword extraction unit 211, an interest estimation unit 212, and an object identification unit 213.

[0029] The keyword extraction unit 211 of the interest target identification unit 2 recognizes the user's speech detected by the speech detection unit 102, and performs natural language analysis of the speech recognition result to extract keywords related to the object. The keyword extraction unit 211 has a speech recognition dictionary, and performs speech recognition processing by collating the speech data of the speech extracted by the speech detection unit 102 with the speech recognition dictionary, and then performs natural language analysis processing using the resulting text data to extract noun morphemes from multiple morphemes included in the text data as keywords related to the object.

[0030] The interest estimation unit 212 analyzes the prosodic features of the speech detected by the speech detection unit 102 to estimate the user's interest in the thing extracted by the keyword extraction unit 211. The prosodic features of the speech refer to features such as the strength or volume of the speech, length, pitch, intonation, and accent. For example, the interest estimation unit 202 estimates that the user is interested in the thing when features such as an increase in volume, a stronger pronunciation, a higher pitch, and a faster speed are observed in the part of the keyword related to the thing extracted by the keyword extraction unit 211. Alternatively, the interest estimation unit 202 may estimate that the user is interested in the thing extracted by the keyword extraction unit 211 from a speech that has a louder volume than the others among the multiple speeches detected by the speech detection unit 102.

[0031] As another example, an estimation model that has learned features related to at least one of the strength, volume, length, pitch, intonation, and accent of a speech voice may be used to estimate whether or not the user is interested in an object from the speech voice detected by the speech voice detection unit 102. The estimation model is a classifier that has been machine-learned to output an evaluation value indicating whether or not the user is interested in an object when prosodic information related to at least one of the strength, volume, length, pitch, intonation, and accent of a speech voice is input.

[0032] For example, by preparing multiple pieces of training data in advance, including prosodic information obtained by analyzing speech data of speech uttered when a person is interested in something, and prosodic information obtained by analyzing speech data of speech uttered when a person is not interested in something, and then assigning each piece a correct label indicating whether or not the user is interested, and then performing machine learning on the estimation model, it is possible to generate an estimation model that estimates whether or not the user is interested in something from the prosodic information of the speech.

[0033] In addition, by including a calculation layer that extracts prosodic information from speech data of utterances as features in the estimation model, it is possible to generate an estimation model that has been machine-learned to output an evaluation value indicating whether or not a user is interested in an object when speech data of utterances is input. In this case, multiple pieces of speech data of utterances uttered when a person is interested in an object and multiple pieces of speech data of utterances uttered when a person is not interested in an object are prepared in advance as learning data, and correct labels regarding the presence or absence of interest are assigned to each piece, and then machine learning of the estimation model is performed.

[0034] When the interest estimation unit 212 estimates that the user is interested in a thing (a thing extracted as a noun keyword by the keyword extraction unit 211), the object identification unit 213 identifies the estimated thing as an object in which the user is interested.

[0035] In this way, according to the second embodiment, the user's object of interest can be inferred from keywords included in the user's utterance and the prosodic features of the speech, and an aroma corresponding to the object of interest can be provided to the user. If it is not inferred from the prosodic features that the user is interested in an object, the aroma providing mechanism 21 is not driven and no aroma is presented to the user. Therefore, an aroma contrary to the user's intention is not presented based on casual utterance content, and only when it is inferred that the user is interested in an object, an aroma related to that object can be presented to the user.

[0036] Although only keywords related to objects are extracted from the user's speech content here, emotion keywords may also be extracted as in the first embodiment, and if emotion keywords are included and it is estimated based on prosodic features that the user is interested in the object, the object extracted as a keyword from the speech content may be specified as the object of interest of the user. In this way, the accuracy of estimating the object of interest can be increased. (Third embodiment) Fig. 5 is a block diagram showing a detailed functional configuration example of a user state detection unit 1B and a subject of interest identification unit 2B according to the third embodiment. As shown in Fig. 5, in the third embodiment, a user state detection unit 1B and a subject of interest identification unit 2B are used instead of the user state detection unit 1 and the subject of interest identification unit 2. The user state detection unit 1B according to the third embodiment includes a gaze detection unit 112 that processes an image captured by a camera 111. In addition, the subject of interest identification unit 2B according to the third embodiment includes a current position / direction detection unit 221, a map data acquisition unit 222, and an object identification unit 223.

[0037] The camera 111 is installed at a predetermined position in the vehicle and captures the face of a user who is in the vehicle. For example, the camera 111 is installed at a position where it can capture the entire area of ​​the vehicle or the area of ​​the front seats, and captures the faces of users who are seated in all seats or the front seats in the vehicle. Alternatively, the camera 111 may be installed at each seat and capture the faces of users who are seated in each seat.

[0038] The gaze detection unit 112 detects the user's gaze by analyzing the position of the pupil or iris based on the image captured by the camera 111. For example, the gaze detection unit 112 detects the relative direction of the user's gaze with respect to the reference direction, which is the forward direction (traveling direction) of the vehicle. The gaze direction detected here is the angle in the left-right direction on a horizontal plane. It is not necessary to detect the angle in the up-down direction based on the horizontal plane.

[0039] Here, by registering in advance information on the installation position of camera 111 (information that can identify the direction and the degree of deviation of camera 111 from the reference direction in front of the vehicle) and information indicating the expected position of the face of the user sitting in each seat, it is possible to identify the direction of the iris when the user's gaze is directed in the reference direction. Gaze detection unit 112 can detect the direction in which the user's gaze is directed by utilizing such a relationship between the iris direction and the reference direction.

[0040] In addition, if the user moves his / her head significantly, the user's eyes may not be included in the captured image. In this case, the gaze detection unit 112 may detect the direction of the user's head and detect the head direction as the gaze direction (regarding the head direction as the gaze direction).

[0041] The interest object identifying unit 2B identifies, as an object of interest to the user, an object that is estimated to be looked at by the user among objects outside the vehicle, based on the user's line of sight detected by the line of sight detecting unit 112 of the user state detecting unit 1B. In the third embodiment, an object that is estimated to be looked at by the user among objects outside the vehicle is identified by the current position / orientation detecting unit 221, the map data acquiring unit 222, and the object identifying unit 223.

[0042] The current position / direction detection unit 221 detects the current position and current direction of the vehicle. The current position / direction detection unit 221 includes, for example, a GPS (Global Positioning System) receiver, receives radio waves transmitted from a plurality of GPS satellites with a GPS antenna, and performs three-dimensional positioning processing or two-dimensional positioning processing to calculate the absolute position and absolute direction of the vehicle.

[0043] The current position / direction detection unit 221 may also include a self-contained navigation sensor using a gyro sensor, an acceleration sensor, and a magnetic direction sensor. The self-contained navigation sensor includes a distance sensor that detects the travel distance of the vehicle by outputting one pulse per predetermined travel distance, and an angular velocity sensor (relative direction sensor) such as a vibration gyro that detects the rotation angle (travel direction) of the vehicle. The self-contained navigation sensor detects the relative position and relative direction of the vehicle using these distance sensor and angular velocity sensor. The position calculation CPU then calculates the absolute vehicle position and vehicle direction based on the data of the relative position and direction of the vehicle output from the self-contained navigation sensor.

[0044] The map data acquisition unit 222 acquires map data from the map data storage unit 50. The map data storage unit 50 may be mounted on the vehicle or may be provided in a server device connected via a communication network. The map data may be the same as that used by a navigation device for route search and route guidance, and includes various drawing data required for map display, road data required for various processes such as map matching, route search, and route guidance, and facility data. The drawing data and facility data include data on a plurality of objects including natural objects such as the sea, mountains, and rivers, as well as man-made objects such as parks, tourist spots, buildings, stores, and local specialties, along with information indicating their locations.

[0045] The object identification unit 223 extracts an object in the user's line of sight on the map data acquired by the map data acquisition unit 222, based on the current position and current direction of the vehicle detected by the current position / direction detection unit 221 and the user's line of sight detected by the line of sight detection unit 112 of the user state detection unit 1B, and identifies the extracted object as an object in which the user is interested.

[0046] That is, the object identification unit 223 can identify the relative direction (gaze direction) of the user's gaze to the current orientation (absolute orientation) of the vehicle, which is the reference direction, based on the current position and orientation of the vehicle detected by the current position / orientation detection unit 221 and the user's gaze detected by the gaze detection unit 112 of the user state detection unit 1B, and can identify an object located in the gaze direction from the current position from the map data. Note that the object identification unit 223 identifies an object located within a predetermined distance from the current position of the vehicle as an object of interest to the user, taking into account the range that the user's gaze can reach.

[0047] It may be possible to extract from the map data a plurality of objects located in the direction of the user's line of sight and within a predetermined distance from the current vehicle position. In this case, the object identification unit 223 may, for example, identify the object located closest to the current vehicle position as the object of interest to the user. Alternatively, when the vehicle is traveling in an area where a plurality of facilities exist at a predetermined density or higher, the object located closest to the current vehicle position may be identified as the object of interest to the user, whereas when the vehicle is traveling in an area where a plurality of facilities exist at a density less than the predetermined density, the object located farthest from the current vehicle position may be identified as the object of interest to the user.

[0048] In this way, according to the third embodiment, it is possible to estimate an object of interest of a user from the user's line of sight, and provide the user with a scent corresponding to the object of interest. (Fourth embodiment) Fig. 6 is a block diagram showing a detailed functional configuration example of the user state detection unit 1B and the interest subject identification unit 2C according to the fourth embodiment. In Fig. 6, the same reference numerals as those shown in Fig. 5 have the same functions, so that the overlapping explanations will be omitted here. As shown in Fig. 6, in the fourth embodiment, the interest subject identification unit 2C is used instead of the interest subject identification unit 2B. The interest subject identification unit 2C according to the fourth embodiment includes an image analysis unit 232 and an object identification unit 233 that process an image captured by a camera 231.

[0049] In the fourth embodiment, similarly to the third embodiment, the interest object identifying unit 2C identifies, among objects outside the vehicle, an object that is estimated to be looked at by the user based on the user's gaze detected by the gaze detecting unit 112 of the user state detecting unit 1B, as an object of interest to the user. However, in the fourth embodiment, the object that is estimated to be looked at by the user among objects outside the vehicle is identified by the image analyzing unit 232 and the object identifying unit 233.

[0050] The camera 231 is installed at a predetermined position inside or outside the vehicle, and captures the outside of the vehicle. For example, as shown in Fig. 7, the front camera 231F captures a predetermined range in front of the vehicle, the right side camera 231R captures a predetermined range to the right of the vehicle, the left side camera 231L captures a predetermined range to the left of the vehicle, and the rear camera 231B captures a predetermined range behind the vehicle, capturing images in all directions around the vehicle. Hereinafter, when the camera 231 is written, it is assumed to include the cameras 231F, 231R, 231L, and 231B in each of these directions.

[0051] The image analysis unit 232 recognizes objects outside the vehicle and recognizes the relative direction of the objects from the vehicle by analyzing the captured image obtained by the camera 231 capturing the outside of the vehicle. The objects to be recognized here include natural objects such as the sea, mountains, and rivers, as well as man-made objects such as parks, tourist spots, buildings, and stores. However, it is not necessary to recognize all objects, and it is sufficient to recognize the objects recorded in the first related information shown in FIG. 2(a). Image recognition can be performed, for example, by pattern matching with a pattern image registered in a database in advance. Alternatively, objects included in the captured image may be recognized using a classification model that has been machine-learned to learn the features of the image.

[0052] The relative direction of an object from the vehicle can be recognized from information on which image of the cameras 231F, 231R, 231L, and 231B in each direction the object is captured in and information on the position of the object in the captured image. For example, it can be recognized that the relative direction from the vehicle of an object captured at the center position in the left-right direction of an image captured by the front camera 231F is the same as the traveling direction of the vehicle (reference direction). In addition, the relative direction from the vehicle of an object captured at a position shifted to the right or left from the center of the image captured by the front camera 231F can be specified from the relationship between the viewing angle of the front camera 231F and the position of the object in the captured image. The same applies when recognizing the relative direction of an object from an image captured by the cameras 231R, 231L, and 231B in other directions.

[0053] In addition, there may be cases where a plurality of objects recorded in the first related information are recognized from the captured image. In this case, the image analysis unit 232, for example, identifies the object closest to the vehicle as the object of interest of the user. Alternatively, when the vehicle is traveling in an area where a plurality of facilities exist at a predetermined density or more, the image analysis unit 232 may recognize the object closest to the vehicle, while when the vehicle is traveling in an area where a plurality of facilities exist at a density less than the predetermined density, the image analysis unit 232 may recognize the object farthest from the vehicle.

[0054] The object identification unit 233 identifies an object in the user's line of sight as an object of interest to the user, based on the relative direction of the object analyzed by the image analysis unit 232 and the user's line of sight detected by the line of sight detection unit 112 of the user state detection unit 1B. That is, the object identification unit 233 identifies an object in the user's line of sight detected by the line of sight detection unit 112 as an object of interest to the user, from among a plurality of objects recognized in a plurality of directions by the image analysis unit 232.

[0055] In this manner, also in the fourth embodiment, the object of interest of the user can be estimated from the user's line of sight, and a scent corresponding to the object of interest can be provided to the user. Fifth embodiment Fig. 8 is a block diagram showing a detailed functional configuration example of a user state detection unit 1D and an interest object identification unit 2D according to the fifth embodiment. In Fig. 8, the same reference numerals as those shown in Fig. 4 and Fig. 5 have the same functions, so duplicated explanations will be omitted here. The fragrance provision system according to the fifth embodiment is configured to identify an object of interest of a user by using both the user's speech and line of sight.

[0056] As shown in Fig. 8, in the fifth embodiment, the user state detection unit 1D includes the speech detection unit 102 shown in Fig. 4 and the gaze detection unit 112 shown in Fig. 5, and detects the user's speech and gaze. In the fifth embodiment, the interest subject identification unit 2D includes the keyword extraction unit 211 shown in Fig. 4, and also includes the current position / direction detection unit 221 and the map data acquisition unit 222 shown in Fig. 5. In addition, the interest subject identification unit 2D includes an object identification unit 243 instead of the object identification unit 223 shown in Fig. 5.

[0057] As described above, the keyword extraction unit 211 performs voice recognition on the speech detected by the speech detection unit 102 of the user state detection unit 1D, and extracts keywords related to things by natural language analysis of the results of the voice recognition.

[0058] 5, the object identification unit 243 has a function of identifying an object presumed to be looked at by the user based on the line of sight of the user detected by the line of sight detection unit 112 of the user state detection unit 1B, the current position and current orientation of the vehicle detected by the current position / orientation detection unit 221, and the map data acquired by the map data acquisition unit 222 (a function of extracting an object in the line of sight of the user on the map data). In the third embodiment shown in FIG. 5, the object extracted in this manner was identified as an object of interest to the user, but this is where the fifth embodiment differs from the third embodiment.

[0059] That is, in the fifth embodiment, when an object estimated to be viewed by the user based on the user's gaze detected by the gaze detection unit 112 matches an object extracted as a noun keyword from the spoken voice by the keyword extraction unit 211, the object identification unit 243 identifies the object as the user's object of interest. In this way, by identifying the user's object of interest using both the user's spoken voice and gaze, it is possible to increase the accuracy of estimating the object of interest.

[0060] Note that, here, an example has been described in which functions similar to the current position / orientation detection unit 221, the map data acquisition unit 222, and the object identification unit 223 in FIG. 5 are used as the function for identifying an object that is presumed to be looked at by the user. However, instead of this, functions similar to the image analysis unit 232 and the object identification unit 233 in FIG. 6 may be used.

[0061] In addition, in the above example, when a thing estimated to be viewed by the user based on the user's line of sight extracted by the keyword extraction unit 211 matches a thing extracted by the keyword extraction unit 211 as a noun keyword from a speech voice, the thing is identified as a target of interest of the user, but the interest estimation unit 212 may be further provided. That is, when a thing extracted by the keyword extraction unit 211 as a noun keyword from a speech voice and estimated to be of interest to the user by the interest estimation unit 212 matches a thing estimated to be viewed by the user based on the user's line of sight, the thing may be identified as a target of interest of the user. In addition, the keyword extraction unit 201 and the interest estimation unit 202 may be used instead of the keyword extraction unit 211 and the interest estimation unit 212.

[0062] Fig. 9 is a flowchart showing an example of the operation of the aroma providing system according to this embodiment configured as described above. The flowchart shown in Fig. 9 starts, for example, when a user activates a function of the aroma providing system. Note that, although an example of the operation of the user state detection unit 1, interest target identification unit 2, and aroma generation unit 3 shown as the first embodiment is shown here, the basic operations shown in the flowchart in Fig. 9 are similar in the second to fifth embodiments.

[0063] First, the user state detection unit 1 detects the user state (step S1). Next, the interest subject identification unit 2 identifies an object in which the user is interested, based on the user state detected by the user state detection unit 1 (step S2). The interest subject identification unit 2 then determines whether or not the user's interest subject has been identified (step S3), and if so, notifies the scent generation unit 3 of the interest subject. On the other hand, if the interest subject has not been identified, the process returns to step S1.

[0064] In the first embodiment, a case where an object of interest cannot be identified is, for example, when the user's speech does not contain an emotion keyword stored in the emotion keyword storage unit 204. Alternatively, even if the user's speech contains an emotion keyword, a case where an object of interest cannot be identified is also when a thing extracted as a noun keyword from the speech is not included in the first related information shown in FIG. 2(a).

[0065] If the object of interest identifying unit 2 has identified the object of interest of the user, the aroma generating unit 3 is notified of the object of interest and causes the aroma providing mechanism 21 to generate an aroma related to the object of interest (step S4). Then, the process of the flowchart shown in Fig. 9 ends. Note that, although this flowchart describes an example in which the process ends when one aroma is generated, the process may return to step S1 after the process of step S4 is performed, and the processes of steps S1 to S4 may be repeated until the function of the aroma providing system is turned off.

[0066] As described above in detail, in this embodiment, a user state including at least one of the voice and the line of sight of a user in a vehicle is detected, and an object in which the user is interested is identified based on the detected user state, and a scent related to the identified object is generated. According to this embodiment configured in this manner, a scent related to the object of interest estimated from the user state is provided, so that the scent provided is more likely to match the user's interest, and a scent that is more in line with the user's intentions can be provided.

[0067] In the above embodiment, an example was shown in which the in-vehicle device 10 is equipped with user state detection units 1, 1B and interest target identification units 2, 2A, 2B, 2C, 2D, and all related processing is performed by the in-vehicle device 10, but some or all of the processing may be performed in cooperation with a server device connected to the in-vehicle device 10 via a communication network.

[0068] In addition, the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited thereby. In other words, the present invention can be implemented in various forms without departing from the gist or main characteristics thereof. [Explanation of symbols]

[0069] 1,1B User status detection section 2, 2A, 2B, 2C, 2D Part of interest identification 3. Fragrance generating unit 21 Fragrance provision mechanism 102 Speech detection unit 112 Line of Sight Detection Unit 201,211 Keyword extraction section 202,212 Interest Estimation Department 203,213,223,233,243 Object Identification Section 221 Current position and direction detection unit 222 Map data acquisition unit 232 Image Analysis Unit

Claims

1. A system for providing an aroma to a user in a vehicle by an aroma providing mechanism mounted in the vehicle, comprising: a user state detection unit for detecting a speech voice of the user; an interest subject identification unit that identifies an object in which the user is interested based on the speech detected by the user state detection unit; an aroma generating unit that generates an aroma related to the object identified by the interest object identifying unit using the aroma providing mechanism, The interest identification unit a keyword extraction unit that performs speech recognition on the speech detected by the user state detection unit and performs natural language analysis on the speech recognition result to extract keywords; an interest estimation unit that estimates a user's interest in the article based on the keywords related to the subject extracted by the keyword extraction unit and the keywords related to the emotion extracted by the keyword extraction unit; and an object specifying unit that specifies the above-mentioned article as an object in which the user is interested when the interest estimating unit estimates that the user is interested in the above-mentioned article. A fragrance providing system comprising:

2. A system for providing a fragrance to a user in a vehicle by a fragrance providing mechanism mounted in the vehicle, comprising: a user state detection unit for detecting a speech voice of the user; an interest subject identification unit that identifies an object in which the user is interested based on the speech detected by the user state detection unit; an aroma generating unit that generates an aroma related to the object identified by the interest object identifying unit using the aroma providing mechanism, The interest identification unit a keyword extraction unit that performs speech recognition on the speech detected by the user state detection unit and performs natural language analysis on the speech recognition result to extract keywords related to things; an interest estimation unit that estimates a user's interest in the article by analyzing a prosodic feature of the speech detected by the user state detection unit; and an object specifying unit that specifies the above-mentioned article as an object in which the user is interested when the interest estimating unit estimates that the user is interested in the above-mentioned article. A fragrance providing system comprising:

3. The interest estimation unit further estimates the user's interest in the article by analyzing prosodic features of the speech detected by the user state detection unit; The object identification unit identifies the above-mentioned article as an object in which the user is interested when it is estimated that the user is interested in the above-mentioned article based on the emotion-related keyword extracted by the keyword extraction unit and when it is estimated that the user is interested in the above-mentioned article based on the prosodic features of the speech voice.

2. The aroma delivery system according to claim 1 .

4. A system for providing a fragrance to a user in a vehicle by a fragrance providing mechanism mounted in the vehicle, comprising: A user state detection unit that detects the line of sight of the user; an interest target identification unit that identifies an object that is estimated to be looked at by the user among objects outside the vehicle based on the line of sight of the user detected by the user state detection unit as an object in which the user is showing interest; an aroma generating unit that generates an aroma related to the object identified by the interest object identifying unit using the aroma providing mechanism, The interest identification unit a current position and direction detection unit for detecting a current position and a current direction of the vehicle; a map data acquisition unit that acquires map data including information indicating the locations of a plurality of objects; and an object identification unit that extracts an object that is in the line of sight of the user on the map data acquired by the map data acquisition unit and that is within a predetermined distance from the current position of the vehicle, based on the current position and current direction of the vehicle detected by the current position / direction detection unit and the line of sight of the user detected by the user state detection unit, and identifies the extracted object as an object in which the user is interested. A fragrance providing system comprising:

5. A system for providing a fragrance to a user in a vehicle by a fragrance providing mechanism mounted in the vehicle, comprising: A user state detection unit that detects the line of sight of the user; an interest target identification unit that identifies an object that is estimated to be looked at by the user among objects outside the vehicle based on the line of sight of the user detected by the user state detection unit as an object in which the user is showing interest; an aroma generating unit that generates an aroma related to the object identified by the interest object identifying unit using the aroma providing mechanism, The interest identification unit an image analysis unit that recognizes objects outside the vehicle by analyzing a captured image obtained by capturing an image of the outside of the vehicle, and recognizes a relative direction of the objects from the vehicle; and an object specifying unit that specifies an object that is in the line of sight of the user and is closest to the vehicle, as an object in which the user is interested, based on the relative direction of the object analyzed by the image analysis unit and the line of sight of the user detected by the user state detection unit. A fragrance providing system comprising:

6. A method for providing an aroma to a user in a vehicle by an aroma providing mechanism mounted in the vehicle, comprising: A first step in which a user state detection unit of a computer detects a speech voice of the user; A second step of an interest object identifying unit of the computer identifying an object in which the user is interested based on the speech detected by the user state detecting unit; and a third step of causing an aroma generating unit of the computer to generate an aroma associated with the object identified by the object of interest identifying unit, by the aroma providing mechanism; In the second step, the subject of interest identification unit a keyword extraction process for performing speech recognition on the speech detected by the user state detection unit and performing natural language analysis on the speech recognition result to extract keywords; an interest estimation process for estimating a user's interest in the article based on the keywords related to the subject extracted by the keyword extraction process and the keywords related to the emotion extracted by the keyword extraction process; When it is estimated by the interest estimation process that the user is interested in the above-mentioned article, an object identification process is executed to identify the above-mentioned article as an object in which the user is interested. A method for providing an aroma, comprising:

7. A method for providing a fragrance to a user in a vehicle by a fragrance providing mechanism mounted in the vehicle, comprising: A first step in which a user state detection unit of a computer detects a speech voice of the user; A second step of an interest object identifying unit of the computer identifying an object in which the user is interested based on the speech detected by the user state detecting unit; and a third step of causing an aroma generating unit of the computer to generate an aroma associated with the object identified by the object of interest identifying unit, by the aroma providing mechanism; In the second step, the subject of interest identification unit a keyword extraction process for performing speech recognition on the speech detected by the user state detection unit, and performing natural language analysis on the speech recognition result to extract keywords related to things; an interest estimation process for estimating a user's interest in the article by analyzing prosodic features of the speech detected by the user state detection unit; When it is estimated by the interest estimation process that the user is interested in the above-mentioned article, an object identification process is executed to identify the above-mentioned article as an object in which the user is interested. A method for providing an aroma, comprising:

8. A method for providing a fragrance to a user in a vehicle by a fragrance providing mechanism mounted in the vehicle, comprising: A first step in which a user state detection unit of a computer detects a line of sight of the user; A second step in which an interest target identification unit of the computer identifies, based on the line of sight of the user detected by the user state detection unit, an object that is estimated to be looked at by the user among objects outside the vehicle, as an object in which the user is showing interest; and a third step of causing an aroma generating unit of the computer to generate an aroma associated with the object identified by the object of interest identifying unit, by the aroma providing mechanism; In the second step, the subject of interest identification unit A current position and direction detection unit process for detecting a current position and a current direction of the vehicle; A map data acquisition process for acquiring map data including information indicating the locations of a plurality of objects; and extracting an object that is in the line of sight of the user and within a predetermined distance from the current position of the vehicle on the map data acquired by the map data acquisition process based on the current position and current direction of the vehicle detected by the current position / direction detection process and the line of sight of the user detected by the user state detection unit, and identifying the extracted object as an object in which the user is interested. A method for providing an aroma, comprising:

9. A method for providing a fragrance to a user in a vehicle by a fragrance providing mechanism mounted in the vehicle, comprising: A first step in which a user state detection unit of a computer detects a line of sight of the user; A second step in which an interest target identification unit of the computer identifies, based on the line of sight of the user detected by the user state detection unit, an object that is estimated to be looked at by the user among objects outside the vehicle, as an object in which the user is showing interest; and a third step of causing an aroma generating unit of the computer to generate an aroma associated with the object identified by the object of interest identifying unit, by the aroma providing mechanism; In the second step, the subject of interest identification unit an image analysis process for analyzing a captured image obtained by photographing the outside of the vehicle to recognize objects outside the vehicle and recognize the relative directions of the objects from the vehicle; and executing an object identification process for identifying an object that is in the line of sight of the user and is closest to the vehicle as an object in which the user is interested, based on the relative direction of the object analyzed by the image analysis process and the line of sight of the user detected by the user state detection unit. A method for providing an aroma, comprising:

Images