Emotion-related information collection system

Abstract

To provide an information collection system which allow for grasping a large amount of information on emotion of a user regarding in-vehicle functions in a timely and quantitative manner.SOLUTION: In an information collection system 1 provided herein, a vehicle 20 is equipped with a camera ECU 43 configured to estimate emotion from expression of a user, and a central ECU 30 configured to acquire a result of estimation by the camera ECU 43 as emotion information and transmit the acquired emotion information and services provided by ECUs 45, 47 to a server 10.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information collection system, and more particularly to an emotion-related information collection system that collects information related to a user's emotion (emotion-related information) acquired by a plurality of vehicles and stores it in a server. [Background technology]

[0002] Traditionally, in vehicle design and development, the biggest concern has been to understand whether users are satisfied with the functions (such as the provision of services) that the company has designed and developed and installed in the vehicle, and to understand which functions users are dissatisfied with.

[0003] However, satisfaction or dissatisfaction is a matter that the user feels internally, and unlike data that can be measured by sensors, etc., it is difficult to quantify it, so there is a problem in that it is difficult to objectively grasp whether or not a user is satisfied with the services, etc. provided.

[0004] In this regard, although not related to vehicles, for example, Patent Document 1 discloses a product supply system that includes a terminal on the customer side that purchases a product and a server of a company that provides the product, in which the server has a receiving function unit that receives request information input from the customer's terminal and a calculation means that extracts module information corresponding to the request information received by the receiving function unit from each piece of information stored in a storage means and combines the extracted module information to generate product model information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2010-238233 A Summary of the Invention [Problem to be solved by the invention]

[0006] According to the above-mentioned Patent Document 1, processing is performed based on request information input from the customer's terminal, making it possible to quantitatively (objectively) grasp whether the customer (user) is satisfied with the functions of the product they are currently using.

[0007] However, in the technology of Patent Document 1, since there are quite a few users who find it bothersome to input the required information from a terminal, there is a possibility that the information (data) will not be collected as expected. For example, in the case of a vehicle, even if users are asked to respond via a mobile device such as a smartphone whether or not they are satisfied with the functions installed in the vehicle, not all users will necessarily respond, and there is a risk that only a small amount of data will be collected.

[0008] Furthermore, even if it is possible to collect a large amount of data, if the data is collected late (for example, if the data is collected after a decision has been made to discontinue a current function and install a new one), it may be difficult to effectively utilize the data that has been collected.

[0009] The present invention has been made in consideration of the above points, and its purpose is to provide an information collection system that is capable of grasping a large amount of information related to a user's feelings toward in-vehicle functions in a timely and quantitative manner. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, the emotion-related information collection system of the present invention estimates emotions from facial expressions of users in a plurality of vehicles, and transmits information about the executed function and emotion information corresponding to the function to a server.

[0011] Specifically, the present invention is directed to an emotion-related information collection system that collects information related to a user's emotion acquired in a plurality of vehicles into a server.

[0012] In the emotion-related information collection system, the server includes a transmission / reception means for transmitting and receiving information to and from the plurality of vehicles, and each of the vehicles includes emotion estimation means for estimating an emotion from a user's facial expression, a function execution means for executing various functions in the vehicle, and a control means configured to be capable of transmitting and receiving information to and from the transmission / reception means and to be capable of acquiring an estimation result of the emotion estimation means as emotion information, and the control means controls the server to receive and transmit information regarding the function executed by the function execution means and , the above feeling and configured to transmit the information to the server. and the control means is configured, when receiving a designation request for designating a specific function from the server, to acquire, as the emotion information, an estimation result of the emotion estimation means before and after execution of the specific function by the function execution means, and to transmit information related to the specific function and the acquired emotion information to the server. It is characterized by the above.

[0013] In the present invention, the "various functions in the vehicle" include the provision of various services to the user.

[0014] According to this configuration, the emotion estimation means automatically estimates emotions such as satisfaction or dissatisfaction from the user's facial expression, so that a large amount of emotion information of users in a plurality of vehicles can be obtained.

[0015] Thus, when various functions are executed by the function executing means while driving, the control means transmits information regarding the executed function and emotion information corresponding to the function to the server. Therefore, even if the user does not spontaneously respond, it is possible to obtain a large amount of information about what emotions the user has in response to what functions in a timely manner.

[0016] Therefore, it is possible to quantitatively grasp user emotions such as satisfaction or dissatisfaction with various functions, which is desired to be understood in the design and development of a vehicle, as emotion-related information related to the user's emotions (information about the executed function and emotion information corresponding to the function). By quantitatively analyzing the emotion-related information collected in this way in the server in the design department, development department, etc., it is possible to use it for example to improve existing functions and develop new functions.

[0018] In this configuration, when a design department, development department, etc. wants to check the user's emotions regarding a specific function, the server transmits a request to specify the specific function to the control means of multiple vehicles. Then, when the control means receives such a request from the server, when the specific function is executed at an appropriate timing, it acquires emotion information before and after the execution of the specific function, and transmits information related to the specific function and the acquired emotion information to the server.

[0019] Here, even if the emotion information after the execution of a specified specific function is "dissatisfied," it is not clear whether the user's dissatisfaction is with that specific function, but if the emotion information before the execution of a specific function is "satisfied" and the emotion information after execution is "dissatisfied," it is possible to confirm that the user is dissatisfied with the specified specific function. Similarly, for example, if the emotion information before the execution of a specific function is "dissatisfied" and the emotion information after execution is "satisfied," it is possible to confirm that the user is satisfied with the specified specific function.

[0020] By transmitting user emotional information before and after the execution of such a specific function from multiple vehicles, the data is accumulated on the server, and this data can be used by design, development, and other departments to further improve the specific function.

[0021] As another solution, the server includes a transmission / reception means for transmitting and receiving information to and from a plurality of vehicles, and each of the vehicles includes an emotion estimation means for estimating an emotion from a user's facial expression, a function execution means for executing various functions in the vehicle, and a control means configured to be capable of transmitting and receiving information to and from the transmission / reception means and to be capable of acquiring an estimation result of the emotion estimation means as emotion information, and the control means is configured to transmit information related to the function executed by the function execution means and the emotion information to the server, The control means is configured, when the emotion estimation means estimates that the user's emotion is dissatisfaction, to store, as history information, a type of function executed by the function execution means immediately before the emotion estimation means estimates dissatisfaction, in association with a time when the function was executed, and to transmit the stored history information to the server, and the server may be configured to identify the function with which the user is dissatisfied based on the received history information.

[0022] According to this configuration, when the control means infers that the user's emotion is dissatisfaction, the control means stores the type of function executed immediately before as historical information in association with the time when the function was executed, thereby making it possible to quantitatively grasp, for example, the frequency with which the user was dissatisfied with a particular function.

[0023] The server then identifies the functions about which the user is dissatisfied based on the received historical information, and this information can be used by design departments, development departments, etc. to help improve those functions. Effect of the Invention

[0024] As described above, the emotion-related information collection system of the present invention makes it possible to grasp a large amount of information regarding a user's emotions, such as satisfaction or dissatisfaction with in-vehicle functions, in a timely and quantitative manner. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a diagram illustrating an outline of an emotion-related information collecting system according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a block diagram illustrating a system for collecting emotion-related information. [Diagram 3] 11 is a flowchart showing an example of information collection control by the emotion-related information collection system. [Figure 4] 10 is a flowchart showing an example of information collection control by the emotion-related information collection system according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0027] (Embodiment 1) Fig. 1 is a diagram showing a schematic overview of an emotion-related information collection system 1 according to this embodiment. As shown in Fig. 1, this emotion-related information collection system 1 collects information related to a user's emotion (emotion-related information) acquired from a plurality of vehicles 20A, 20B, 20C, 20D, 20E, 20F, ... in a server 10 installed outside the vehicle 20 (for example, a research laboratory or a development facility) (see the two-dot chain line arrow in Fig. 1), and uses the collected emotion-related information for, for example, improving existing functions of the vehicle 20 or developing new functions for the vehicle 20.

[0028] -System configuration- Fig. 2 is a block diagram showing a schematic diagram of the emotion-related information collection system 1. For ease of understanding, Fig. 2 shows only one vehicle 20 among a plurality of vehicles 20A, 20B, 20C, 20D, 20E, 20F, .... As shown in Fig. 2, the emotion-related information collection system 1 includes a server 10 and a vehicle 20.

[0029] The server 10 includes a transmitting unit 11 , a receiving unit 13 , a function designating unit 15 , a data storage unit 17 , and a data analyzing unit 19 .

[0030] The transmitting unit 11 is configured to transmit information, requests, commands, etc. from the server 10 to the multiple vehicles 20, while the receiving unit 13 is configured to receive information, etc. from the multiple vehicles 20. Therefore, in relation to the claims, the transmitting unit 11 and the receiving unit 13 of this embodiment correspond to the "transmitting / receiving means for transmitting and receiving information to and from the multiple vehicles" of the present invention.

[0031] The function designation unit 15 is configured to generate designation requests that designate specific functions to be executed in the multiple vehicles 20 and designation requests that designate specific functions to be added to the multiple vehicles 20, based on commands input via input means (e.g., a keyboard, etc.) not shown. The designation requests that designate specific functions, generated by the function designation unit 15, are transmitted by the transmission unit 11 to the central ECUs 30 of the multiple vehicles 20.

[0032] The data storage unit 17 is configured to store data etc. from the multiple vehicles 20 received by the receiving unit 13. The data analysis unit 19 is configured to analyze trends based on the data stored in the data storage unit 17 by applying known technology such as artificial intelligence (AI) and to identify the types of functions etc. that the user is dissatisfied with.

[0033] On the other hand, the vehicle 20 includes a central ECU 30, a DCM (Data Communication Module) 40, a camera 41, a camera ECU 43, and ECUs 45 and 47, as shown in FIG.

[0034] The camera 41 is disposed in a position in the vehicle 20 where it can capture an image of the user's face. The camera ECU 43 is configured to perform image processing on image data captured by the camera 41, and has a feeling estimation unit 43a. The feeling estimation unit 43a is configured to perform image analysis processing on the image data of the user captured by the camera 41, and estimate the user's "three feelings (dissatisfied, satisfied, other)" based on the user's facial expression in the image. Therefore, in relation to the claims, the camera 41 of this embodiment and the feeling estimation unit 43a of the camera ECU 43 correspond to the "feeling estimation means for estimating feelings from the user's facial expression" of the present invention.

[0035] The ECUs (Electronic Control Units) 45 and 47 are configured to execute various functions in the vehicle 20. Examples of the various functions include a service that informs users that popular restaurants and entertainment facilities are near the vehicle 20, a service that enables users of the vehicles 20A, 20B, 20C, 20D, 20E, 20F, ... to communicate with each other using a social networking service (SNS), a service that identifies a user and reads emails addressed to the user, a service that automatically determines the user's field of interest and distributes content, and a service that conveys information such as emergency information such as earthquake alerts and vehicle warning information. The ECUs 45 and 47 share the provision of these services and transmit signals representing the executed services to the central ECU 30. Therefore, in relation to the claims, the ECUs 45 and 47 in this embodiment correspond to the "function executing means for executing various functions in the vehicle" in the present invention.

[0036] The central ECU 30 includes a transmitting unit 31 , a receiving unit 33 , a data linking unit 35 , a data storage unit 37 , and a clock 39 .

[0037] The transmitting unit 31 is configured to transmit information from the vehicle 20 to the server 10, while the receiving unit 33 is configured to receive information, requests, commands, etc. from the server 10. The transmitting unit 31 and the receiving unit 33 are configured to communicate with the transmitting unit 11 and the receiving unit 13 of the server 10 by wireless communication via a network using the DCM 40. Therefore, in relation to the claims, the central ECU 30 having the transmitting unit 31 and the receiving unit 33 corresponds to the "control means configured to be able to transmit and receive information to and from the transmitting and receiving means" of the present invention.

[0038] The data linking unit 35 is configured to create historical information that associates (links) the type of service provided by the ECUs 45, 47 (such as the service name or a symbol representing it) with the time measured by the clock 39.

[0039] The data storage unit 37 is configured to store the types of services executed by the ECUs 45 and 47, and history information created by the data linking unit 35. The data storage unit 37 is also configured to store the estimation results (the three values ​​of the user's emotions) estimated by the emotion estimation unit 43a of the camera ECU 43 as emotion information. Therefore, in relation to the claims, the central ECU 30 having the data storage unit 37 corresponds to the "control means configured to be able to acquire the estimation results of the emotion estimation means as emotion information" according to the present invention.

[0040] -Information collection control- In the design and development of a vehicle, the greatest concern is to understand whether users are satisfied with the functions (such as the provision of services) that the company has designed and developed and that are installed in the vehicle 20, and to understand which functions the users are dissatisfied with.

[0041] Therefore, in the emotion-related information collecting system 1 of the present embodiment, the central ECU 30 is configured to transmit to the server 10 information related to the services (functions) provided by the ECUs 45, 47 and emotion information corresponding to the services.

[0042] Specifically, when a request for specifying the provision of a specific service is received from the server 10, the central ECU 30 is configured to obtain, as emotion information, the estimation results of the emotion estimation unit 43a of the camera ECU 43 before and after the provision of the specific service by the ECUs 45 and 47, and to transmit information related to the specific service and the obtained emotion information to the server 10.

[0043] More specifically, for example, in a design department or a development department, when a user wants to check the user's feelings toward a specific service, the specific service is input to the server 10 via an input means, and the function designation unit 15 of the server 10 generates a designation request that designates the specific service to be executed (or added) in the multiple vehicles 20. When the designation request is generated by the function designation unit 15 in this manner, the designation request is transmitted from the transmission unit 11 of the server 10 to the reception unit 33 of the central ECU 30 of the multiple vehicles 20A, 20B, 20C, 20D, 20E, 20F, ...

[0044] Then, when the specified specific service is provided at an appropriate timing upon receiving the designation request from the server 10, the central ECU 30 acquires the estimation results of the emotion estimation unit 43a before and after the provision of the specific service from the camera ECU 43, and stores the estimation results as emotion information in the data storage unit 37. In this manner, since the emotion estimation unit 43a automatically estimates emotions such as satisfaction and dissatisfaction from the facial expressions of the users in the multiple vehicles 20A, 20B, 20C, 20D, 20E, 20F, ..., it is possible to acquire a large amount of emotion information of the users in the multiple vehicles 20A, 20B, 20C, 20D, 20E, 20F, ....

[0045] Here, the reason why emotion information before and after the provision of a service is acquired is as follows. That is, even if the emotion information after the provision of a specified specific service is "dissatisfied", it is not clear whether the user's dissatisfaction is with the specified specific service, but if the emotion information before the provision of a specified specific service is "satisfied" and the emotion information after the provision is "dissatisfied", it is possible to confirm that the user is dissatisfied with the specified specific service. Similarly, if the emotion information before the provision of a specified specific service is "dissatisfied" and the emotion information after the provision is "satisfied", it is possible to confirm that the user is satisfied with the specified specific service.

[0046] Then, the central ECU 30 transmits the information on the provided specific service and the emotion information before and after the provision of the service, which are stored in the data storage unit 37, from the transmission unit 31 to the reception unit 13 of the server 10 by wireless communication. This makes it possible to obtain a large amount of information on the emotions of the user toward the specified specific service in a timely manner. In other words, it becomes possible to obtain a large amount of user emotions, such as satisfaction or dissatisfaction toward a specific service, which is desired to be understood in the design department, development department, etc., in a timely and quantitative manner.

[0047] In this way, the user's emotional information regarding a specific service collected by the server 10 (stored in the data storage unit 17 of the server 10) can be quantitatively analyzed in the design department, development department, etc., and the information can be used to improve existing functions and develop new functions, for example.

[0048] -Control Flow- Next, an example of information collection control by the emotion-related information collection system 1 of the present embodiment will be described with reference to the flowchart shown in FIG.

[0049] First, in step S1, the central ECU 30 judges whether or not a request for specifying the provision of a specific service has been received from the server 10. If the judgment in step S1 is NO, the process ends. On the other hand, if the judgment in step S1 is YES, that is, if a request for specifying the provision of a specific service has been received, the process proceeds to step S2.

[0050] In the next step S2, the central ECU 30 adds the specified function (provision of a specific service), and then proceeds to step S3. Specifically, when a predetermined condition is satisfied, the central ECU 30 causes the ECUs 45 and 47 to register the specified function so as to provide the specified specific service. Note that, when the specified function has already been registered in the ECUs 45 and 47, step S2 can be omitted.

[0051] In the next step S3, the central ECU 30 judges whether or not the specified function has been activated, i.e., whether or not the specified specific service has actually been provided, based on the signals from the ECUs 45 and 47. The judgment in this step S3 is repeated until the judgment becomes YES, and when the judgment becomes YES, the process proceeds to step S4.

[0052] In the next step S4, the data storage unit 37 of the central ECU 30 stores the activated function (the provided service) and the emotion information estimated by the emotion estimation unit 43a from the user's facial expression before and after the service is provided, and then the process proceeds to step S5.

[0053] In the next step S5, the central ECU 30 determines whether or not there is a read request from the server 10. The determination in step S5 is repeated until the result is YES, and when the result is YES, the process proceeds to step S6.

[0054] In the next step S6, the central ECU 30 transmits the emotion-related information (information about the provided service and the user's emotion information before and after the provision of the service) stored in the data storage unit 37 to the server 10, and then ends the process.

[0055] (Embodiment 2) This embodiment differs from the above-described first embodiment in that emotion-related information is acquired without waiting for a designation request from the server 10, and that the emotion-related information is stored as history information. The following description will focus on the differences from the first embodiment.

[0056] -Information collection control- In the above-mentioned first embodiment, emotion-related information is acquired when a specific service for which the design department, development department, etc., wants to check the user's emotions is specified. However, when the user is dissatisfied with services other than the specific service, it is necessary to understand this.

[0057] Therefore, in the emotion-related information collection system 1 of the present embodiment, when the emotion estimation unit 43a of the camera ECU 43 estimates that the user's emotion is dissatisfaction, the central ECU 30 is configured to store in the data storage unit 37 the type of service provided by the ECUs 45, 47 immediately before the emotion estimation unit 43a estimates the dissatisfaction as history information associated with the time when the service was provided and to transmit the stored history information to the server 10. The server 10 is also configured to identify a tendency for the user to be dissatisfied based on the received history information.

[0058] More specifically, when the emotion estimation unit 43a of the camera ECU 43 estimates dissatisfaction as the emotion of the user, the data linking unit 35 of the central ECU 30 creates history information that associates (links) the type of the service provided immediately before with the time when the service was provided, and the data storage unit 37 stores the history information. This makes it possible to quantitatively grasp, for example, the frequency with which the user was dissatisfied with a certain service during a month by referring to the history information.

[0059] The data analysis unit 19 of the server 10 then applies known technologies, such as artificial intelligence, to analyze trends based on the received historical information and identify functions that users are dissatisfied with, which can be used by design departments, development departments, etc. to improve various functions.

[0060] -Control Flow- Next, an example of information collection control by the emotion-related information collection system 1 of this embodiment will be described with reference to the flowchart shown in FIG.

[0061] First, in step S1, the central ECU 30 judges whether or not dissatisfaction has been estimated as the user's emotion by the emotion estimation unit 43a of the camera ECU 43. If the judgment in step S1 is NO, the process ends. On the other hand, if the judgment in step S1 is YES, that is, if dissatisfaction has been estimated as the user's emotion, the process proceeds to step S2.

[0062] In the next step S2, the central ECU 30 identifies the function (such as the type of service provided) executed by the ECUs 45 and 47 immediately before the emotion estimation unit 43a estimated dissatisfaction based on the signals from the ECUs 45 and 47, and then proceeds to step S3.

[0063] In the next step S3, the data linking unit 35 creates historical information that associates (links) the functions executed by the ECUs 45, 47 (such as the type of service provided) with the time measured by the clock 39, and the data memory unit 37 stores this historical information, after which the process proceeds to step S4.

[0064] In the next step S4, the central ECU 30 determines whether or not there is a read request from the server 10. If the determination in step S4 is NO, the process ends. On the other hand, if the determination in step S4 is YES, that is, if there is a read request from the server 10, the process proceeds to step S5.

[0065] In the next step S5, the central ECU 30 transmits the history information stored in the data storage unit 37 to the receiving unit 13 of the server 10, and then the process proceeds to step S6.

[0066] In the next step S6, the data analysis unit 19 of the server 10 analyzes trends based on the historical information received by the receiving unit 13 and stored in the data storage unit 17, identifies the functions (types of services) that the user is dissatisfied with, and then ends.

[0067] (Other embodiments) The present invention is not limited to the embodiments, and can be embodied in various other forms without departing from the spirit or main characteristics thereof.

[0068] In the above embodiment 1, since it is desired to actively check the user's feelings toward a specific service, the data collected in the server 10 is manually analyzed in a design department, development department, or the like. However, this is not limited to the above, and the data may be analyzed by, for example, the data analysis unit 19 of the server 10.

[0069] In addition, in the above embodiment 2, the data collected in the server 10 is analyzed by the data analysis unit 19. However, this is not limited to the above, and the data stored in the data storage unit 17 may be analyzed manually, for example, in a design department or a development department.

[0070] As such, the above-described embodiment is merely illustrative in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention. [Industrial Applicability]

[0071] According to the present invention, it is possible to grasp a large amount of information related to a user's emotions, such as satisfaction or dissatisfaction with in-vehicle functions, in a timely and quantitative manner, and therefore it is extremely useful to apply this invention to an emotion-related information collection system that collects information related to a user's emotions in a server. [Explanation of symbols]

[0072] 1 Emotion-related information collection system 10 Server 11 Transmission unit (transmission and reception means) 13 Receiving unit (transmitting / receiving means) 20 Vehicles 30 Central ECU (control means) 41 Camera (Means of Emotion Estimation) 43a Emotion estimation unit (emotion estimation means) 45 ECU (function execution means) 47 ECU (function execution means)

Claims

1. An emotion-related information collection system that collects information related to a user's emotion acquired in a plurality of vehicles in a server, comprising: The server includes a transmitting / receiving means for transmitting and receiving information between the server and the plurality of vehicles, The above vehicles are Emotion estimation means for estimating an emotion from a facial expression of a user; A function execution means for executing various functions in a vehicle; a control means configured to be able to transmit and receive information to and from the transmission / reception means and to be able to acquire the estimation result of the emotion estimation means as emotion information; the control means is configured to transmit information relating to the function executed by the function execution means and the emotion information to the server; the control means is configured, when receiving a designation request designating a specific function from the server, to acquire, as the emotion information, estimation results of the emotion estimating means before and after execution of the specific function by the function executing means, and to transmit information related to the specific function and the acquired emotion information to the server.

2. An emotion-related information collection system for collecting information related to a user's emotion acquired in a plurality of vehicles in a server, comprising: The server includes a transmitting / receiving means for transmitting and receiving information between the server and the plurality of vehicles, The above vehicles are Emotion estimation means for estimating an emotion from a facial expression of a user; A function execution means for executing various functions in a vehicle; a control means configured to be able to transmit and receive information to and from the transmission / reception means and to be able to acquire the estimation result of the emotion estimation means as emotion information; the control means is configured to transmit information relating to the function executed by the function execution means and the emotion information to the server; the control means is configured to, when the feeling estimation means estimates dissatisfaction as the user's feeling, store a type of function executed by the function execution means immediately before the feeling estimation means estimates dissatisfaction as history information in association with a time when the function was executed, and to transmit the stored history information to the server; The emotion-related information collecting system is characterized in that the server is configured to identify a function for which the user feels dissatisfied based on the received history information.

Images